Self-adaptive calibration method and device of projection touch system

Abstract

The invention relates to the field of projection image calibration, in particular to a self-adaptive calibration method and device of a projection touch system. The method comprises the following steps: identifying pre-projection image information, carrying out projection area layout design, and outputting a pattern layout projection effect; performing pattern positioning calculation on the pattern layout projection effect one by one, and generating a sub-pixel positioning coordinate of each pattern; calculating a central point of a projection area according to the sub-pixel positioning coordinates, and constructing an actual projection positioning coordinate system; theoretical position deviation calculation is carried out according to the actual projection positioning coordinate system, and projection pixel point position deviation of each pattern is extracted; and performing multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation to obtain a geometric transformation compensation result. The pattern position precision of the projection touch control image is improved, and the visual effect and precision of the projection pattern are optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of projecting image calibration, in particular to a self-adaptive calibration method and device of a projection touch system. BACKGROUND

[0002] With the continuous development of human-computer interaction technology and spatial computing devices, projection touch systems, as a new interactive form integrating visual display and touch input, are widely used in smart classrooms, intelligent meetings, exhibition displays, virtual simulation, smart homes and other application scenarios. Such systems project image content onto physical surfaces (such as desktops, walls, floors, etc.), and combine infrared, cameras or depth sensors to realize user touch operations, making it possible to achieve a natural and intuitive interactive experience. Especially in the context of growing demand for multi-user collaborative operation, large-screen information display, and immersive teaching, the functional complexity and performance requirements of projection touch systems continue to improve. However, due to differences in the position, shape, material and lighting environment of the projection surface, how to ensure accurate projection of content and accurate response to touch input has become one of the key technical challenges for stable operation of the system.

[0003] In practical applications, the position, scale and geometric shape of the projected image often need to be dynamically adjusted according to different projection surfaces and environments to ensure the integrity of the projected image and the consistency of the touch area. This adjustment process involves pattern arrangement, geometric alignment, coordinate mapping and other steps, and is influenced by a combination of factors such as the installation angle of the projector, lens distortion, projection surface curvature and external obstructions. Traditional projection systems rely on static parameter configuration or manual calibration to complete pattern calibration and touch mapping, such as manually dragging control points or relying on calibration grid patterns for geometric alignment. While this method may have some effect in static or standardized environments, it often faces issues such as low calibration accuracy, high maintenance costs and low efficiency in dynamic, complex environments or large-scale irregular projection surfaces, making it difficult to meet the needs of modern intelligent interactive scenarios for "self-adaptive, high-precision, low-latency" calibration. SUMMARY

[0004] To solve the above technical problems, the present application provides a self-adaptive calibration method and device for a projection touch system to solve at least one of the above technical problems.

[0005] To achieve the above purpose, the present application provides a self-adaptive calibration method for a projection touch system, comprising the following steps: Step S1: identifying pre-projected image information, designing projection area layout, and outputting pattern layout projection effect; Step S2: performing individual pattern positioning calculation on the pattern layout projection effect to generate sub-pixel positioning coordinates for each pattern; Step S3: Calculate the projection area center point according to the sub-pixel positioning coordinates, and construct an actual projection positioning coordinate system; Step S4: Calculate the theoretical position deviation according to the actual projection positioning coordinate system, and extract the projection pixel point position deviation of each pattern; Step S5: Perform multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation to obtain a geometric transformation compensation result.

[0006] In the present specification, an adaptive calibration device of a projection touch system is provided for performing the adaptive calibration method of the projection touch system as described above, comprising: A projection layout module for identifying pre-projection image information, performing projection area layout design, and outputting pattern layout projection effect; A positioning calculation module for performing pattern-by-pattern positioning calculation on the pattern layout projection effect to generate sub-pixel positioning coordinates of each pattern; A coordinate system construction module for calculating the projection area center point according to the sub-pixel positioning coordinates and constructing an actual projection positioning coordinate system; A deviation calculation module for calculating the theoretical position deviation according to the actual projection positioning coordinate system and extracting the projection pixel point position deviation of each pattern; A calibration compensation module for performing multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation to obtain a geometric transformation compensation result.

[0007] The beneficial effects of the present application are as follows: by identifying the preset pattern image information and planning the projection area, the system can flexibly generate the pattern layout most suitable for the current projection geometry according to different projection scenes (such as desktop, wall, arc surface, etc.). In this process, factors such as edge margin, pattern spacing, and visual range obstruction are considered to avoid the risk of pattern overlap, obstruction, or failure in dead zones. The accurate identification of pre-projection image information also ensures that the subsequent patterns can be spread in the most optimal distribution manner to fill the effective projection area, improving the coverage rate and space utilization of the patterns. By performing image recognition and sub-pixel level center point calculation on each projection pattern, the system can effectively improve the positioning accuracy and significantly reduce the positioning error caused by the resolution limit of image acquisition. Using sub-pixel technology such as pattern fitting, centroid method, or Gaussian interpolation, the positioning accuracy is better than traditional pixel-level positioning, and the average error can be controlled within 0.1 pixels. By calculating the spatial reference center of the entire pattern set through the center of pattern distribution or the center point of the smallest bounding box, the system can define a local coordinate system that fits the actual projection scene. This coordinate system is not dependent on the theoretical design drawing, but is dynamically generated based on the actual projection imaging effect, and has stronger environmental adaptability. By comparing the position difference of the pattern in the theoretical design coordinate and the actual recognition coordinate, the system can accurately extract the offset (pixel level or sub-pixel level) of each pattern to form a complete deviation vector set. This difference information reveals various error sources in the current projection state, including geometric distortion, position drift, lens deviation, surface unevenness, etc. Through global affine transformation, linear errors such as projection offset and angle tilt are eliminated; then nonlinear modeling techniques (such as B-spline or local polynomial fitting) are applied in local areas to correct regional distortion problems; finally, pixel-level point-by-point fine tuning is performed to complete the final fine correction, so that the pattern coincidence degree reaches the sub-pixel level. Compared with single linear transformation, this method has stronger error repair capability and scene adaptability, especially in complex application environments such as non-standard plane projection, uneven surface curvature, and distortion caused by aging of projection equipment, and shows superior robustness. BRIEF DESCRIPTION OF DRAWINGS

[0008] Figure 1 FIG. 1 is a schematic diagram of the step flow of the adaptive calibration method of the projection touch system of the present application; Figure 2 FIG. 2 is a schematic diagram of the detailed implementation step flow of step S1; Figure 3 FIG. 3 is a schematic diagram of the detailed implementation step flow of step S2; Figure 4 FIG. 4 is a schematic diagram of the detailed implementation step flow of step S3. DETAILED DESCRIPTION

[0009] It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0010] The application example provides an adaptive calibration method and device of a projection touch system. The execution subject of the adaptive calibration method and device of the projection touch system includes but is not limited to the following: mechanical equipment, a data processing platform, a cloud server node, a network upload device and the like, which can be regarded as a general computing node of the application. The data processing platform includes but is not limited to the following: an audio image management system, an information management system, a cloud data management system and at least one of the like.

[0011] Please refer to Figures 1 to 4 The application provides an adaptive calibration method of a projection touch system, which includes the following steps: Step S1: identifying pre-projection image information, performing projection area layout design, and outputting a pattern layout projection effect; Step S2: performing individual pattern positioning calculation on the pattern layout projection effect, and generating sub-pixel positioning coordinates of each pattern; Step S3: calculating a projection area center point according to the sub-pixel positioning coordinates, and constructing an actual projection positioning coordinate system; Step S4: performing theoretical position deviation calculation according to the actual projection positioning coordinate system, and extracting a projection pixel point position deviation of each pattern; Step S5: performing multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation, to obtain a geometric transformation compensation result.

[0012] In the embodiment of the application, please refer to Figure 1 The application provides an adaptive calibration method of a projection touch system, which includes the following steps: Step S1: identifying pre-projection image information, performing projection area layout design, and outputting a pattern layout projection effect; In the embodiment, an initial projection frame image is obtained. The pre-projection image information includes light distribution, background color, shielding condition and physical boundary features and the like. The current projection available area is quickly identified through edge extraction (such as a Canny algorithm), color histogram equalization, background modeling and the like. Subsequently, the system performs area and morphology analysis on the available area, and designs a pattern layout scheme according to the size (such as 20 mm x 20 mm for each pattern) and tolerance parameter of a pattern unit. The layout not only needs to maximize the filling of the projection area, but also needs to consider the edge blank (usually 5% of the area width) to avoid edge deformation and optical interference. Finally, the system outputs a pattern layout projection effect image, and projects it completely to a target surface through a projector, while retaining digital pattern matrix index and theoretical coordinate information, to provide basic data support for subsequent pattern recognition and space mapping.

[0013] Step S2: Perform individual pattern positioning calculation on the pattern layout projection effect to generate sub-pixel positioning coordinates of each pattern; In this embodiment, the current projection pattern is captured at a high frame rate by the image acquisition device, and pattern template matching, corner detection (such as Harris or FAST algorithm) and contour extraction method are used to realize accurate identification of the boundary of each pattern. After the basic identification is completed, the system further uses sub-pixel level image processing method (such as sub-pixel edge fitting or gray centroid positioning algorithm) to perform high-precision positioning on the key features such as pattern center point and corner point. For example, in the gray centroid algorithm, the system calculates the fine coordinates of the pattern center point by weighting the pixel gray values in the pattern area, and the positioning accuracy can reach within 0.1px. In the experimental setup, under the projection scene with uniformity not less than 80%, the average error of the system positioning for 64 pattern units is 0.25px, and the minimum value reaches 0.1px. Finally, this step outputs the sub-pixel positioning coordinates of each pattern in the image coordinate system, and establishes the mapping relationship between the pattern number and its corresponding position, forming a complete positioning data set.

[0014] Step S3: Calculate the projection area center point according to the sub-pixel positioning coordinates, and construct the actual projection positioning coordinate system; In this embodiment, all pattern coordinate sets are input and the geometric center (i.e. the geometric center point of the projection area) is calculated, and the commonly used method is to calculate the arithmetic mean of all pattern center coordinates in X and Y directions. The center point is defined as the origin (0, 0) of the actual projection coordinate system, and the coordinate axis direction is constructed with the main direction of the pattern array as the reference. In order to enhance the anti-interference ability, the system also introduces topological structure analysis method to identify the pattern arrangement direction and symmetry axis, and automatically correct the coordinate axis rotation error caused by perspective distortion or physical offset. After the construction is completed, the system forms a two-dimensional projection coordinate system based on the actual pattern position, which has the function of absolute position reference. On this basis, all subsequent error analysis, compensation calculation and pattern optimization are referenced to this coordinate system to realize the space consistency of the calibration logic. Experimental verification shows that compared with the coordinate system based on the center point of the static image, the stability of the actual coordinate system based on the sub-pixel coordinate center point of the pattern is improved by about 18% in the dynamic lighting environment, which plays a key role in ensuring the positioning accuracy of the system.

[0015] Step S4: Calculate the theoretical position deviation according to the actual projection positioning coordinate system, and extract the projection pixel point position deviation of each pattern; In this embodiment, the actual projection position of the current pattern is compared with the preset theoretical position to obtain the pixel point position deviation of each pattern. The theoretical pattern position is determined by the pattern layout design in step S1, and has a fixed arrangement interval and starting coordinates; and the actual coordinates are obtained from the sub-pixel coordinates calculated in step S2. The system performs position vector difference calculation on each pattern to obtain the offset values (Δx_i, Δy_i) of the two coordinates in the X and Y axis directions, forming an offset vector field. To enhance the robustness of the calculation, the system will eliminate patterns with low signal-to-noise ratio (such as boundary patterns or unevenly illuminated areas), and only keep patterns with an identification rate higher than 95% to participate in the deviation modeling. Further, the system analyzes the deformation trend of the projection system as a whole by fitting the deviation trend surface (such as bivariate polynomial surface fitting), such as whether there is a systematic rotation, scaling or edge deformation, etc. In a data set containing 48 effective patterns, the maximum pixel deviation calculated in this step is 3.7px, the minimum is 0.2px, and the average deviation is 1.4px. The position deviation data is not only used for geometric compensation modeling, but also provides basic input data for subsequent heat map drawing, deviation clustering optimization, etc.

[0016] Step S5: Multi-stage progressive pattern calibration compensation based on projection pixel point position deviation to obtain geometric transformation compensation results.

[0017] In this embodiment, a multi-stage progressive compensation mechanism is started to maximize the restoration of the theoretical position of the pattern in the projection interface. The compensation process is divided into three stages: the first stage is global linear transformation correction, the system constructs an affine transformation matrix of three parameters of translation, rotation and scaling by least squares fitting, and performs one-time coarse adjustment on all pattern coordinates; the second stage is local nonlinear transformation compensation, the system divides the projection area into several local units (such as 16 patterns per block) according to the regional error trend, respectively establishes a local bi-cubic interpolation transformation model, and corrects the local perspective and deformation distortion; the third stage is pixel-level fine adjustment, which performs individual coordinate adjustment based on error vector for each pattern, further reduces the residual error through high-precision morphological fitting and edge pixel repositioning. After performing the whole process compensation under the actual projection interface (1m×0.6m, projection resolution 1280×800), the average error of the system pattern is reduced from 1.4px to 0.3px, the recognition rate is increased to 99.6%, and a set of high-precision geometric transformation compensation matrix is successfully constructed, providing stable output for the final calibration.

[0018] In this embodiment, reference is made to Figure 2 The detailed implementation steps of step S1 include: identifying pre-projection image information; and marking a plurality of projection patterns according to the pre-projection image information; A three-layer nested calibration design is performed on multiple projection patterns to obtain a three-layer nested pattern architecture; A projection display area is identified; an area of the projection display area is calculated and an area shape is analyzed; An area space feature is analyzed according to the area and the area shape to obtain a display area space feature; A pattern layout projection effect is output according to the display area space feature and the three-layer nested pattern architecture.

[0019] In this embodiment, an image frame that has not been actually projected is obtained from an image generation unit in a projection touch system, and a resolution of 1920x1080 is usually taken as a standard input. The image contains pattern units, layer structures, and auxiliary identification elements for subsequent calibration. To achieve high-precision recognition processing, a contour extraction algorithm based on Canny edge detection is adopted, and a Hough line detection technology is used to perform high-robustness extraction on the pattern boundary. At the same time, color space conversion (RGB to HSV) is used to separate each pattern level, and K-means clustering is used for image color block classification to ensure that each type of pattern element has a clear identifiable label. The system converts the recognition result into structured data, extracts key information such as pattern center coordinates, boundary polygon, layer number, and color attribute, and forms a set of image description vectors as the input basis for subsequent pattern marking and nested construction. In the experiment, the test image contains a maximum of 54 basic pattern units, and through the recognition process, the average image processing time is about 0.8 seconds, and the pattern structure recognition accuracy is 98.4%, which can effectively support the high-precision requirements of the subsequent space mapping and calibration process. All pattern units are preliminarily located through the image bounding box (Bounding Box) information, and then numbered and sorted according to the layer attributes and pattern shapes of the image structure. An image correction technology based on affine transformation is used to correct the deformation of the pattern in the image, to ensure that all markers are based on a unified spatial reference coordinate system. In the experimental setup, each pattern is limited within a range of at least 30x30 pixels to ensure the pattern recognition degree in the subsequent optical projection process; at the same time, to avoid pattern repetition or omission, the system sets a maximum of 64 pattern marker units per frame of image, each unit being attached with unique ID, position coordinates, rotation angle, and layer attribute fields. In addition, the system also identifies and classifies the boundaries of the marker patterns (such as circles, rectangles, triangles, etc.), and in the case of high complexity of pattern design, uses contour decomposition and shape approximation methods to improve the recognition and marking accuracy. During the test, a group of pattern samples containing multiple nested structures are processed, and the average pattern marking time is controlled within 1 second, with an error of less than 2px, providing accurate and stable input data support for pattern nesting design.

[0020] All projection patterns are nested in a multi-level structure to support complex touch recognition, image analysis, and interface instruction response mechanisms. The three-layer structure is the basic calibration layer, the structure recognition layer, and the interactive indication layer. The basic calibration layer is mainly used to build a standard geometric grid, with uniformly distributed rectangular units as the system coordinate reference; the structure recognition layer is nested in the basic layer pattern, introducing deformed patterns such as polygons, embedded rings, diagonal line marks, and other pattern features to enhance the recognition accuracy of the recognition algorithm for the internal structure of the pattern; the outermost layer is the interactive indication layer, which constructs a structure with a boundary halo or color block prompt to prompt the user interaction boundary range and the system response area. In the experimental design, the size of each layer of pattern is set to 100%, 70%, and 120% of the basic pattern, respectively, and the center alignment is maintained between the levels. All patterns use high-contrast grayscale design to adapt to various projection background materials. The system uses image synthesis algorithms to dynamically generate nested structures, maintaining geometric symmetry and proportional consistency between patterns. Test data shows that the recognition rate of three-layer nested patterns on irregular projection surfaces is improved from 82% of single-layer structure to 94%, significantly enhancing the system's adaptability in non-ideal projection environments. The actual image range projected by the projection device under the current spatial conditions is detected and recognized in real time. Usually, a depth-sensing camera installed on the system samples the projection picture, and background modeling and foreground extraction algorithms (such as MOG2 or frame difference method) are applied to accurately separate the projection image area. Then, the system uses contour tracking algorithms to extract the region boundary, and combines with the polygon approximation technique to convert it into a quantifiable geometric shape description (such as approximate rectangle, polygon, twisted surface, etc.). The system further calculates the actual area of the region, converts it into the actual physical area (unit: cm²) by using the pixel counting method combined with the camera field of view parameters. At the same time, the system analyzes whether the region shape is distorted, such as trapezoidal distortion caused by oblique projection, irregular contour caused by edge occlusion, etc. In the experimental setup, the projection in wooden plane, glass desktop, curved wall, etc. environments is tested respectively, and the system can complete the region recognition and accurately estimate the area within 2 seconds, with an error of ±3%. The display area geometry model output by this step provides a spatial boundary basis for subsequent pattern arrangement, and also provides a structure basis for pattern deformation and adaptive layout.

[0021] The space strategy parameters available for pattern arrangement are extracted, and the carrying capacity of the display area for the three-layer nested pattern architecture is determined. The analysis includes space availability evaluation, form adaptation capacity evaluation, and area complexity calculation. First, by dividing the display area into fixed-pitch grid cells (such as 50px x 50px per cell), the number of actual available cells and the total number of area cells are counted, and the space availability rate is calculated. Second, the system analyzes the symmetry and regularity of the area, including the number of boundary symmetry axes, the proportion of the largest inscribed rectangle, and the edge curvature variation, etc. The shape feature vector of the entire area is calculated using Hu and Zernike moments and other invariant moments. In addition, to evaluate the arrangement capacity of the nested pattern in irregular areas, the system also introduces the area distortion metric index to evaluate the deviation degree of the geometric center and the outer contour of the area. In the experiment, different desktop shapes (such as regular rectangle, ellipse, irregular polygon, etc.) are modeled, and it is found that irregular areas are on average 18.6% lower in space utilization than standard rectangular areas. The analysis output forms a set of feature parameters, including area utilization level (high / medium / low), pattern nesting adaptation coefficient, and pattern deployable density, which are called by the final pattern layout optimization algorithm.

[0022] In this embodiment, the three-layer nested calibration design is specifically: The three-layer nested calibration design includes an outer boundary positioning ring, a middle layer precision calibration grid, and an inner layer feature recognition code. The outer boundary positioning ring provides coarse positioning information. The middle layer precision calibration grid provides subdivided position coordinates. The inner layer feature recognition code provides a unique ID and direction information.

[0023] In this embodiment, the pattern design structure includes an outer boundary positioning ring, a middle layer precision calibration grid, and an inner layer feature recognition code, which are symmetrically nested around the pattern center to form a standard pattern unit. The outer boundary positioning ring, as the largest part in the three-layer structure with the most prominent recognition features, is designed as a coarse positioning auxiliary structure. Its shape is usually a high-contrast concentric circle or a rounded rectangular frame, with good edge characteristics. It can be quickly extracted through edge detection algorithms such as Canny + Hough circle detection. In the experiment, the length of the ring area is set to 100% of the total pattern structure, and the thickness is 10% of the pattern size, ensuring that it can effectively separate the background and provide initial positioning reference even in complex projection backgrounds.

[0024] The middle layer precision calibration grid is used to provide high-density coordinate reference information in a local area, usually adopts a regular rectangular grid structure, and the grid spacing is set to be 15px to 25px according to the projection precision requirement. The grid intersection coordinates are accurately extracted by a sub-pixel level corner detection algorithm (such as a sub-pixel Harris corner), to realize fine calibration and geometric correction of coordinates. The grid structure not only supports geometric deformation correction on a twisted and tilted projection surface, but also can be used as a positioning reference for a touch interaction area to improve response accuracy.

[0025] The inner layer feature recognition code is the core of the entire nested structure, which is used to indicate the unique identity and orientation of each pattern unit. The system uses an improved visual marker code (similar to AprilTag or ArUco code), which expresses the pattern ID information through a 16x16 binary coding array, and embeds a direction positioning marker (such as a specific corner black block) to determine the pattern direction, which remains stable under different projection angles and image distortion.

[0026] The recognition code area is set to be 30% of the center of the pattern structure, and the average recognition time of the OpenCV recognition algorithm is less than 30ms, and the recognition accuracy is 99.2%. The three-layer structure is symmetrically arranged through a unified geometric transformation, and the robust pattern recognition is realized through boundary contrast, hierarchical feature enhancement, and pattern fault tolerance strategy. The overall test shows that the three-layer nested calibration structure has good recognition ability and positioning accuracy under different light conditions and different projection angles (0°~35° inclination), which supports the subsequent pattern adaptive adjustment, projection layout optimization, and efficient execution of the touch recognition module.

[0027] In this embodiment, referring to Figure 3 For the detailed implementation step flowchart of step S2, in this embodiment, the detailed implementation steps of step S2 include: Performing depth visual recognition on the pattern layout projection effect, and performing image threshold segmentation to extract a plurality of pattern frame images; Performing geometric morphological feature analysis on the plurality of pattern frame images to generate geometric morphological parameters of each pattern; Performing morphological feature point positioning marking according to the geometric morphological parameters to obtain feature point positions of each pattern; Performing sub-pixel level positioning calculation based on the feature point positions to generate sub-pixel positioning coordinates of each pattern.

[0028] In this embodiment, the high-resolution camera (resolution not less than 1920x1080, frame rate 30fps) installed on the touch system is used to collect the image of the projected pattern interface. In order to enhance the separation degree between the pattern edge and the background, the system uses adaptive gray scale conversion combined with local contrast enhancement processing, and carries out image binarization processing through Otsu threshold algorithm or threshold segmentation method based on Gaussian distribution modeling, so as to realize accurate extraction of the pattern boundary. After image segmentation is completed, the contour tracing and boundary extraction technology are used to extract the frame area of multiple independent patterns, and the frame image corresponding to each pattern is generated. The pattern frame image extracted in the previous stage is analyzed by geometric modeling, and the main task is to generate a parameter set describing the shape of each pattern. First, the contour fitting algorithm (such as minimum circumscribed rectangle fitting, ellipse fitting, least square boundary polygon approximation) is used to fit the geometric features of the boundary image, and the area, perimeter, side length ratio, angle information, circularity, rectangularity, edge curvature distribution and other parameters of the pattern are extracted. At the same time, the system uses Fourier shape descriptor to analyze the shape of the complex contour in the frequency domain, so as to identify the shape feature deviation of the irregular boundary pattern under the influence of projection distortion. Experimental tests show that for rectangular patterns, regular polygon patterns and patterns with slightly curved boundaries, the geometric parameter extraction accuracy is more than 95%, and the spatial compensation of deformation error can be carried out, which provides mathematical model support for subsequent feature point positioning and correction.

[0029] The key points used for spatial mapping and pose recognition are accurately extracted from the boundary geometry of the pattern. Typically, corner points, intersection points, and symmetry axis intersection points are selected as feature points, and the Harris corner detection, Shi-Tomasi algorithm, or FAST feature point extraction method is used to achieve preliminary positioning. In the experiment, at least 4 effective corner points are set for each pattern as spatial pose reference points, and the extraction results are further optimized by non-maximum suppression (NMS) and local window intensity screening to remove false detection points and edge interference points. After extraction, each pattern forms a set of feature points including coordinate position, boundary direction, and point type (corner / intersection), which serves as the basis for subsequent sub-pixel level processing. Based on the gray interpolation model and edge sub-pixel fitting algorithm, sub-pixel estimation is performed on each corner or edge point within a local 3x3 or 5x5 pixel window. Common methods include sub-pixel fitting based on image gradient, interpolation positioning based on gray centroid, and Gaussian surface fitting. The system uses a method based on Sobel gradient and bilinear interpolation, which can obtain stable sub-pixel coordinate output without significantly increasing the computational burden. In the experiment, the sub-pixel positioning accuracy is verified by comparing the physical size of the calibration board with the actual pixel point correspondence, and it is found that the system can achieve an average sub-pixel accuracy of ±0.15 pixels, with a positioning deviation of less than 0.2 mm. In touch applications, the positioning error is less than ±1.2 mm, which meets the actual needs of most high-precision projection interaction systems. The sub-pixel coordinate results are mapped to the system reference coordinate system, which serves as an important input for subsequent touch mapping and deformation correction model calculation.

[0030] In this embodiment, reference Figure 4 For the detailed implementation step flowchart of step S3, in this embodiment, the detailed implementation steps of step S3 include: Perform inter-pattern topological correlation analysis on multiple pattern frame images to obtain a topological correlation relationship; Based on the topological correlation relationship and the sub-pixel positioning coordinates, calculate the optimal center point of the region, and mark the center point of the projection region; Divide the projection display region into a grid distribution to construct a projection region grid background; Based on the projection region center point and the projection region grid background, construct an actual projection positioning coordinate system.

[0031] In this embodiment, the connection relationship, relative position and arrangement pattern between each pattern in the spatial structure are identified to obtain the overall topological structure of the pattern arrangement on the projection surface. The system first constructs a set of pattern center points and calculates the Euclidean distance and direction vector between any two pattern center points. By setting a threshold (such as a distance threshold of ±10% of the expected distance between patterns), adjacent pattern pairs are identified, and a pattern topology graph is formed according to the number of connections between patterns, arrangement direction and shape similarity. This topology graph takes pattern ID as node and adjacency relationship as edge, supporting structural connectivity analysis and clustering. The system further uses a graph neural network model to optimize the identification of the topological structure, eliminating isolated patterns and incorrect connection points. In the experiment, the system performs topological analysis on a group of pattern networks composed of 48 patterns, successfully identifies 5 regular sub-regional clusters, and establishes a stable pattern connection map, providing correlation support for subsequent regional center point calculation.

[0032] After obtaining the topological structure between patterns, the system calculates the optimal center point of the entire projection area based on these relationships combined with the sub-pixel positioning coordinates of each pattern, which is used to determine the spatial geometric center and guide the positioning of the coordinate system origin. The calculation of the center point not only uses the geometric center solution (i.e., the arithmetic mean of all pattern centers), but also introduces topological weight factors such as the connectivity, boundary distance, and density distribution of each pattern, to obtain a more spatially representative center point through weighted center method. To avoid the influence of boundary distortion on center deviation, the system sets the weight coefficient of boundary patterns to 0.6 and the weight of central area patterns to 1.0 when calculating the center point. Experimental data shows that the center point calculated using the weighted center model has an error of less than 1.8 mm from the true projection surface physical center point, reducing the deviation by 27% compared to the traditional arithmetic center method. This center point will serve as the origin marker for subsequent grid construction and projection coordinate system establishment, with the advantages of stability and geometric symmetry.

[0033] The system performs grid distribution division on the projection display area, constructs a set of grid background layers consistent with the actual projection area, and serves as the basic framework for spatial mapping and positioning. The grid construction adopts an adaptive grid division strategy, determines the grid density according to the aspect ratio and physical resolution of the projection area, for example, under the standard projection resolution of 1280x800, the grid unit is divided into 32 columns x 20 rows, and the size of each grid is 40px x 40px. In the grid construction process, the overlap degree of the boundary occlusion area and the available projection area is considered, and the region binary mask graph is used to correct the validity of the edge grid unit. The system also allocates a unique number and its relative position in the global coordinate system to each grid unit, and records its relative offset vector with the projection center point. In the experiment, the grid background can be dynamically generated and adaptively adjusted within 1 second, supporting up to 1024 grid units and maintaining real-time performance, providing a good positioning reference basis, and supporting the mapping and re-calibration of touch interaction points.

[0034] After obtaining the center point and the grid division background, the system constructs the actual projection positioning coordinate system. The coordinate system takes the optimal center point of the projection area as the coordinate origin, combines the horizontal and vertical division axes in the grid background, and establishes a local two-dimensional rectangular coordinate system. The system adopts the left-hand coordinate system convention (X right, Y down), and corrects the coordinate axes to match the tilt angle and perspective distortion of the actual projection area. To further improve the accuracy, the system introduces an affine transformation correction parameter based on the projection perspective matrix, and performs projection geometric inverse transformation processing on each coordinate point to ensure that the coordinate axes are aligned with the physical surface. In the experiment, the positioning coordinate system is established for a projection surface with a tilt angle of 20°. Through the overlap error evaluation of the reference pattern edge and the grid axis, the maximum axis offset is controlled within 1.2°, and the coordinate mapping error is not more than ±0.3%, which has good spatial consistency and reusability. The coordinate system serves as a unified reference system for panoramic pattern positioning, touch recognition point projection, and subsequent interactive event response, and constitutes the core support structure of the system's spatial calibration capability.

[0035] In this embodiment, step S4 includes the following steps: According to the actual projection positioning coordinate system, the sub-pixel positioning coordinates are projected to calculate the projection error, and the geometric error is corrected to obtain the geometric correction coordinates; According to the pre-projection image information, the theoretical coordinate calculation is performed to obtain the theoretical pattern projection position; According to the theoretical pattern projection position, the pixel position deviation of the geometric correction coordinates is calculated, and the projection pixel point position deviation of each pattern is extracted.

[0036] In this embodiment, the previously acquired sub-pixel positioning coordinates are compared with the theoretical pattern projection positions for accuracy, and then the spatial errors caused by optical distortion, projection surface deformation, equipment jitter and other factors are identified and corrected. The first task is to map the sub-pixel coordinates to the actual positioning coordinate system, and to complete the coordinate normalization process by combining the previously constructed grid coordinate background and the vector relationship of the regional center points. On this basis, the system performs projection error calculation, the core of which is to calculate the Euclidean distance error between the sub-pixel coordinates and the ideal grid nodes through geometric comparison, and to determine whether the error belongs to systematic deviation or local geometric distortion according to the topology between patterns. In the experiment, the system calculates the error of 54 pattern units and finds that the average initial deviation is 2.8px, and the maximum deviation appears in the edge area, reaching 5.2px. To improve the correction accuracy, the system uses a geometric error correction method based on a bidirectional affine transformation model, uses the optimal center point of the region as a reference, combines the distance between patterns, the direction angle and the corner point distribution, and performs local geometric correction on each pattern position to correct its spatial mapping deviation on the projection surface. After correction, each pattern generates a set of geometric correction coordinates, which have consistency with the system positioning coordinate system and effectively reduce the influence of subsequent pixel-level errors. In multiple scene tests (including curved walls, inclined desktops, boundary occlusion, etc.), the system's geometric correction strategy controls the average deviation error within 0.9px, providing a high-precision spatial foundation for pattern accuracy identification and interactive response.

[0037] To compare the differences between the actual projection effect and the theoretical design output, the theoretical pattern projection position of each pattern needs to be calculated based on the pre-projection image information. The pre-projection image information contains the original design position of all patterns, layer relationship, distance between patterns and graphic structure, which is initially generated in the pattern construction and nesting design stage and has a completely idealized geometric structure without any optical distortion factors. To map this ideal structure to the current projection environment, the system first performs image geometric projection transformation deduction and uses the standard perspective projection matrix model (3x3 homogeneous transformation matrix) to map the theoretical pattern coordinates from the image space to the physical projection coordinate system. The transformation relationship is obtained by reversing the positions of four projection reference points (such as the four corners of the pattern matrix) in the actual camera observation image. In this process, the system adaptively corrects the image rotation, projection translation and scale difference, and further corrects the transformation accuracy through an external reference calibration board (set at the projection corners). Experimental results show that through this theoretical position calculation method, the system can complete the theoretical mapping coordinate calculation of all pattern positions in an average of 1.2 seconds, with a mapping accuracy of more than 98%, especially in the central area with the smallest error, and the boundary area is further corrected through subsequent geometric constraints.

[0038] After the actual geometric correction coordinates and the theoretical pattern positions are obtained, pixel position deviation calculation is performed to accurately extract the image position error of each pattern in the final projection process. The calculation takes the pattern center point as the reference, compares the pixel offset value between the theoretical coordinates and the corrected actual coordinates, and expresses the offset direction and size (Δx, Δy) in a vector manner. In addition, the system also calculates the relative error percentage of the offset amount to identify whether there is a systematic deviation pattern in the pattern, such as horizontal stretching, vertical compression, perspective distortion, etc. During the calculation process, all error data is archived to an independent record unit for each pattern, supporting pattern-level offset analysis and dynamic reconstruction. In the experiment, this step is used to verify whether the overall projection accuracy of the system meets the calibration requirements - if the deviation of a pattern exceeds ±2px, the pattern re-calibration instruction will be triggered. Under standard conditions (indoor 300 lux illumination, 1.5 meter projection distance), the average pixel deviation of the system is 1.1px, and 95% of the patterns are controlled within a 2px error range, which is much lower than the tolerable threshold of the visual recognition system, fully guaranteeing the consistency and accuracy of touch response. This deviation data can also be used to build a subsequent global distortion correction model, further realizing system-level geometric optimization.

[0039] In this embodiment, the specific steps of step S5 are: Multi-stage progressive pattern calibration compensation based on projection pixel point position deviation is performed to obtain a geometric transformation compensation result; The geometric transformation compensation result is input to the projection touch system and secondary projection processing is performed to extract a secondary projection image; The pattern calibration effect of the secondary projection image is evaluated to obtain a calibration evaluation report; Based on the calibration evaluation report, projection parameter closed-loop iterative optimization is performed to execute intelligent projection calibration work.

[0040] In this embodiment, a multi-stage progressive pattern calibration compensation based on the position deviation of the projected pixel points is started. This method differentiates the compensation processing of global and local patterns through a step-by-step geometric transformation model. In the initial stage, a global affine transformation is used to uniformly adjust the coordinate set of all patterns to the theoretical reference position; then a bidirectional nonlinear geometric correction is applied in the local area (such as a cluster of every 4x4 pattern), a bicubic interpolation and a local perspective transformation matrix fitting are used for fine adjustment of edge distortion and rotation deviation. The system constructs a pattern offset heat map based on the error distribution characteristics, identifies high error areas for fine compensation. In the experiment, the progressive compensation is divided into three rounds, and the error is reduced by about 35% in each round. After the overall compensation is completed, the average pixel deviation is reduced from the initial 1.8px to within 0.5px, greatly improving the stability and symmetry of the projected pattern. The compensation result is finally output as a set of pattern transformation matrices and spatial position parameters, forming a geometric transformation compensation result dataset, and is transmitted to the projection touch system.

[0041] The system directly applies the above-mentioned geometric transformation compensation result to the image output module in the projection touch system, starts the secondary projection processing mechanism, and collects the output image in real time for quality verification. In this process, the projection system redraws the panoramic pattern layout with the updated pattern coordinate parameters, performs transformation mapping, coordinate correction, and edge fusion on all patterns through the GPU rendering engine, and generates a compensated pattern frame image. The secondary projection image is collected and saved by the built-in camera (1080p resolution, 60fps) in the system for subsequent analysis. This process usually completes rendering and projection image capture within 1 second, ensuring that the entire system can complete dynamic recalibration without affecting user operation. The system pre-processes the collected image, including contrast enhancement, boundary extraction, and pattern template matching, to ensure the consistency of pattern recognition. The collected image will be used for secondary analysis of pattern recognition to evaluate the compensation effect and provide input basis for the closed-loop optimization mechanism.

[0042] The system performs pattern calibration effect evaluation on the acquired secondary projection image, and automatically generates a calibration evaluation report. The evaluation indicators include pattern recognition rate, boundary fitting accuracy, center point offset, corner point matching consistency, and regional pattern density uniformity, etc. The actual coordinates and theoretical coordinates of each pattern are compared again, and the overall calibration quality score is calculated combined with the regional error distribution. In addition, the system identifies the pattern cluster group with potential error accumulation through pattern clustering analysis and error matrix heatmap visualization, as the key area for the next round of optimization. During the experiment, the system evaluates 48 patterns, and the evaluation report shows that the recognition rate is improved to 99.7%, and the average value of the pattern center point offset is 0.3px, which is reduced by nearly 60% compared with the first calibration. The report is output in a structured form, including overall error distribution graph, pattern error ranking list, key pattern error details, etc. At the same time, it makes suggestions for further optimization, such as "continue optimization", "meets the standard", "needs manual intervention", etc., providing data support for subsequent system strategy selection.

[0043] The system starts the closed-loop iterative optimization mechanism of the projection parameters based on the error information feedback in the evaluation report to complete the final intelligent projection calibration task. The mechanism takes the minimization of the objective function as the core drive, combines error feedback and pattern feature historical data, and adjusts the pattern coordinates, scaling ratio, rotation angle, etc. through incremental optimization. The optimization algorithm selects local optimization combined with gradient descent and global parameter reconstruction technology based on error propagation model to correct the projection parameters in the error-prone area in each iteration, while maintaining the stability of the low-error area. The system sets the maximum number of iterations to 5 rounds, controls the optimization period to be within 1.5 seconds in each round, and re-executes the secondary projection and evaluation process after each round. The termination conditions of the closed-loop mechanism include error stable convergence within 0.3px, pattern recognition rate higher than 99.5%, or the evaluation system sends a "meets the standard" signal. Experimental data shows that using the closed-loop mechanism can improve the overall calibration accuracy to within ±0.2px, and the projection system can finally achieve 1mm-level interactive precision, marking that the system has reached the industrial-level panoramic projection calibration level and has the ability of self-adaptive update in dynamic environment, providing a solid foundation for multi-user touch interaction and high-precision visual recognition.

[0044] In this embodiment, the multi-stage progressive pattern calibration compensation is specifically; The multi-stage progressive pattern calibration compensation is specifically: one-stage global linear transformation correction, two-stage local nonlinear transformation compensation, and three-stage pixel-level fine adjustment; The one-stage global linear transformation correction is specifically: global geometric deviation parameters are calculated according to the position deviation of the projection pixels, and global geometric transformation is performed, including translation, rotation, and scaling, to output the one-stage correction result; The two-stage local nonlinear transformation compensation specifically comprises: analyzing projection distortion information based on the one-stage correction result, performing nonlinear distortion compensation, and outputting a two-stage correction result. The three-stage pixel-level fine adjustment specifically comprises: fine adjustment of each deviated pixel point according to the two-stage correction result, and outputting a geometric transformation compensation result.

[0045] In this embodiment, the first step of the multi-stage progressive pattern calibration compensation mechanism is one-stage global linear transformation correction. This stage mainly faces the overall deviation problem of the projection system at the macro level, such as unified deviation, rotation or proportional deformation of the entire pattern due to installation errors, projection angles or projection distances. In specific operation, the system first counts the position deviation of all pattern projection points, taking the offset vector between the theoretical center point and the actual center point of the pattern as the input, calculates the global offset parameter set (Δx, Δy, θ, s), which respectively corresponds to the horizontal offset, vertical offset, overall rotation angle and scaling factor. Then, a linear affine transformation model is used to perform geometric transformation processing on the entire pattern. The formula uses a 2x3 transformation matrix (linear form) to perform coordinate unified adjustment. The system takes this stage as a preliminary coarse correction to quickly compress the overall error bandwidth. In the experiment, for a projection surface covering an area of 1.2m x 0.75m, with a total of 64 patterns, the initial maximum error is ±5.3px, and after one-stage global linear transformation, the error is reduced to ±2.1px, the error distribution is more concentrated, and the pattern array structure tends to be stable, laying a foundation for the next stage of local compensation.

[0046] In the second stage of local nonlinear transformation compensation, the system focuses on compensating for nonlinear image distortion caused by projection lens distortion, projection surface curvature or local occlusion. In this stage, the system takes the one-stage correction result as the input basis and performs error clustering analysis on the area where each pattern is located, dividing it into several local compensation units (usually 3x3 or 4x4 patterns as a local block). By analyzing the boundary curvature, center offset angle, edge length deformation and other indicators of the patterns in each block, the system determines the distortion type (such as barrel, pillow or wave distortion) existing in the region. The system uses a bidirectional nonlinear interpolation model in the compensation strategy, combining bicubic interpolation and local Bezier curve fitting methods to independently model and correct the X-axis and Y-axis offset trends. When processing the edge area, a radial distortion model (Radial Distortion Model) is also introduced to improve the recovery accuracy of the edge patterns. The test data shows that after two-stage nonlinear compensation, the pattern error in the region is further compressed to ±0.9px, some high error points are effectively pulled back to a reasonable range, and the conformity of the compensated pattern outline with the theoretical template is significantly improved (the average shape matching rate reaches 96.8%), ensuring the local consistency of subsequent high-precision interaction.

[0047] The last three-stage pixel-level fine adjustment is aimed at individual fine-tuning at the pattern level, with the goal of eliminating residual errors caused by local complex environmental interference (such as reflection, corner occlusion), and making the spatial coordinates of the pattern reach sub-pixel level accuracy. The specific approach is: the system extracts residual deviation data from the coordinate results output by the second stage, and processes the patterns whose errors are still higher than the set threshold (such as ±0.8px) individually. Using image differential enhancement technology and local contrast enhancement method, the system performs sub-pixel level feature point positioning on the edges of the pattern, and introduces Gaussian filtering and morphological gradient analysis to enhance the clarity of the boundary contour. Subsequently, the system corrects the contour of the actual projection shape of each pattern through edge fitting algorithm (such as Sobel edge detection + least squares fitting), and adjusts the pixel position point by point. Each fine-tuning is performed with 0.1px as the unit of iterative optimization until the error is lower than the termination threshold set by the system. Under the actual measurement conditions (600 lux indoor uniform illumination, camera resolution 1920x1080), the average time consumption of the three-stage fine adjustment is 2.3 seconds, and the final average error is ±0.25px. The overall pattern recognition accuracy of the system reaches the sub-pixel level, meeting the application requirements of high-precision projection touch system in education, industry and interactive art and other scenes.

[0048] In the embodiment, an adaptive calibration device of a projection touch system is provided for performing the adaptive calibration method of the projection touch system as described above, comprising: A projection layout module is configured to identify pre-projection image information, perform projection area layout design, and output pattern layout projection effect. A positioning calculation module is configured to perform individual pattern positioning calculation on the pattern layout projection effect, and generate sub-pixel positioning coordinates of each pattern. A coordinate system construction module is configured to calculate a projection area center point according to the sub-pixel positioning coordinates, and construct an actual projection positioning coordinate system. A deviation calculation module is configured to calculate theoretical position deviation according to the actual projection positioning coordinate system, and extract projection pixel point position deviation of each pattern. A calibration compensation module is configured to perform multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation, to obtain a geometric transformation compensation result.

[0049] Therefore, regardless of the point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application being defined by the appended claims rather than the above description, and all variations falling within the meaning and scope of the equivalent elements of the application file are intended to be included in the present application.

[0050] The foregoing merely illustrates the principles of the application and application of its principles. Various modifications and alterations to this implementation will occur to those skilled in the art. The scope of the application should be determined, however, by the following claims rather than by the embodiments shown.

Claims

1. An adaptive calibration method of a projection touch system, characterized in that, The method comprises the following steps: Step S1: identifying pre-projection image information, designing projection area layout, and outputting pattern layout projection effect; Step S2: performing individual pattern positioning calculation on the pattern layout projection effect, and generating sub-pixel positioning coordinates of each pattern; Step S3: calculating a projection area center point according to the sub-pixel positioning coordinates, and constructing an actual projection positioning coordinate system; Step S4: performing theoretical position deviation calculation according to the actual projection positioning coordinate system, and extracting projection pixel point position deviation of each pattern; Step S5: performing multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation, to obtain a geometric transformation compensation result. 2.The method of claim 1, wherein, The specific steps of step S1 are as follows: identifying pre-projection image information; marking a plurality of projection patterns according to the pre-projection image information; performing three-layer nested calibration design on the plurality of projection patterns, to obtain a three-layer nested pattern architecture; identifying a projection display area; calculating an area of the projection display area and analyzing an area form; performing area space feature analysis according to the area and the area form, to obtain display area space features; performing pattern layout coverage rate maximization calculation according to the display area space features and the three-layer nested pattern architecture, and outputting the pattern layout projection effect. 3.The method of claim 2, wherein, The three-layer nested calibration design specifically comprises: the three-layer nested calibration design comprises an outer boundary positioning ring, a middle layer precision calibration grid, and an inner layer feature identification code; the outer boundary positioning ring provides coarse positioning information; the middle layer precision calibration grid provides subdivided position coordinates; the inner layer feature identification code provides unique ID and direction information. 4.The method of claim 1, wherein, The specific steps of step S2 are as follows: performing depth visual recognition on the pattern layout projection effect, and performing image threshold segmentation, to extract a plurality of pattern border images; performing geometric form feature analysis on the plurality of pattern border images, to generate geometric form parameters of each pattern; performing form feature point positioning marking according to the geometric form parameters, to obtain feature point positions of each pattern; performing sub-pixel level positioning calculation based on the feature point positions, to generate sub-pixel positioning coordinates of each pattern.

5. The self-adapting calibration method of the projection touch system according to claim 1, wherein, The specific steps of step S3 are as follows: performing inter-pattern topological correlation analysis on the plurality of pattern border images, to obtain a topological correlation relationship; performing region optimal center point calculation based on the topological correlation relationship and the sub-pixel positioning coordinates, to mark a projection area center point; performing grid distribution division on the projection display area, to construct a projection area grid background; constructing an actual projection positioning coordinate system based on the projection area center point and the projection area grid background. 6.The method of claim 1, wherein, The specific steps of step S4 are as follows: performing projection error calculation on the sub-pixel positioning coordinates according to the actual projection positioning coordinate system, and performing geometric error correction, to obtain geometric correction coordinates; performing theoretical coordinate calculation according to the pre-projection image information, to obtain theoretical pattern projection positions; performing pixel position deviation calculation on the geometric correction coordinates according to the theoretical pattern projection positions, to extract projection pixel point position deviation of each pattern. 7.The method of claim 1, wherein, The specific steps of step S5 are as follows: performing multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation, to obtain a geometric transformation compensation result. The geometric transformation compensation result is input to the projection touch system and is subjected to secondary projection processing to extract a secondary projection image; The secondary projection image is subjected to pattern calibration effect evaluation to obtain a calibration evaluation report; Based on the calibration evaluation report, projection parameter closed-loop iterative optimization is performed to execute intelligent projection calibration work. 8.The self-adapting calibration method of the projection touch system according to claim 7, characterized in that, The multi-stage progressive pattern calibration compensation specifically includes: The multi-stage progressive pattern calibration compensation specifically includes: one-stage global linear transformation correction, two-stage local nonlinear transformation compensation, and three-stage pixel-level fine adjustment. The one-stage global linear transformation correction specifically includes: global geometric deviation parameters are calculated according to projection pixel point position deviation, and global geometric transformation is performed, including translation, rotation, and scaling, to output one-stage correction results. The two-stage local nonlinear transformation compensation specifically includes: projection distortion information is analyzed based on the one-stage correction results, nonlinear distortion compensation is performed, and two-stage correction results are output. The three-stage pixel-level fine adjustment specifically includes: fine adjustment of each deviation pixel point is performed according to the two-stage correction results, and geometric transformation compensation results are output.

9. An adaptive calibration device for a projection touch system, characterized in that, A method for performing adaptive calibration of a projection touch system as claimed in claim 1, comprising: A projection layout module for identifying pre-projection image information, designing a projection area layout, and outputting a pattern layout projection effect; A positioning calculation module for calculating the position of each pattern based on the sub-pixel positioning coordinates to generate sub-pixel positioning coordinates of each pattern; A coordinate system construction module for calculating the center point of the projection area based on the sub-pixel positioning coordinates to construct an actual projection positioning coordinate system; A deviation calculation module for calculating the theoretical position deviation based on the actual projection positioning coordinate system to extract the projection pixel point position deviation of each pattern; A calibration compensation module for performing multi-stage progressive pattern calibration compensation based on the projection pixel point position deviation to obtain geometric transformation compensation results.

Images