A method and system for displaying large-format tactile graphics

Abstract

This invention relates to the field of human-computer interaction and haptic feedback technology, and in particular to a method and system for displaying large-format haptic graphics. The system includes a two-dimensional motion platform, a local dynamic haptic module, a visual sensing unit, and a central control unit. The visual sensing unit acquires and tracks the user's fingertip position in real time. The central control unit performs local dynamic processing on the graphic data based on the fingertip position information and drives the haptic array to output corresponding haptic information. Simultaneously, it controls the two-dimensional motion platform to maintain a correspondence between the haptic module and the fingertip position within a planar area. Through this method, high-refresh-rate haptic presentation of large-format graphics can be achieved without requiring a large-area haptic array coverage, offering advantages such as low system complexity, fast response speed, and high reliability.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction and haptic feedback technology, and more specifically, to a method and system for displaying large-format haptic graphics. Background Technology

[0002] Providing visually impaired individuals with tactile interfaces capable of perceiving large-format, high-resolution graphics is an important research direction in assistive technology. In existing technologies, a common approach is to employ a full-array architecture where "one touch point corresponds to one driving unit." This architecture can, in principle, directly map image information to tactile output; however, as the display area increases, the number of required driving units grows linearly, leading to significant challenges in system integration, manufacturing costs, power management, heat dissipation design, and control complexity.

[0003] To address the aforementioned issues, Tsinghua University proposed an improved solution by separating the driving units from the contact array, constructing a "1:n" driving structure. This solution places a two-dimensional moving platform with a small number of driving devices beneath the array containing numerous passive contacts. The platform traverses the entire contact area, setting the overall haptic image point-by-point through a "refresh" process. This method effectively reduces the number of driving units and alleviates the resource consumption problem associated with a full-array architecture. However, this solution requires an independent mechanical self-locking mechanism for each contact to maintain its state. For conventional haptic display devices, this means deploying tens of thousands of micro-mechanical components. Such a design places high demands on manufacturing processes and assembly precision in practical applications, and may lead to component jamming or functional failure due to mechanical wear or foreign object intrusion during long-term operation, thus affecting the system's reliability and lifespan.

[0004] Therefore, there is an urgent need to invent a large-format graphic tactile display method and system based on visual tracking and dynamic tactile array following, which aims to solve the problems of system complexity, integration difficulties, low reliability and poor refresh efficiency caused by the use of full array drive structure or reliance on a large number of mechanical self-locking units in existing large-size tactile display devices. Summary of the Invention

[0005] In view of this, the present invention proposes a method and system for displaying large-format tactile graphics, aiming to solve the problems of system complexity, integration difficulty, low reliability and poor refresh efficiency caused by the use of full array driving structure or reliance on a large number of mechanical self-locking units in current large-size tactile display devices.

[0006] This invention proposes a large-format graphic haptic display system based on visual tracking and dynamic haptic array following, comprising: A two-dimensional motion platform is used to provide two-dimensional motion within a planar range to carry and drive a local dynamic haptic module to move within a working area; the two-dimensional motion platform can be composed of mutually orthogonal X-axis motion components and Y-axis motion components; A local dynamic tactile module is installed on the two-dimensional motion platform. The local dynamic tactile module includes a tactile array composed of multiple tactile units and a corresponding driving circuit, which is used to output tactile feedback in a local area. The tactile array can adopt an m×n array arrangement of tactile units, and the tactile units can be piezoelectric driven, electromagnetic driven, shape memory alloy driven, or other driving forms that can realize tactile output. A visual sensing unit is used to collect image information of user touch behavior within the work area. The visual sensing unit can be an image acquisition device such as an industrial camera, set at a preset position in the work area to obtain image information of the user's fingertip / touch position. Preferably, it is set above a two-dimensional motion platform with the optical axis approximately perpendicular to the work surface to achieve observation of the fingertip / touch position. The central control unit is connected to the two-dimensional motion platform, the local dynamic tactile module, and the visual sensing unit, respectively, and is used to uniformly schedule and control the visual data processing, local graphic extraction, tactile rendering output, and platform motion.

[0007] Furthermore, the central control unit includes: The image preprocessing unit is used to perform step-by-step downsampling processing on the virtual graphics after loading the static virtual graphics to be displayed, construct the Mipmap image pyramid, and generate multiple image layers including the original resolution image and its step-by-step halved resolution images; The coordinate calculation unit is used to receive the original position information of the fingertip / touch position output by the visual sensing unit, and to calculate and stabilize the position to output the corresponding smooth physical coordinates. The mapping calculation unit is electrically connected to the image preprocessing unit and the coordinate calculation unit. The mapping calculation unit is used to map the smooth physical coordinates to the virtual graphic coordinate system according to the preset scaling factor and image placement offset, generate the corresponding pixel coordinates, and perform boundary constraint processing on the pixel coordinates. The rendering scheduling unit is electrically connected to the mapping calculation unit. The rendering scheduling unit is used to determine the pixel size of the target sampling window based on the physical size and scaling factor of the local dynamic haptic module, and select the image level that matches the target sampling window size from the Mipmap image pyramid as the source image, and extract the local image region centered on the pixel coordinates; the rendering scheduling unit is also used to resample and threshold the local image region to generate a haptic instruction sequence for driving the local dynamic haptic module; The motion control unit is used to calculate the target position of the two-dimensional motion platform based on the fingertip / touch position information and generate platform drive control commands so that the local dynamic haptic module moves with the touch position. Furthermore, when the mapping calculation unit performs coordinate transformation, it includes: After receiving the physical coordinates of the fingertip, the mapping calculation unit performs a linear transformation on the physical coordinates according to a preset scaling factor and image placement offset to obtain the virtual image pixel coordinates corresponding to the physical coordinates of the fingertip / touch position, where: The mapping calculation unit is also used to perform boundary constraint processing on pixel coordinates along the horizontal and vertical directions, respectively, where: When pixel coordinates exceed the effective range of the virtual graphics, the mapping calculation unit restricts them to boundary values ​​in the corresponding direction; When the pixel coordinates do not exceed the valid range of the virtual graphics, the mapping calculation unit rounds them to the nearest integer pixel coordinates. When the pixel coordinates in any direction are subject to boundary constraints, it is determined that the fingertip has touched the edge of the virtual graphic, and an edge feedback signal is generated. The edge feedback signal is used to control the local dynamic haptic module to provide haptic boundary prompts to the user.

[0008] Furthermore, when selecting the Mipmap level, the rendering scheduling unit includes: The rendering scheduling unit is also used to determine the target pixel sampling window size corresponding to the current haptic display area based on the physical size of the local dynamic haptic module and the preset coordinate scaling factor, and to impose a minimum pixel constraint on the sampling window size; The rendering scheduling unit is also used to traverse the image size of each resolution level in the Mipmap image pyramid, and select the resolution level with the smallest deviation as the source image for the current sampling based on the size deviation between the image size of each level and the target sampling window size. The rendering scheduling unit is also used to extract a local image region from the source image at the selected resolution level, centered on the currently mapped fingertip pixel coordinates and with a size corresponding to the target sampling window; The rendering scheduling unit is also used to resample local image regions, map them into grayscale matrices of the same size as the touch column array, and perform thresholding on the grayscale matrix to generate a haptic drive instruction sequence for controlling the lifting and lowering state of the touch columns.

[0009] Furthermore, the central control unit also includes the following when it detects a fingertip touching the image boundary: The central control unit is also used to send deceleration commands to the drivers of the X-axis and Y-axis guide rail assemblies, reducing the platform's movement speed to 0 until the fingertip coordinates return to the effective area, wherein: The central control unit is also used to maintain the current touch state of the local dynamic haptic module.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) By setting up local dynamic tactile modules on a two-dimensional motion platform, the tactile array moves with the platform within a planar range, thereby achieving continuous tactile coverage of a large-format virtual graphic area without the need for a large-size overall tactile array. Compared with a fixed large-area tactile array solution, this invention can complete a large-scale tactile display using only local tactile units, effectively reducing the number of touch points and driving channels, lowering system manufacturing costs and power consumption, and improving system scalability.

[0011] (2) The visual sensing unit obtains the position change information of the user's fingertip in the working area in real time, and the central control unit processes and tracks it so that the local dynamic tactile module always moves to the area corresponding to the fingertip position, thereby ensuring the consistency between the tactile feedback area and the virtual graphics in space, avoiding tactile delay or misalignment, and improving the realism and immersion of the interaction.

[0012] (3) The local dynamic tactile module can form a high-density, controllable tactile output in a local area, which is conducive to accurately presenting the edge and detail features of virtual graphics.

[0013] (4) The system has good scalability and can be adapted to different types of tactile array scale, visual processing methods and motion control platforms to realize tactile display of graphics of various sizes and resolutions. It can also realize larger-format tactile graphic display by extending the travel of the two-dimensional motion platform.

[0014] On the other hand, this application also provides a method for displaying large-format haptic graphics, including: S1: Load static virtual graphics and preprocess the graphics data to obtain multi-scale / multi-resolution representation; S2: Collect user touch behavior image information, identify and obtain the user's fingertip / touch position, perform position calculation and stabilization processing, and obtain smooth position coordinates; S3: Based on the preset scaling factor and offset, map the smooth coordinates to the pixel coordinates of the virtual image and perform boundary truncation; S4: Based on the physical size of the haptic module and the scaling factor, obtain the required pixel window size, and extract the local graphic data centered on the mapped coordinates from the optimal level; S5: Render local graphic data into haptic commands that match the haptic array, driving the local dynamic haptic module to output haptic feedback; S6: Synchronously control the movement of the two-dimensional motion platform so that the haptic module always follows the fingertip / touch position; S7: Repeatedly execute S2 to S6 to achieve continuous tactile display of large-format graphics.

[0015] It is understood that the large-format tactile graphic display method and system in the above embodiments of the present invention have the same beneficial effects, and will not be described again. Attached Figure Description

[0016] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A functional block diagram of a large-format tactile graphic tactile display system provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a large-format graphic haptic display system based on visual tracking and dynamic haptic array following, provided in an embodiment of the present invention. Figure 3 A sampling principle diagram for Mipmap optimization provided in this embodiment of the invention; Figure 4 A structural diagram of a two-dimensional motion platform provided in an embodiment of the present invention; Figure 5 A flowchart illustrating a large-format tactile graphic display method provided in this embodiment of the invention; The components include: 1. Two-dimensional motion platform; 2. Local dynamic tactile module; 3. Visual sensing unit; 4. Central control unit; 5. Piezoelectric ceramic drive unit array; 6. Contact column array; and 7. Printed circuit board. Detailed Implementation

[0017] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] A piezoelectric actuator array is a driving structure formed by arranging multiple piezoelectric actuating units in a predetermined row and column pattern. It is used to generate a small displacement output after receiving an electric driving signal, so as to realize the independent or coordinated driving of multiple touch points in the haptic display unit, and provide the execution basis for the rapid refresh of local haptic graphics.

[0019] A tactile pin array is a tactile output structure formed by arranging multiple tactile pins that can reciprocate in a vertical direction in an array. It is used to directly contact the user's skin and present a spatially distributed tactile form to realize the tactile expression of graphic, contour or texture information.

[0020] Drive signal decoding refers to the process of parsing and format conversion of tactile drive commands from the central control unit. It is used to convert digital control signals into control signals that can drive each execution unit separately, so as to ensure that the tactile output is consistent with the command content.

[0021] A printed circuit board (PCB) is a basic circuit carrier used to carry electronic components and realize electrical connections. It is used to integrate drive signal decoding, voltage conversion, communication interface and control logic to achieve unified control of piezoelectric ceramic drive unit array.

[0022] USB 3.0 (Universal Serial Bus 3.0) refers to a high-speed serial communication interface standard used to transmit high-bandwidth image data between visual sensing units and central control units to meet the communication requirements of real-time visual perception and position calculation.

[0023] An industrial camera is a visual acquisition device used to stably acquire high-resolution images in industrial or scientific research scenarios. It is used to obtain image information of target objects within the working area to provide data input for location recognition and visual tracking.

[0024] RS485 bus refers to a serial bus interface that supports long-distance, multi-node communication. It is used to transmit control commands and status information between the central control unit and the motion actuator to achieve reliable communication for motion control.

[0025] Static virtual graphics refer to pre-generated and stored two-dimensional image data used as a source of content for tactile display. During interaction, it is dynamically sampled according to the user's position and converted into tactile output.

[0026] Mipmap Image Pyramid refers to a collection of multi-resolution images generated by progressively downsampling the same virtual graphic. It is used to quickly select image data of appropriate resolution at different interactive scales to improve real-time rendering efficiency and maintain perceptual consistency.

[0027] Kalman filtering is a recursive estimation algorithm based on a state-space model. It is used to fuse historical states with current observation data to smooth the target position and reduce the impact of noise on the position calculation results.

[0028] Raw pixel coordinates refer to pixel position data extracted directly from an image without filtering or mapping, used to characterize the initial position of a target in the image coordinate system.

[0029] Image placement offset refers to the offset parameter used to describe the spatial position of a virtual graphic relative to the physical interaction area. It is used to align physical coordinates with image coordinates during coordinate mapping.

[0030] Linear transformation refers to a mathematical processing method that combines scaling and translation operations on input coordinates to achieve a unified mapping relationship between different coordinate systems.

[0031] Boundary limitation processing refers to the process of constraining coordinate values ​​that exceed a predetermined valid range during coordinate calculation. This is used to prevent index out-of-bounds errors and ensure the stability of system operation.

[0032] Boundary value refers to the maximum or minimum value used in boundary constraint processing to limit the coordinate range, and is used to characterize the effective edge position of virtual graphics or working area.

[0033] Edge feedback signal refers to the control signal generated when a user's operation is detected to be close to or touching the boundary of a virtual graphic. It is used to trigger haptic or motion feedback to indicate to the user that they have reached the edge of the effective interaction range.

[0034] Pulsed Vibration Mode refers to a periodic vibration drive mode that outputs at a preset frequency and duration, used to convey boundary, warning, or status change information to the user through tactile means.

[0035] The target pixel sampling window size refers to the size of the image sampling area determined based on the physical size of the haptic module and the coordinate scaling relationship. It is used to limit the range of local images extracted from virtual graphics.

[0036] The minimum pixel constraint is a lower limit set when calculating the size of the sampling window to prevent information loss or calculation errors caused by an excessively small sampling area.

[0037] A grayscale matrix is ​​a data structure composed of multiple grayscale values ​​arranged in rows and columns. It is used to characterize the brightness distribution of a local image region and serves as an intermediate data form for generating tactile commands.

[0038] A tactile driving instruction sequence refers to control data formed by arranging multiple touch control signals in a time or spatial order, which is used to drive the tactile output unit to generate corresponding tactile patterns.

[0039] An RGB image is an image data format that uses three color channels—red, green, and blue—to describe the color information of pixels. It is used to fully represent the color and brightness information of a visual scene and serves as the basic input data for visual processing.

[0040] It is important to note that all content not described in detail in the specification belongs to the prior art known to those skilled in the art, and the model parameters of each electrical appliance are not specifically limited; conventional equipment can be used. Electrical control components not mentioned in this technical solution are not shown in the figures because they belong to the prior art, and will not be described here. Various preset thresholds not mentioned in this technical solution also belong to the prior art known to those skilled in the art.

[0041] like Figures 1-4 As shown in some embodiments of this application, this embodiment provides a large-format graphic tactile display system based on visual tracking and dynamic tactile array following, including: a two-dimensional motion platform 1, a local dynamic tactile module 2, a visual sensing unit 3, and a central control unit 4.

[0042] Specifically, the two-dimensional motion platform 1 is composed of an X-axis guide rail assembly and a Y-axis guide rail assembly orthogonally assembled.

[0043] Specifically, when the two-dimensional motion platform 1 is orthogonally assembled from the X-axis guide rail assembly and the Y-axis guide rail assembly, it includes: the X-axis guide rail assembly is equipped with an X-axis linear guide rail fixed on the base, an X-axis slide mounted on the X-axis linear guide rail, an X-axis stepper motor fixed at one end of the base, a coupling connecting the output shaft of the X-axis stepper motor and the X-axis ball screw, and an X-axis ball screw threaded with the X-axis slide; the Y-axis guide rail assembly is equipped with a Y-axis linear guide rail fixed on the X-axis slide, a Y-axis slide mounted on the Y-axis guide rail, a Y-axis stepper motor fixed at one end of the X-axis slide, a coupling connecting the output shaft of the Y-axis stepper motor and the Y-axis ball screw, and a Y-axis ball screw threaded with the Y-axis slide.

[0044] Specifically, the X-axis slide and the Y-axis slide are connected by four M3 screws. The bottom surface of the Y-axis linear guide has a locating pin hole, which cooperates with the locating pin on the upper surface of the X-axis slide to ensure that the installation position error of the Y-axis assembly is less than ±0.05mm. The bottom of the housing of the local dynamic haptic module 2 has four countersunk holes, which are aligned with the threaded holes on the upper surface of the Y-axis slide and then locked with M2 screws. The geometric center of the Y-axis slide and the center of the local dynamic haptic module 2 are no more than 0.1mm apart.

[0045] Understandably, the two-dimensional motion platform 1 achieves high-precision linear motion control with two degrees of freedom in a plane by modularly superimposing the X-axis and Y-axis guide rail assemblies in an orthogonal manner. This is achieved by decomposing planar motion into two independent and non-interfering one-dimensional linear motions in orthogonal directions, each implemented by an independent drive and guide mechanism. Specifically, the X-axis guide rail assembly uses the base as the mounting reference, constrains the slide's motion trajectory through linear guide rails, and utilizes a transmission mechanism composed of a stepper motor and a ball screw to precisely convert the motor's rotational motion into linear displacement along the X-axis direction. Based on this, the Y-axis guide rail assembly is further mounted on the X-axis slide, allowing the Y-axis motion to be superimposed on the X-axis motion, thus forming a motion capability covering the two-dimensional working area. The threaded fit between the ball screw and the slide effectively reduces transmission friction and backlash, improving positioning accuracy and repeatability. Meanwhile, by introducing a positioning pin-positioning hole assembly reference structure between the X-axis slide and the Y-axis linear guide, the installation position of the Y-axis component is constrained with high precision, avoiding orthogonality deviation and cumulative motion trajectory errors caused by assembly errors. Furthermore, the local dynamic haptic module 2 is fixed by precisely aligning with the geometric center of the Y-axis slide, ensuring a high degree of consistency between the physical motion coordinates of the haptic output unit and the visual and haptic mapping coordinates within the system. This structurally guarantees the spatial consistency and interactive stability of the haptic display system during operation across a large format.

[0046] For example, the X-axis guide rail assembly of the two-dimensional motion platform 1 is first installed on the equipment base. The X-axis stepper motor is fixed to one end of the base, and its output shaft is connected to the X-axis ball screw through a coupling. When the stepper motor rotates according to the control command, the ball screw drives the X-axis slide, which is threaded to it, to make precise linear motion along the X-axis linear guide rail direction. Subsequently, the Y-axis guide rail assembly is installed on the X-axis slide, so that the Y-axis linear guide rail moves synchronously with the X-axis slide. The Y-axis stepper motor also drives the Y-axis ball screw through a coupling, so that the Y-axis slide moves in a direction perpendicular to the X-axis, thereby achieving precise arrival at any two-dimensional coordinate position during actual operation. To ensure assembly accuracy and long-term operational stability, a positioning pin hole is set at the bottom of the Y-axis linear guide rail, which cooperates with the positioning pin set on the upper surface of the X-axis slide, so that the installation position error of the Y-axis assembly is controlled within ±0.05mm, effectively avoiding motion jitter or trajectory deviation caused by guide rail offset. Furthermore, the local dynamic haptic module 2 is aligned with the threaded hole on the Y-axis slide via a countersunk hole on the bottom of its housing, and is secured with an M2 screw, ensuring that the deviation between the geometric center of the haptic module and the center of the Y-axis slide does not exceed 0.1mm. In this embodiment, when the system drives the two-dimensional motion platform 1 to move according to the user's fingertip position, the local dynamic haptic module 2 can stably and accurately follow the target position, thereby achieving a haptic feedback effect highly consistent with the visual display during actual haptic interaction.

[0047] Specifically, the local dynamic haptic module 2 is mounted on the upper surface of the Y-axis guide rail assembly using a screw fastening method. The local dynamic haptic module 2 is configured with a 16×16 array of piezoelectric ceramic driving units 5, a contact post array 6 covering the piezoelectric ceramic driving unit array 5, a limiting plate for limiting the vertical travel of the contact posts, and a printed circuit board 7 integrating drive signal decoding and voltage conversion functions. The top of each piezoelectric ceramic driving unit is bonded to the bottom of a contact post, the contact post passes through a through hole in the limiting plate, and the limiting plate is fixed above the printed circuit board 7 by studs.

[0048] Understandably, the local dynamic tactile module 2 highly integrates the piezoelectric ceramic driving unit array 5, the contact post array 6, the limiting plate, and the driving circuit to construct a micro tactile display structure capable of achieving high-density, controllable tactile output within a local area. This structure utilizes the inverse piezoelectric effect generated by the piezoelectric ceramic material under an applied voltage to convert electrical signals into minute but high-speed mechanical displacements. Specifically, the 16×16 array of piezoelectric ceramic driving units, under the unified control of the printed circuit board 7, can operate independently or in combination based on the decoding results of the driving signals, thereby forming a spatially adjustable tactile excitation mode within the array plane. The top of each piezoelectric ceramic driving unit is rigidly bonded to the bottom of the corresponding contact post, allowing the axial deformation of the piezoelectric ceramic to be directly transmitted to the contact post, driving the contact post to move vertically. The limiting plate, through through holes corresponding to each contact post, guides and constrains the movement trajectory of the contact post and limits its maximum stroke range, thereby preventing structural fatigue or unstable tactile output due to excessive displacement of the contact post. By fixing the limiting plate above the printed circuit board 7 and maintaining a stable relative position with the piezoelectric ceramic drive unit, the entire tactile module can maintain good structural consistency and tactile output repeatability under high-speed vibration and long-term working conditions.

[0049] For example, the local dynamic haptic module 2 is mounted on the upper surface of the Y-axis guide rail assembly using screws, allowing it to move in the plane with the two-dimensional motion platform 1. The haptic module internally houses a 16×16 array of piezoelectric ceramic drive units, which are soldered and fixed to a printed circuit board 7. The printed circuit board 7 integrates a drive signal decoding circuit and a voltage conversion circuit, used to convert the digital haptic commands output from the upper control unit into drive voltage signals with corresponding amplitude and timing. In this embodiment, the top of each piezoelectric ceramic drive unit is bonded to the bottom of a contact post using high-strength adhesive. The contact post passes vertically through a through-hole in a limiting plate and extends upwards to the user-accessible area. The limiting plate is fixed above the printed circuit board 7 by studs, and the distance between the limiting plate and the printed circuit board 7 limits the maximum lifting height of the contact post. When the control system sends a specific tactile drive command sequence to the printed circuit board 7, the selected piezoelectric ceramic drive unit undergoes axial deformation, causing the corresponding contact post to rise or vibrate synchronously, thereby creating a tactile feedback effect corresponding to the virtual image or interactive content when the user's finger touches a local area.

[0050] Specifically, the vision sensing unit 3 includes at least one industrial camera with a USB 3.0 interface. The industrial camera is vertically mounted on a rigid bracket directly above the two-dimensional motion platform 1. The vertical distance between the industrial camera and the working surface of the platform is 500mm, and the optical axis of the industrial camera lens coincides with the normal direction of the working surface of the two-dimensional motion platform 1.

[0051] Specifically, the industrial camera has a lens focal length of 2.1mm, a field of view of 90°, and a coverage area of ​​600mm×600mm when the working distance is 500mm; the industrial camera outputs 30 frames of RGB images per second and sends them to the central control unit 4.

[0052] Understandably, the visual sensing unit 3 achieves stable and accurate visual acquisition of the fingertip position and movement trajectory within the platform's working area by arranging the industrial camera directly above the two-dimensional motion platform 1 with strict geometric constraints. This is achieved through vertical top-down imaging, which eliminates the impact of perspective distortion on planar coordinate calculation. Specifically, the industrial camera is mounted on a rigid bracket, aligning its lens optical axis with the normal direction of the working surface of the two-dimensional motion platform 1. This creates an approximately linear mapping between the pixel coordinates in the acquired image and the platform's physical coordinates, reducing the complexity of subsequent coordinate calibration and transformation calculations. By fixing the distance between the camera and the working surface at 500mm and combining specific focal length and field-of-view parameters, the camera ensures sufficient spatial resolution while completely covering the effective working area of ​​the two-dimensional motion platform 1, avoiding imaging blind spots or excessive edge distortion. The industrial camera continuously outputs high-frame-rate RGB images to the central control unit 4 via a high-speed USB 3.0 interface, enabling the system to capture subtle changes in fingertip movement in real time during dynamic interaction. This provides a reliable and low-latency visual input foundation for subsequent coordinate calculation, filtering, and synchronous tracking by the haptic module.

[0053] For example, the visual sensing unit 3 uses an industrial camera with a USB 3.0 high-speed interface, which is rigidly mounted above the two-dimensional motion platform 1, ensuring that the camera remains perpendicular to the platform's working surface. This industrial camera uses a 2.1mm lens, forming a field of view of approximately 90° at a working distance of 500mm, thus covering a planar area of ​​approximately 600mm × 600mm in actual imaging, fully covering the predetermined operating range of the two-dimensional motion platform 1. In this embodiment, the industrial camera continuously acquires RGB color images at a rate of 30 frames per second and transmits them in real-time to the central control unit 4 via the USB 3.0 interface. The central control unit 4 performs fingertip recognition and pixel coordinate extraction processing on the received image data, thereby driving the two-dimensional motion platform 1 and the local dynamic tactile module 2 to move synchronously. When the user performs continuous touch or swipe operations within the platform's working area, the visual sensing unit 3 can stably acquire information about changes in fingertip position, providing a practical example for the system to achieve low-latency, high-consistency visual-tactile linkage display.

[0054] Specifically, the central control unit 4 is connected to the drivers corresponding to the X-axis guide rail assembly and the Y-axis guide rail assembly via RS485 bus. The central control unit 4 is also connected to the printed circuit board 7 of the local dynamic tactile module 2 via SPI bus. The central control unit 4 is also connected to the visual sensing unit 3 via USB 3.0 interface.

[0055] Specifically, the synchronization scheduler of the central control unit 4 adopts a time-slice polling mechanism, and executes the following sequentially in each 10ms cycle: receiving visual data, running Kalman filtering, calculating the target platform position, generating motor pulse commands, extracting local image regions, generating tactile commands, and sending control signals; parallel processing between tasks is achieved through hardware interrupts and DMA transfer.

[0056] Specifically, when the central control unit 4 detects that the fingertip touches the image boundary, it also includes: the central control unit 4 is also used to send a deceleration command to the driver of the X-axis guide rail assembly and the Y-axis guide rail assembly to reduce the platform movement speed to 0 until the fingertip coordinates return to the inside of the effective area, wherein: the central control unit 4 is also used to maintain the current touch state of the local dynamic haptic module 2 unchanged.

[0057] Specifically, the central control unit 4 includes: an image preprocessing unit, used to perform one-time multi-resolution downsampling processing on the virtual graphics after loading the static virtual graphics to be displayed, construct a Mipmap image pyramid, and generate multiple image layers including the original resolution image and its progressively halved resolution images; a coordinate calculation unit, electrically connected to the image preprocessing unit, used to receive the original pixel coordinates of the fingertip output by the visual sensing unit 3, and process the original pixel coordinates based on Kalman filtering to output the corresponding smooth physical coordinates; and a mapping calculation unit, electrically connected to the coordinate calculation unit, used to calculate the corresponding smooth physical coordinates based on a preset scaling factor. The image placement offset is used to smooth the physical coordinates and map them to the virtual graphics coordinate system, generating the corresponding pixel coordinates and performing boundary constraint processing on the pixel coordinates. The rendering scheduling unit is electrically connected to the mapping calculation unit. The rendering scheduling unit is used to determine the pixel size of the target sampling window based on the physical size and scaling factor of the local dynamic haptic module 2, and select the image level that matches the target sampling window size from the Mipmap image pyramid as the source image, and extract the local image region centered on the pixel coordinates. The rendering scheduling unit is also used to resample and threshold the local image region to generate a haptic command matrix for driving the local dynamic haptic module 2.

[0058] Specifically, when the mapping calculation unit performs coordinate transformation, it includes: after receiving the physical coordinates of the fingertip, the mapping calculation unit performs a linear transformation on the physical coordinates according to a preset scaling factor and image placement offset to obtain the virtual image pixel coordinates corresponding to the physical coordinates of the fingertip, wherein: the mapping calculation unit also performs boundary restriction processing on the pixel coordinates along the horizontal and vertical directions respectively, wherein: when the pixel coordinates exceed the effective range of the virtual graphic, the mapping calculation unit restricts them to the boundary value in the corresponding direction; when the pixel coordinates do not exceed the effective range of the virtual graphic, the mapping calculation unit rounds them to the nearest integer pixel coordinates; when the pixel coordinates in either direction are subject to boundary restriction, it is determined that the fingertip has touched the edge of the virtual graphic and generates an edge feedback signal. The edge feedback signal is used to control the local dynamic haptic module 2 to output a pulse vibration mode with a preset frequency and duration to provide the user with haptic boundary prompts.

[0059] Specifically, when selecting a Mipmap level, the rendering scheduling unit includes: determining the target pixel sampling window size corresponding to the current haptic display area based on the physical size of the local dynamic haptic module 2 and a preset coordinate scaling factor, and imposing a minimum pixel constraint on the sampling window size; traversing the image sizes of each resolution level in the Mipmap image pyramid, and selecting the resolution level with the smallest deviation as the source image for current sampling based on the size deviation between the image size of each level and the target sampling window size; extracting a local image region centered on the currently mapped fingertip pixel coordinates and with a size corresponding to the target sampling window from the source image of the selected resolution level; and resampling the local image region, mapping it to a grayscale matrix of the same size as the touch column array 6, and performing thresholding on the grayscale matrix to generate a haptic drive command sequence for controlling the lifting and lowering state of the touch columns.

[0060] Understandably, the central control unit 4, acting as the core collaborative hub between visual acquisition, motion control, and tactile rendering, achieves low-latency closed-loop collaboration of visual perception, position calculation, motion control, and tactile output through a multi-bus heterogeneous communication architecture and a time-deterministic scheduling mechanism. Specifically, the central control unit 4 establishes stable industrial-grade communication connections with the drivers of the X-axis and Y-axis guide rail components via RS485 buses to send highly reliable motion control commands. Simultaneously, it engages in high-speed, low-latency data interaction with the printed circuit board 7 of the local dynamic tactile module 2 via an SPI bus to meet the parallel driving requirements of the high-density tactile array. Furthermore, it receives high-frame-rate image data output from the visual sensing unit 3 via a USB 3.0 interface, thereby constructing a control foundation for synchronous input of multi-source information. Building upon this foundation, the synchronization scheduler within the central control unit 4 employs a fixed-time-slice polling mechanism. Within a preset 10ms control cycle, it sequentially completes key tasks such as visual data reception, Kalman filtering, target motion position calculation, motor pulse command generation, local image region extraction, and tactile command generation. Parallel data transfer and processing between tasks are achieved through hardware interrupts and DMA transfer, reducing computational latency and jitter at the system architecture level. When the mapping calculation unit determines that the fingertip touches the virtual graphic boundary, the central control unit 4 further coordinates motion control and tactile output strategies, sending a deceleration command to the guide rail driver to reduce the platform's movement speed while maintaining the current touch state of the local dynamic tactile module 2, thus avoiding discontinuous interaction caused by sudden braking or tactile abrupt changes. By incorporating image preprocessing, coordinate calculation, mapping calculation, and rendering scheduling functions within the central control unit 4, and introducing a multi-resolution Mipmap image pyramid and adaptive sampling window matching mechanism, the system can consistently select image data of appropriate resolution for tactile rendering under different tactile display scales and motion speeds, ensuring consistency in tactile details while maintaining real-time performance and computational efficiency.

[0061] For example, the central control unit 4 uses an embedded control platform with multiple communication interfaces. Its RS485 interface is connected to the corresponding drivers of the X-axis guide rail assembly and the Y-axis guide rail assembly, respectively, for periodically issuing position and speed control commands; its SPI interface is connected to the printed circuit board 7 of the local dynamic tactile module 2, for high-speed transmission of a 16×16 scale tactile drive command matrix; and its USB3.0 interface is connected to the industrial camera installed on the two-dimensional motion platform 1, receiving RGB image data at a rate of 30 frames per second. In this embodiment, the synchronization scheduler inside the central control unit 4 completes fingertip image data acquisition, pixel coordinate extraction, and Kalman filtering smoothing in each complete control cycle of 10ms. It then calculates the target position of the two-dimensional motion platform 1 based on the filtered fingertip physical coordinates and generates corresponding motor pulse commands to send to the guide rail driver. At the same time, the central control unit 4 selects the image level that best matches the current tactile display area size from the pre-constructed Mipmap image pyramid, extracts the local image region centered on the fingertip mapped pixel coordinates, and resamples the local image region into a grayscale matrix consistent with the touch column array 6. After thresholding, it generates a tactile drive command sequence and sends it to the local dynamic tactile module 2 through the SPI interface. When the user's fingertip moves to the edge of the virtual graphic, the central control unit 4 detects that the pixel coordinates are limited by the boundary and triggers the edge feedback signal. On the one hand, it controls the local dynamic haptic module 2 to output a preset pulse vibration mode to indicate the boundary position. On the other hand, it sends deceleration commands to the X-axis and Y-axis guide rail drivers to gradually reduce the platform's movement speed from the normal operating speed to a lower value and maintain the current state of the contact post unchanged until the fingertip returns to the effective area of ​​the virtual graphic. Thus, in this embodiment, a smooth and safe human-computer interaction experience with highly consistent visual, motion and haptic feedback is achieved.

[0062] To enable those skilled in the art to fully understand and implement this invention, the specific implementation principles of this invention are further supplemented below with a specific application scenario.

[0063] Step 1: System Initialization and Image Preprocessing. The central control unit 4 first loads a static grayscale map image with a resolution of 1024×1024 pixels from the external storage medium as the virtual graphic to be displayed. Then, it calls the embedded Mipmap image pyramid generation module to perform a one-time downsampling process on the image, generating four low-resolution levels in sequence: 512×512 (k=1), 256×256 (k=2), 128×128 (k=3), and 64×64 (k=4). Together with the original image, these levels form a five-layer Mipmap pyramid structure. All level data is completely written into the memory of the central control unit 4, providing a multi-scale image source for subsequent real-time local rendering. This preprocessing is only performed once when the system starts or a new graphic is switched, avoiding repeated calculations during interaction, thereby freeing up CPU resources for coordinate calculation and motion control.

[0064] Step 2, the fingertip position perception and coordinate smoothing stage: The industrial camera in the visual sensing unit 3 continuously acquires RGB images of the area above the working surface of the two-dimensional motion platform 1 at 30 frames per second, and transmits the image stream to the central control unit 4 via the USB 3.0 interface; the central control unit 4 extracts the image brightness channel, performs binarization segmentation on the image using a preset skin color threshold range [100, 180], and then performs morphological closing operation on a 3×3 circular structuring element to eliminate noise holes. Finally, the original pixel coordinates (uraw, vraw) of the fingertip in the image coordinate system are calculated by the centroid algorithm; this coordinate sequence is fed into a Kalman filter, which establishes a dynamic model based on the state vector xk=[u,v,udot,vdot]^T, fuses historical trajectories with current observations, and outputs smooth physical coordinates (x_smooth, y_smooth) with a high signal-to-noise ratio in millimeters, effectively suppressing coordinate jumps caused by changes in lighting, finger tremors, or image noise, ensuring the stability of subsequent platform motion and haptic rendering.

[0065] Step 3, Coordinate Mapping and Edge Boundary Processing Stage: The mapping calculation unit, based on the system's preset scaling factor αfit = 2.0 pixels / mm and image offset (xoff = 50mm, yoff = 50mm), converts the smoothed physical coordinates into floating-point pixel coordinates in the virtual image: u =(xsmooth-50)×2.0, v=(ysmooth-50)×2.0; then perform boundary clamping operations on u and v respectively. If u<0, set uclamped=0; if u>1023, set uclamped=1023; otherwise, round to the nearest integer. The v coordinate is processed in the same way. When any coordinate is clamped to the boundary value, it is determined that the user's fingertip has touched the edge of the virtual image. At this time, the central control unit 4 immediately generates an edge feedback signal. On the one hand, it sends a unified vibration command to the printed circuit board 7 through the SPI bus, driving all 256 piezoelectric ceramic drive unit arrays 5 to synchronously output a 200Hz square wave voltage for 100ms, so that the contact column array 6 as a whole generates a perceptible pulse vibration. On the other hand, it sends a speed limit command to the X-axis and Y-axis stepper motor drivers through the RS485 bus, forcibly reducing the maximum moving speed of the platform to 0, preventing the user from quickly sliding out of the effective area due to the lack of tactile prompts, while maintaining the current contact state of the local dynamic tactile module 2 unchanged, avoiding cognitive confusion caused by sudden changes in the graphic.

[0066] Step 4: Multi-scale local image sampling and haptic command generation stage. The rendering scheduling unit calculates the required sampling window size as Wpix=Hpix=round(32×2.0)=64 pixels based on the physical dimensions of the local dynamic haptic module 2 (Fw=Fh=32mm) and the scaling factor α_fit=2.0 pixel / mm. Then, it iterates through the resolution (Wk,Hk) of each level of the Mipmap pyramid, calculating the sum of the absolute size deviations between each level and the target window: |Wk-64|+|Hk-64|. The level with the smallest deviation is selected as the sampling source. In this example, level k=4 (64×64) has zero deviation and is therefore selected as the optimal level. The system extracts a 64×64 pixel rectangular region centered at (uclamped,vclamped) from the image at this level. If the region exceeds the image boundary, a mirror filling strategy is used to fill in the missing pixels. Subsequently, this region... The target matrix is ​​resampled to 16×16 using a bilinear interpolation algorithm. Each output pixel is obtained by averaging four neighboring source pixels by distance, preserving local grayscale transition features. Finally, the 16×16 grayscale matrix is ​​binarized with a threshold of 0.5 to generate a command matrix corresponding to the lifting and lowering states of the contact post: when the value is ≥0.5, the printed circuit board 7 drives the corresponding piezoelectric ceramic drive unit 5 to apply a 120V voltage, causing it to extend by 0.2mm and push the contact post to rise; when the value is <0.5, the voltage returns to zero, and the contact post falls back under its own weight and the constraint of the limiting plate. This command matrix is ​​transmitted to the printed circuit board 7 via the SPI bus at a rate of 1Mbps, where its local microcontroller parses and drives the synchronous output of 256 high-voltage channels.

[0067] Step 5: Platform-Haptic Synchronous Closed-Loop Control Stage. Within each 10ms control cycle, the synchronization scheduler first receives a new image frame via USBDMA, then completes Kalman filtering and coordinate mapping. Simultaneously, it calculates the target position of the local dynamic haptic module 2 based on (xsmooth, ysmooth), requiring the module's geometric center to coincide with the physical position of the fingertip, thus generating target coordinates on the X and Y axes. These target coordinates, after being calculated by the position loop, are converted into pulse quantity and direction signals, which are sent to the X and Y axis stepper motor drivers via the RS485 bus, driving the ball screw to move the Y-axis slide and... The onboard local dynamic haptic module 2 moves precisely; at the same time, the rendering scheduling unit completes local image extraction, resampling and binarization in parallel, and sends haptic commands to the printed circuit board 7 through SPIDMA; since image reception, filtering calculation, motor control and haptic refresh are all decoupled and parallelized through hardware interrupt and DMA mechanism, the end-to-end latency of the entire perception-decision-execution link is strictly controlled within 20ms, ensuring that when the user moves his finger, the local dynamic haptic module 2 is always located directly below the fingertip and presents the graphic details corresponding to the current position in real time, realizing a continuous haptic browsing experience of "what you touch is what you see".

[0068] In the above embodiments, by setting a local dynamic haptic module 2 on the two-dimensional motion platform 1, the haptic array moves with the platform within a planar range, thereby achieving continuous haptic coverage of a large-format virtual graphic area without the need for a large-size overall haptic array. Compared with a fixed large-area haptic array solution, this invention only uses a 16×16 array of local haptic units to complete a large-area haptic display, effectively reducing the number of touch points and driving channels, lowering system manufacturing costs and power consumption, and improving system scalability. By arranging an industrial camera vertically above the two-dimensional motion platform 1, the visual sensing unit 3 can acquire real-time information on the positional changes of the user's fingertips within the working area, which is then processed and tracked by the central control unit 4. This ensures that the local dynamic haptic module 2 always moves to the area corresponding to the fingertip position, thereby guaranteeing the spatial consistency between the haptic feedback area and the virtual graphics, avoiding haptic delay or misalignment, and enhancing the realism and immersion of the interaction. The local dynamic tactile module 2 adopts a one-to-one structure of piezoelectric ceramic drive unit array 5 and contact column array 6, and constrains the vertical stroke of the contact columns by a limiting plate, so that each contact can independently and stably generate lifting or vibration, thereby forming a high-density and controllable tactile output in a local area, which is conducive to accurately presenting the edge and detail features of virtual graphics. At the same time, the central control unit 4 communicates with the two-dimensional motion platform 1, the local dynamic tactile module 2 and the vision sensing unit 3 through RS485 bus, SPI bus and USB3.0 interface respectively, so that motion control, tactile drive and vision data transmission each adopt a matching communication method, thereby improving the overall stability and real-time response capability of the system architecture.

[0069] In another preferred embodiment based on the above embodiments, such as Figure 5 As shown, this embodiment provides a method for large-format graphic haptic display based on visual tracking and dynamic following of a haptic array, including: Step S100: After loading the static virtual graphics, construct a Mipmap image pyramid containing five resolution levels.

[0070] Step S200: Continuously acquire the original coordinates of the user's fingertip in the platform work area, and output smooth coordinates after Kalman filtering.

[0071] Step S300: Based on the preset scaling factor and offset, map the smooth coordinates to the pixel coordinates of the virtual image and perform boundary truncation.

[0072] Step S400: Based on the physical size of the haptic module and the scaling factor, obtain the required pixel window size and select the optimal level from the Mipmap pyramid.

[0073] Step S500: Extract sub-images centered on the mapped coordinates from the selected layer, resample them into a 16×16 matrix, and then drive the haptic module.

[0074] Step S600: Synchronously control the movement of the two-dimensional motion platform so that the tactile module is always located directly below the fingertip.

[0075] It is understood that the large-format tactile graphic display method and system in the above embodiments of the present invention have the same beneficial effects, and will not be described again.

[0076] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A large-format haptic graphic display system, characterized in that, include: A two-dimensional motion platform is used to provide two-dimensional motion within a planar range to carry and drive a local dynamic haptic module to move within a working area; the two-dimensional motion platform can be composed of mutually orthogonal X-axis motion components and Y-axis motion components; A local dynamic tactile module, installed on the two-dimensional motion platform, includes a tactile array composed of multiple tactile units and a corresponding driving circuit, used to output tactile feedback in a local area; the tactile array can adopt an m×n array arrangement of tactile units, and the tactile units can be piezoelectric driven, electromagnetic driven, shape memory alloy driven, or other driving forms that can realize tactile output; The visual sensing unit is used to collect image information of user touch behavior in the work area. It can be an image acquisition device such as an industrial camera set at a preset position in the work area to obtain image information of the user's fingertip / touch position. It is preferably set above the two-dimensional motion platform and the optical axis is approximately perpendicular to the work surface to realize the observation of the fingertip / touch position. The central control unit is connected to the two-dimensional motion platform, the local dynamic tactile module, and the visual sensing unit, respectively, and is used to uniformly schedule and control the visual data processing, local graphic extraction, tactile rendering output, and platform motion.

2. The large-format haptic graphic display system as claimed in claim 1, characterized in that, The central control unit includes: The image preprocessing unit is used to perform step-by-step downsampling processing on the virtual graphics after loading the static virtual graphics to be displayed, construct the Mipmap image pyramid, and generate multiple image layers including the original resolution image and its step-by-step halved resolution images; The coordinate calculation unit is used to receive the original position information of the fingertip / touch position output by the visual sensing unit, and to calculate and stabilize the position to output the corresponding smooth physical coordinates. The mapping calculation unit is electrically connected to the image preprocessing unit and the coordinate calculation unit. The mapping calculation unit is used to map the smooth physical coordinates to the virtual graphic coordinate system according to the preset scaling factor and image placement offset, generate the corresponding pixel coordinates, and perform boundary constraint processing on the pixel coordinates. The rendering scheduling unit is electrically connected to the mapping calculation unit. The rendering scheduling unit is used to determine the pixel size of the target sampling window based on the physical size and scaling factor of the local dynamic haptic module, and select the image level that matches the target sampling window size from the Mipmap image pyramid as the source image, and extract the local image region centered on the pixel coordinates; the rendering scheduling unit is also used to resample and threshold the local image region to generate a haptic instruction sequence for driving the local dynamic haptic module; The motion control unit is used to calculate the target position of the two-dimensional motion platform based on the fingertip / touch position information and generate platform drive control commands so that the local dynamic haptic module moves with the touch position.

3. The large-format haptic graphic display system as described in claim 2, characterized in that, When the mapping calculation unit performs coordinate transformation, it includes: After receiving the physical coordinates of the fingertip, the mapping calculation unit performs a linear transformation on the physical coordinates according to a preset scaling factor and image placement offset to obtain the virtual image pixel coordinates corresponding to the fingertip physical coordinates, where: The mapping calculation unit is also used to perform boundary constraint processing on pixel coordinates along the horizontal and vertical directions, respectively, where: When pixel coordinates exceed the effective range of the virtual graphics, the mapping calculation unit restricts them to boundary values ​​in the corresponding direction; When the pixel coordinates do not exceed the valid range of the virtual graphics, the mapping calculation unit rounds them to the nearest integer pixel coordinates. When the pixel coordinates in any direction reach the boundary, it is determined that the fingertip has touched the edge of the virtual graphic, and an edge feedback signal is generated to provide the user with tactile boundary prompts.

4. The large-format haptic graphic display system as claimed in claim 3, characterized in that, When selecting a Mipmap level, the rendering scheduling unit includes: The rendering scheduling unit is also used to determine the target pixel sampling window size corresponding to the current haptic display area based on the physical size of the local dynamic haptic module and the preset coordinate scaling factor, and to impose a minimum pixel constraint on the sampling window size; The rendering scheduling unit is also used to traverse the image size of each resolution level in the Mipmap image pyramid, and select the resolution level with the smallest deviation as the source image for the current sampling based on the size deviation between the image size of each level and the target sampling window size. The rendering scheduling unit is also used to extract a local image region from the source image at the selected resolution level, centered on the currently mapped fingertip pixel coordinates and with a size corresponding to the target sampling window; The rendering scheduling unit is also used to resample local image regions, map them into grayscale matrices of the same size as the touch column array, and perform thresholding on the grayscale matrix to generate a haptic drive instruction sequence for controlling the lifting and lowering state of the touch columns.

5. The large-format haptic graphic display system as claimed in claim 3, characterized in that, The central control unit also includes the following when it detects a fingertip touching the image boundary: The central control unit is also used to send deceleration commands to the two-dimensional motion platform driver, reducing the platform's moving speed to 0 until the fingertip coordinates return to the effective area, wherein: The central control unit is also used to maintain the current touch state of the local dynamic haptic module.

6. The large-format haptic graphic display system as claimed in claim 1, characterized in that, The camera module can cover the haptic graphics display area when in the working position.

7. The large-format haptic graphic display system as claimed in claim 2, characterized in that, The central control unit's synchronous scheduler uses a time-slice polling mechanism to perform the following tasks in each work cycle: receiving visual data, position calculation and stabilization processing, calculating the target location of the computing platform, generating two-dimensional platform positioning instructions, extracting local image regions, generating tactile instructions, and sending control signals. Parallel processing between tasks is achieved through hardware interrupts and DMA transfer.

8. A method for displaying large-format haptic graphics, applicable to the large-format haptic graphics display system as described in any one of claims 1-7, characterized in that, include: S1: Load static virtual graphics and preprocess the graphics data to obtain multi-scale / multi-resolution representation; S2: Collect user touch behavior image information, identify and obtain the user's fingertip / touch position, perform position calculation and stabilization processing, and obtain smooth position coordinates; S3: Based on the preset scaling factor and offset, map the smooth coordinates to the pixel coordinates of the virtual image and perform boundary truncation; S4: Based on the physical size of the haptic module and the scaling factor, obtain the required pixel window size, and extract the local graphic data centered on the mapped coordinates from the optimal level; S5: Render local graphic data into haptic commands that match the haptic array, driving the local dynamic haptic module to output haptic feedback; S6: Synchronously control the movement of the two-dimensional motion platform so that the haptic module always follows the fingertip / touch position; S7: Repeatedly execute S2 to S6 to achieve continuous tactile display of large-format graphics.

Images