A calligraphy and painting appreciation experience cabin, interactive method and device

Abstract

This invention belongs to the field of human-computer interaction technology. It discloses a calligraphy and painting appreciation experience cabin, an interaction method, and a device. Through a immersive digital experience cabin, a somatosensory gesture interaction system collects the user's hand spatial motion data stream. Combined with ultra-high-definition original image layers and pre-annotated semantic partition maps of the image, a precise appreciation interaction experience is achieved. Based on the spatial curvature sequence and velocity decay characteristics of gesture trajectories, user behavior is classified into three trajectory types: global exploration, regional wandering, and fixed-point approach. Gestures are associated with semantic partitions of the image through projection mapping, constructing a two-dimensional intent index key to retrieve matching professional content. The method also includes multi-scale cropping and progressive detail enhancement techniques, allowing image details to gradually appear as the user's attention increases, and achieving a smooth transition of intent conversion through spatial deviation rate monitoring. This invention achieves a natural, intelligent, and immersive calligraphy and painting appreciation experience.

Description

Technical Field

[0001] This invention relates to the field of human-computer interaction technology, and more specifically, to a calligraphy and painting appreciation experience cabin, interaction method, and device. Background Technology

[0002] Existing digital art appreciation systems generally suffer from a disconnect between interactive experience and actual appreciation behavior. In traditional exhibition environments, viewers can freely adjust viewing distance, change angles, and focus on details, while digital systems are limited to mechanical operations such as touchscreen clicking or mouse dragging. This disconnect prevents professional connoisseurs from immersing themselves in the experience. Current technology lacks the ability to accurately identify appreciation intentions, failing to distinguish whether a user is grasping the overall aesthetic, exploring local techniques, or examining detailed features, resulting in a significant discrepancy between system response and user expectations. In practical applications, when a viewer attempts to closely examine a seal, the system may misinterpret it as ordinary browsing, providing a general overview rather than professional seal carving analysis. Furthermore, existing systems lack structured semantic understanding of artworks, failing to automatically identify specialized sections such as inscription areas, seal areas, thematic content areas, and colophons, making the interaction process untargeted. At the content response level, they cannot provide hierarchical professional content based on different appreciation scenarios, leaving beginners feeling overwhelmed and experts feeling only scratching the surface. In terms of visual presentation, the lack of a gradual, detailed display mechanism often results in abrupt image transitions or pixelation when users switch between different viewing scales, disrupting the coherent visual experience. When the viewer's attention shifts from the distant view of a landscape painting to the details of trees and rocks in the foreground, the existing system often fails to transition smoothly, instead switching abruptly and losing the contextual coherence of spatial relationships. A deeper problem lies in the fact that current technology cannot simulate the layered experience of "viewing the grandeur from afar and examining the brushwork up close" in traditional appreciation, lacking the integration and presentation of professional knowledge. This leaves digital appreciation at a superficial level of visual reproduction, failing to delve into the essence of art and the transmission of cultural connotations.

[0003] In view of this, the present invention proposes a calligraphy and painting appreciation experience cabin, an interactive method, and a device to solve the above problems. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art and to achieve the above objectives, the present invention provides the following technical solution: an interactive method for appreciating calligraphy and painting, comprising: Step S1: Obtain the user's hand spatial motion data stream collected by the immersive digital experience cabin's body-sensing gesture interaction system, and load the ultra-high-definition original painting layer data of the calligraphy and painting works to be appreciated and the pre-annotated image semantic partition map. The image semantic partition map includes the boundary of the inscription area, the boundary of the seal area, the boundary of the brush and ink main body area, and the boundary of the colophon area. Step S2: The hand spatial motion data stream is divided into sliding segments according to time windows. The spatial curvature sequence and velocity decay features of the gesture motion trajectory within each time window are extracted. Based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay features, the trajectory classification result of the current gesture is determined. The trajectory classification results include global tour trajectory, regional wandering trajectory and fixed-point approach trajectory. Step S3: Project the spatial coordinates of the gesture movement trajectory onto the image coordinate system of the ultra-high-definition original image layer data, calculate the shortest distance sequence between the projected trajectory and the boundaries of each region in the image semantic partition map, and perform weighted attenuation on the shortest distance sequence in combination with the trajectory classification results to determine the current appreciation focus area and its semantic partition type. Step S4: Construct a two-dimensional intent index key based on the trajectory classification results and semantic partition type. Retrieve the appreciation response content package corresponding to the two-dimensional intent index key in the pre-set multi-level appreciation content library. The appreciation response content package contains local enhanced rendering data of the original painting, identification and character analysis data, or multi-point expert interpretation data that match the appreciation focus area. Step S5: Based on the position range of the appreciation focus area in the screen coordinate system and the content type of the appreciation response content package, calculate the visual focus offset and rendering level switching parameters of the immersive display screen. Perform multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data based on the visual focus offset. Overlay and render the appreciation response content package onto the enhanced screen layer to generate an immersive appreciation interactive screen. Step S6: During the immersive appreciation interaction screen output, continuously collect subsequent hand spatial motion data streams, calculate the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area, and when the spatial deviation rate exceeds the preset intention switching threshold, re-execute step S2 with the current screen state as the initial context.

[0005] An interactive device for appreciating calligraphy and painting, used to implement interactive methods for appreciating calligraphy and painting, includes: The interactive acquisition module is used to acquire the user's hand spatial motion data stream collected by the immersive digital experience cabin's body-sensing gesture interaction system, and load the ultra-high-definition original painting layer data of the calligraphy and painting works to be appreciated and the pre-annotated image semantic partition map. The image semantic partition map includes the boundary of the inscription area, the boundary of the seal area, the boundary of the brush and ink main body area, and the boundary of the colophon area. The trajectory analysis module is used to slide and segment the hand spatial motion data stream according to time windows, extract the spatial curvature sequence and velocity decay characteristics of the gesture motion trajectory within each time window, and determine the trajectory classification result of the current gesture based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay characteristics. The trajectory classification results include global tour trajectory, regional wandering trajectory and fixed-point approach trajectory. The focus area determination module is used to project the spatial coordinates of the gesture movement trajectory onto the screen coordinate system of the ultra-high-definition original image layer data, calculate the shortest distance sequence between the projected trajectory and the boundaries of each region in the semantic partition map of the screen, and perform weighted attenuation on the shortest distance sequence in combination with the trajectory classification results to determine the current appreciation focus area and its semantic partition type. The content retrieval module is used to construct a two-dimensional intent index key based on the trajectory classification results and semantic partition type. It retrieves the appreciation response content package corresponding to the two-dimensional intent index key in the pre-set multi-level appreciation content library. The appreciation response content package contains local enhanced rendering data of the original painting, identification and character analysis data, or multi-point expert interpretation data that match the appreciation focus area. The image rendering module is used to calculate the visual focus offset and rendering level switching parameters of the immersive display image based on the position range of the appreciation focus area in the image coordinate system and the content type of the appreciation response content package. Based on the visual focus offset, it performs multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data, and overlays and renders the appreciation response content package onto the enhanced image layer to generate an immersive appreciation interactive image. The interaction monitoring module is used to continuously collect subsequent hand spatial motion data streams during the immersive appreciation interaction screen output, calculate the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area, and when the spatial deviation rate exceeds the preset intention switching threshold, the trajectory analysis module is re-executed with the current screen state as the initial context.

[0006] A calligraphy and painting appreciation experience cabin includes: An immersive display system for displaying ultra-high-definition images of calligraphy and painting works; A motion-sensing gesture interaction system is used to collect data streams of spatial movement of the user's hands; The processing unit is connected to the immersive display system and the motion-sensing interaction system. The processing unit is used to execute the functions of the calligraphy and painting appreciation interactive device.

[0007] The technical effects and advantages of the calligraphy and painting appreciation experience cabin, interactive method and device of the present invention are as follows: This invention achieves a precise mapping mechanism between gesture intentions and content being appreciated, improving interaction accuracy and response matching. Through dual-dimensional correlation analysis of trajectory features and semantic partitioning, it can accurately distinguish and respond to different levels of appreciation behavior patterns, achieving full-spectrum intention recognition from macro-browsing to micro-details. The projection trajectory analysis mechanism based on the image semantic partitioning map eliminates the mapping barrier between user behavior and content areas, accurately perceiving the specific semantic units that the user focuses on. It also provides highly relevant professional analysis for different appreciation scenarios, achieving a continuous visual experience from global to local. It effectively captures the user's attention shift process, ensuring contextual coherence in multi-focus appreciation. Dynamically adjusting response parameters according to different users' operating habits improves interaction tolerance and user adaptability. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of an interactive method for appreciating calligraphy and painting according to the present invention; Figure 2 This is a schematic diagram of an interactive device for appreciating calligraphy and painting according to the present invention. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] This application example provides a calligraphy and painting appreciation experience cabin, an interactive method, and a device. Please refer to [link / reference]. Figure 1 In this embodiment of the invention, an interactive method for appreciating calligraphy and painting includes the following steps: Step S1 involves acquiring the user's hand spatial motion data stream collected by the immersive digital experience cabin's motion-sensing interaction system, and loading the ultra-high-definition original image layer data and pre-annotated semantic partition map of the artwork to be appreciated. The user's hand spatial motion data stream includes key information such as hand 3D coordinates, posture angle, and movement speed, acquired in real-time using an infrared depth camera and motion capture technology. The ultra-high-definition original image layer data preserves the fine brushstrokes, color gradations, and texture details of the artwork, typically consisting of lossless image data at a resolution of 16K or higher. The semantic partition map is a pre-annotated structural analysis diagram of the artwork, including the boundaries of the inscription area, seal area, main brush and ink area, and colophon area, used to accurately map the user's gesture intentions to the corresponding semantic areas of the image. This data provides the basic information source for subsequent appreciation and interaction, ensuring the accuracy of the interaction and the high quality of the image presentation.

[0011] Step S2 involves sliding segmenting the hand spatial motion data stream according to time windows, extracting the spatial curvature sequence and velocity decay features of the gesture trajectory within each time window, and determining the trajectory classification result of the current gesture based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay features. The length of the time window is typically set to 2-3 seconds to capture the complete expression of the gesture intent. The spatial curvature sequence describes the change in the curvature of the gesture movement path, while the velocity decay features quantify the velocity change trend of the gesture from start to finish. The periodic distribution pattern reflects the repetitiveness and regularity of the gesture, while the convergence trend reflects the target orientation of the gesture. The trajectory classification results include global browsing trajectory (fast, wide-range browsing gestures), regional wandering trajectory (reciprocating exploratory gestures within a specific area), and fixed-point approach trajectory (precise gestures that gradually decelerate and point towards a specific point). These three trajectory types correspond to the three main appreciation behavior modes of users in calligraphy and painting appreciation: macro-level browsing, local exploration, and detailed examination, respectively.

[0012] Step S3 involves projecting the spatial coordinates of the gesture trajectory onto the image coordinate system of the ultra-high-definition original image layer data. The shortest distance sequence between the projected trajectory and the boundaries of each region in the image semantic partition map is calculated. This shortest distance sequence is then weighted and attenuated based on the trajectory classification results to determine the current focus area and its corresponding semantic partition type. The spatial coordinate projection uses a perspective projection algorithm to map the three-dimensional gesture space to a two-dimensional image coordinate system, preserving the shape and proportions of the gesture trajectory. The shortest distance sequence calculation employs an improved Hausdorff distance algorithm to accurately quantify the spatial relationship between the projected trajectory and the boundaries of each semantic region. Weighted attenuation assigns different distance attenuation coefficients based on different trajectory types, allowing the system to flexibly adjust its sensitivity to distance factors according to the clarity of the user's gesture intent. The focus area is the user's current focus determined by the system, providing spatial positioning for subsequent content response and image rendering.

[0013] Step S4: Construct a two-dimensional intent index key based on the trajectory classification results and semantic partitioning type. Retrieve the corresponding appreciation response content package from the pre-built multi-level appreciation content library. The two-dimensional intent index key is a mapping mechanism that associates user gesture intent with the content area of ​​the screen. It is formed by encoding and concatenating the trajectory classification results and semantic partitioning type. The multi-level appreciation content library is a pre-established hierarchical knowledge base containing professional interpretation content for different appreciation intents and screen areas. The appreciation response content package has different content compositions depending on the different intent index keys, including original painting partial enhanced rendering data (providing clearer detail display), signature and character identification analysis data (textual interpretation and analysis of signatures and seals), or multi-point expert interpretation data (professional analysis of the composition, brushwork, and artistic features of the painting). This two-dimensional indexing mechanism can accurately match the user's current appreciation intent with the screen content, providing the most relevant appreciation assistance information.

[0014] Step S5: Based on the position range of the appreciation focus area in the screen coordinate system and the content type of the appreciation response content package, calculate the visual focus offset and rendering level switching parameters of the immersive display screen. Based on the visual focus offset, perform multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data. Then, overlay and render the appreciation response content package onto the enhanced image layer to generate an immersive appreciation interactive screen. The visual focus offset determines the center position of the screen display, while the rendering level switching parameters control the scaling and detail rendering level of the image. Multi-scale cropping technology extracts the optimal visual area from the original ultra-high-definition layer according to different attention ranges and detail levels. Progressive detail enhancement uses multi-level image processing algorithms to gradually reveal richer detail information as the user's attention increases. The overlay rendering of the appreciation response content package uses intelligent layer blending technology to ensure that the additional information is harmoniously unified with the original image, without compromising the artistic feel of the original work. The final generated immersive appreciation interactive screen retains the artistic authenticity of the original work while adding rich interactive appreciation auxiliary content.

[0015] Step S6 involves continuously collecting subsequent hand spatial motion data streams during the immersive viewing interaction screen output, calculating the spatial deviation rate between the subsequent gesture trajectory and the viewing focus area, and re-executing step S2 with the current screen state as the initial context when the spatial deviation rate exceeds a preset intent switching threshold. The spatial deviation rate is a key indicator for measuring user attention shift, calculated by the rate of change of distance between the center of the gesture trajectory and the geometric center of the current viewing focus area. The intent switching threshold is set to a moderate value, which can respond promptly to clear changes in user intent while avoiding frequent screen jumps caused by slight hand tremors. When the system recognizes a significant shift in user attention, it smoothly transitions to a new viewing focus using the current screen state as the context, ensuring the continuity and naturalness of the interactive experience and avoiding abrupt screen transitions that could cause visual discomfort to the user.

[0016] The various steps described above are connected through internal data flows to achieve an end-to-end interactive processing flow.

[0017] In this embodiment of the invention, the trajectory classification result of the current gesture is determined based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay characteristics, including: The variance of the spatial curvature sequence within a time window is calculated as an indicator of trajectory curvature fluctuation. The spatial curvature sequence is a key feature describing changes in the shape of a hand gesture trajectory, and its variance reflects the fluctuation in the degree of trajectory curvature. The calculation process first estimates the local curvature using the three-point method based on discrete sampling points of the hand position, and then calculates the statistical variance of the curvature sequence within the time window. The curvature calculation employs a numerical differentiation method. For a three-dimensional trajectory point sequence in space, the local curvature... The calculation formula is: ; in, For velocity vectors, The acceleration vector is calculated using finite differences between adjacent sampling points. The trajectory curvature fluctuation index directly reflects the complexity and frequency of change of the gesture, and is an important basis for distinguishing different trajectory types.

[0018] The deceleration duration is defined as the time it takes for the speed value to drop from its peak to below a preset speed threshold. Speed ​​decay describes the change in hand gesture movement from fast to slow and is a crucial clue for determining changes in user intent. The feature extraction process first identifies the speed peak within a time window, then tracks the speed curve until it drops below the preset threshold, recording the duration of this process. The deceleration duration reflects the transition from rapid movement to focused observation; a longer duration indicates a greater inclination for the user to fixate on a specific area.

[0019] If the trajectory curvature fluctuation index is below the first curvature threshold and the deceleration duration is shorter than the first duration threshold, the trajectory classification result is determined to be a global browsing trajectory. A global browsing trajectory typically exhibits relatively smooth (low curvature fluctuation) gestures with insignificant speed changes (short deceleration duration), reflecting the user's overall browsing intent. The first curvature threshold and the first duration threshold are determined based on user behavior statistical analysis, set as the 25th percentile of the curvature variance and the 20th percentile of the deceleration duration, respectively, to ensure accurate capture of the user's macroscopic browsing behavior.

[0020] If the trajectory curvature fluctuation index is between the first curvature threshold and the second curvature threshold, and the deceleration duration is between the first duration threshold and the second duration threshold, then the trajectory classification result is determined to be a regional wandering trajectory. A regional wandering trajectory exhibits moderate curvature fluctuations and a moderate deceleration process, reflecting the user's intention to repeatedly explore a specific area. The second curvature threshold and the second duration threshold are set to the 75th quantile of the curvature variance and the 70th quantile of the deceleration duration, respectively, defining a reasonable characteristic range for regional wandering trajectories.

[0021] If the trajectory curvature fluctuation index is higher than the second curvature threshold and the deceleration duration exceeds the second duration threshold, the trajectory is classified as a point-approaching trajectory. Point-approaching trajectories are characterized by significant curvature fluctuations (potentially including multiple directional adjustments) and a clear deceleration process (indicating a clear user focus), reflecting the user's intention to meticulously observe specific details. This judgment logic, through the combination of dual feature conditions, improves the accuracy and robustness of trajectory classification.

[0022] In this embodiment of the invention, the shortest distance sequence is weighted and attenuated based on the trajectory classification results to determine the current appreciation focus area and its semantic partition type, including: A first distance attenuation coefficient, a second distance attenuation coefficient, and a third distance attenuation coefficient are configured for global browsing trajectories, regional wandering trajectories, and fixed-point approach trajectories, respectively. The third distance attenuation coefficient is greater than the second distance attenuation coefficient, and the second distance attenuation coefficient is greater than the first distance attenuation coefficient. The distance attenuation coefficient is set based on the degree of precision required for different trajectory types. The third attenuation coefficient is the largest for fixed-point approach trajectories (usually set to 0.8-0.9), indicating a high level of trust in this type of precise positioning gesture. The second attenuation coefficient is in the middle for regional wandering trajectories (usually set to 0.5-0.7), indicating a moderate level of positioning trust. The first attenuation coefficient is the smallest for global browsing trajectories (usually set to 0.2-0.4), indicating a lower requirement for positioning accuracy for broad browsing gestures. This progressive attenuation coefficient design allows the system to adaptively adjust its sensitivity to spatial distance based on the clarity of the user's gestures.

[0023] The distance value corresponding to each semantic partition in the shortest distance sequence is multiplied by the distance decay coefficient corresponding to the current trajectory classification result to obtain the decayed distance value of each semantic partition. The shortest distance sequence records the minimum distance between the projected trajectory and the boundary of each semantic partition, and is a direct measure of the spatial relationship between the trajectory and the partition. The weighted decay process adjusts the distance metric by multiplying the original distance value by the corresponding decay coefficient, achieving a comprehensive evaluation combining gesture intent strength and spatial location. The formula for calculating the decayed distance value is: ; in, semantic partitioning The distance value after attenuation, This is the original shortest distance value. Trajectory type The corresponding distance attenuation coefficient. This weighted processing enables the system to more accurately locate the target area when the user's gestures are clear (such as a fixed-point approach trajectory), while maintaining a certain degree of fault tolerance when the gestures are ambiguous (such as a global browsing trajectory).

[0024] The semantic partition with the smallest distance value after attenuation is selected as the hit partition. The geometric intersection area between the boundary of the hit partition and the projection trajectory is determined as the appreciation focus area. The type label of the hit partition is determined as the semantic partition type. The selection of the hit partition follows the minimum distance principle, meaning that the semantic partition with the smallest distance value after attenuation is most likely the target currently of user attention. The calculation of the geometric intersection area uses computational geometry algorithms to accurately determine the overlapping part between the projection trajectory and the boundary of the hit partition, which serves as the final appreciation focus area. This precise positioning method not only considers the spatial position of the gesture trajectory but also incorporates the strength of the user's intent, greatly improving the accuracy and naturalness of the appreciation interaction.

[0025] In this embodiment of the invention, a two-dimensional intent index key is constructed based on the trajectory classification result and semantic partitioning type. The corresponding appreciation response content package is then retrieved from a pre-set multi-level appreciation content library, including: The trajectory classification results are encoded as first-dimensional index values, and the semantic partition type is encoded as a second-dimensional index value. The first-dimensional and second-dimensional index values ​​are then concatenated to generate a two-dimensional intent index key. The encoding process uses a hash mapping method, mapping trajectory classification results (global tour, regional wandering, fixed-point approach) to numerical indices (e.g., 1, 2, 3), and semantic partition types (inscription area, seal area, main ink area, colophon area) to numerical indices (e.g., 1, 2, 3, 4). The concatenation process uses a displacement combination method to ensure the uniqueness and resolvability of the generated two-dimensional intent index key. This encoding mechanism organically combines the user's behavioral intent (trajectory type) with the content of interest (semantic partition), forming a complete expression of interactive intent.

[0026] The system searches for content entries matching the two-dimensional intent index key in the multi-level appreciation content library's index table. Each content entry is associated with a content priority identifier and a content presentation method identifier. The multi-level appreciation content library serves as the system's knowledge repository, employing a hierarchical index structure to organize various appreciation and interpretation content. The index table utilizes an efficient hash lookup algorithm, ensuring millisecond-level response times. Content entries contain rich metadata; the content priority identifier determines the display order when multiple pieces of content are accessed concurrently, while the content presentation method identifier guides the system on how to render and display the corresponding content (such as text annotation, highlighting, expert audio narration, etc.). This multi-dimensional indexing mechanism ensures that the system can accurately extract the most relevant appreciation support information based on different user intents and concerns.

[0027] When the trajectory classification result is a fixed-point approach trajectory and the semantic partitioning type is a seal area, the identification and character analysis data in the appreciation response package includes the seal outline extraction result, character-by-character interpretation information of the seal inscription, and historical period information of the seal. This reflects the user's intention to meticulously examine the details of the seal, and the response content provided by the system emphasizes the seal's textual content, artistic features, and historical background. The outline extraction result is generated through image segmentation algorithms, the text interpretation information is based on seal engraving dictionaries and optical character recognition technology, and the period information is derived by combining seal style characteristics and historical data analysis. This professional information helps users deeply understand the artistic value and historical significance of the seal.

[0028] When the trajectory classification result is a regional wandering trajectory and the semantic partition type is the main area of ​​brush and ink, the multi-point expert interpretation data in the appreciation response content package includes explanations of brushwork techniques and compositional analysis corresponding to the appreciation focus area. This reflects the user's intention to explore the artistic expression of the painting, and the response content provided by the system focuses on professional analysis of artistic techniques and compositional principles. The brushwork technique explanations analyze the painter's brushwork characteristics and ink color variations, while the compositional analysis interprets the spatial structure and artistic layout of the painting. This professional interpretation content is pre-recorded by art history experts and precisely correlated with the areas of the painting, providing users with an authoritative perspective on art appreciation.

[0029] In this embodiment of the invention, multi-scale cropping and progressive detail enhancement are performed on the ultra-high-definition original image layer data based on the visual focus offset, including: The absolute coordinates of the cropping center point in the ultra-high-definition original image layer data are determined based on the visual focus offset. The visual focus offset is the displacement vector of the geometric center of the viewing focus area relative to the center of the currently displayed screen. The absolute coordinates of the cropping center point are calculated through coordinate transformation. The transformation process takes into account the current display ratio and screen offset state to ensure that the cropping center point accurately corresponds to the area of ​​the screen that the user is focusing on. This precise positioning mechanism is the foundation for achieving natural screen transitions, enabling the system to accurately capture the user's focus.

[0030] The current image scaling ratio is calculated based on the rendering level switching parameters. The size range of the cropping window is then determined with the cropping center point as the geometric center, based on this scaling ratio. The rendering level switching parameters reflect the level of detail the user is focusing on and are determined by the trajectory type and user interaction history. The image scaling ratio is a continuous variable, smoothly transitioning from the global view (smallest scaling ratio) to the local detail view (largest scaling ratio). The formula for calculating the size of the cropping window is: ; in, and To cut the width and height of the window, and For the physical resolution of the display device, This is the scaling factor. Properly setting the crop window ensures the visual integrity and contextual consistency of the displayed content.

[0031] This system extracts layer pixel data from the cropping window within the ultra-high-definition original image layer data. The extraction process employs efficient image processing algorithms, supporting rapid local access to ultra-high-resolution images. For ultra-high-definition original images (typically 16K or higher resolution), the system uses a pyramid multi-resolution storage structure to achieve rapid data extraction from any region at any scaling level. The extracted pixel data retains all the detailed information of the original image, providing a high-quality data foundation for subsequent detail enhancement.

[0032] The layer pixel data undergoes tiered interpolation enhancement according to the image scaling ratio. For each preset scaling increment, an additional layer of pixel detail enhancement is added, allowing image details to gradually appear as the user's gestures approach the viewer. Tiered interpolation enhancement is an intelligent image processing technique that adaptively adjusts the level of detail presentation based on different viewing distances and levels of attention. The enhancement process includes various image processing algorithms such as adaptive sharpening, local contrast enhancement, and texture enhancement, which are activated progressively with increasing scaling. For example, at the global view level (scaling ratio < 2), only basic image resampling is applied; at medium scaling levels (2 ≤ scaling ratio < 5), local contrast enhancement is added; and at high scaling levels (scaling ratio ≥ 5), further texture enhancement and detail sharpening are added. This progressive detail enhancement mechanism simulates the natural visual experience of humans observing artwork, gradually revealing more details as attention increases.

[0033] In this embodiment of the invention, the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area is calculated. When the spatial deviation rate exceeds a preset intent switching threshold, step S2 is re-executed with the current screen state as the initial context, including: The Euclidean distance between the instantaneous projected coordinates of the subsequent gesture trajectory and the geometric center of the viewing focus area is calculated using a fixed sampling period. The spatial deviation rate is obtained by dividing the difference in Euclidean distance between adjacent sampling periods by the sampling period. The sampling period is typically set to 50-100 milliseconds to ensure the system can capture rapid changes in the gesture. The instantaneous projected coordinates are the real-time projection points of the hand position in the screen coordinate system, and the Euclidean distance is calculated using a standard two-dimensional distance formula. The formula for calculating the spatial deviation rate is: ; in, For a moment Spatial deviation rate, For a moment Euclidean distance, The Euclidean distance at the previous sampling time. The sampling period is defined as the deviation rate. A positive deviation rate indicates that the user's gesture is moving away from the current focus area, while a negative rate indicates that it is moving closer. This rate-based analysis method is more accurate in capturing dynamic changes in user intent than simple distance-based judgment.

[0034] When the spatial deviation rate exceeds the intent switching threshold for a consecutive preset number of sampling periods, a shift in the user's viewing intent is confirmed. The preset number is typically set to 3-5 sampling periods, and the intent switching threshold is typically set to 5%-10% of the display screen width per second. This continuous multi-period judgment mechanism effectively avoids misjudgments caused by slight hand tremors or brief deviations, ensuring that the system only responds to clear changes in the user's intent. The intent transfer confirmation process also incorporates trajectory direction consistency analysis, further improving the reliability of the judgment.

[0035] The system records the rendering level and visual focus position of the current immersive viewing interaction screen as a screen state snapshot. This snapshot is a complete record of the current display state before the intention switch, including key parameters such as rendering level (scaling ratio), visual focus coordinates, and active content overlays. These parameters provide necessary contextual information for a smooth transition, ensuring a natural and coherent screen change.

[0036] Using a snapshot of the scene as the initial context, the rendering level is smoothly rolled back to the global view level before step S2 is executed again, maintaining visual continuity during the intention transition. Smooth rollback is an animation transition technique that uses interpolation algorithms to gradually transition the scene from the current detail view to the global view within 0.5-1 seconds, and then refocuses on the new area of ​​interest. This "zoom out and zoom in" transition pattern preserves the continuity of spatial relationships while providing users with contextual awareness of positional changes. During the transition, the system continues to monitor gesture input to ensure it can respond to any interruptions or corrections from the user. This natural and smooth visual transition greatly enhances the comfort and intuitiveness of the immersive viewing experience.

[0037] The foregoing has described an interactive method for appreciating calligraphy and painting in an embodiment of this application. The following describes an interactive device for appreciating calligraphy and painting in an embodiment of this application. Please refer to [link / reference]. Figure 2 One embodiment of the interactive device for appreciating calligraphy and painting in this application includes: The interactive acquisition module is used to acquire the user's hand spatial motion data stream collected by the immersive digital experience cabin's body-sensing gesture interaction system, and load the ultra-high-definition original painting layer data of the calligraphy and painting works to be appreciated and the pre-annotated image semantic partition map. The image semantic partition map includes the boundary of the inscription area, the boundary of the seal area, the boundary of the brush and ink main body area, and the boundary of the colophon area. The trajectory analysis module is used to slide and segment the hand spatial motion data stream according to time windows, extract the spatial curvature sequence and velocity decay characteristics of the gesture motion trajectory within each time window, and determine the trajectory classification result of the current gesture based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay characteristics. The trajectory classification results include global tour trajectory, regional wandering trajectory and fixed-point approach trajectory. The focus area determination module is used to project the spatial coordinates of the gesture movement trajectory onto the screen coordinate system of the ultra-high-definition original image layer data, calculate the shortest distance sequence between the projected trajectory and the boundaries of each region in the semantic partition map of the screen, and perform weighted attenuation on the shortest distance sequence in combination with the trajectory classification results to determine the current appreciation focus area and its semantic partition type. The content retrieval module is used to construct a two-dimensional intent index key based on the trajectory classification results and semantic partition type. It retrieves the appreciation response content package corresponding to the two-dimensional intent index key in the pre-set multi-level appreciation content library. The appreciation response content package contains local enhanced rendering data of the original painting, identification and character analysis data, or multi-point expert interpretation data that match the appreciation focus area. The image rendering module is used to calculate the visual focus offset and rendering level switching parameters of the immersive display image based on the position range of the appreciation focus area in the image coordinate system and the content type of the appreciation response content package. Based on the visual focus offset, it performs multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data, and overlays and renders the appreciation response content package onto the enhanced image layer to generate an immersive appreciation interactive image. The interaction monitoring module is used to continuously collect subsequent hand spatial motion data streams during the immersive appreciation interaction screen output, calculate the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area, and when the spatial deviation rate exceeds the preset intention switching threshold, the trajectory analysis module is re-executed with the current screen state as the initial context. The modules are connected via wired and / or wireless means to enable data transmission between them.

[0038] One embodiment of a calligraphy and painting appreciation experience cabin in this application includes: An immersive display system for displaying ultra-high-definition images of calligraphy and painting works; A motion-sensing gesture interaction system is used to collect data streams of spatial movement of the user's hands; The processing unit is connected to the immersive display system and the motion-sensing interaction system. The processing unit is used to execute the functions of the calligraphy and painting appreciation interactive device.

[0039] This invention achieves immersive intelligent appreciation interaction of calligraphy and painting works through gesture spatial motion analysis, trajectory classification, semantic partitioning mapping, multi-level content retrieval, and progressive rendering. The gesture intent recognition method of this invention can accurately capture the user's appreciation behavior patterns and effectively provide professional content that matches the current focus of attention.

[0040] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0041] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0042] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. An interactive method for appreciating calligraphy and painting, characterized in that, include: Step S1: Obtain the user's hand spatial motion data stream collected by the immersive digital experience cabin's body-sensing gesture interaction system, and load the ultra-high-definition original painting layer data and pre-annotated image semantic partition map of the calligraphy and painting works to be appreciated. The image semantic partition map includes the boundary of the inscription area, the boundary of the seal area, the boundary of the brush and ink main body area, and the boundary of the colophon area. Step S2: The hand spatial motion data stream is divided into sliding segments according to time windows. The spatial curvature sequence and velocity decay features of the gesture motion trajectory within each time window are extracted. Based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay features, the trajectory classification result of the current gesture is determined. The trajectory classification result includes global tour trajectory, regional wandering trajectory and fixed-point approach trajectory. Step S3: Project the spatial coordinates of the gesture movement trajectory onto the image coordinate system of the ultra-high-definition original image layer data, calculate the shortest distance sequence between the projected trajectory and the boundaries of each region in the image semantic partition map, and perform weighted attenuation on the shortest distance sequence in combination with the trajectory classification results to determine the current appreciation focus area and its semantic partition type. Step S4: Construct a two-dimensional intent index key based on the trajectory classification result and the semantic partition type, and retrieve the appreciation response content package corresponding to the two-dimensional intent index key in the preset multi-level appreciation content library. The appreciation response content package includes original painting local enhanced rendering data, signature identification and character analysis data, or multi-point expert interpretation data that match the appreciation focus area. Step S5: Based on the position range of the appreciation focus area in the screen coordinate system and the content type of the appreciation response content package, calculate the visual focus offset and rendering level switching parameters of the immersive display screen, perform multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data based on the visual focus offset, and overlay and render the appreciation response content package onto the enhanced screen layer to generate an immersive appreciation interactive screen. Step S6: During the immersive appreciation interaction screen output, continuously collect subsequent hand spatial motion data streams, calculate the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area, and when the spatial deviation rate exceeds the preset intention switching threshold, re-execute step S2 with the current screen state as the initial context.

2. The interaction method according to claim 1, characterized in that, The step of determining the trajectory classification result of the current gesture based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay feature includes: Calculate the variance of the spatial curvature sequence within the time window as an indicator of trajectory curvature fluctuation. Extract the time length during which the speed value in the speed decay feature drops from its peak value to below a preset speed threshold, and use this as the deceleration duration. If the trajectory curvature fluctuation index is lower than the first curvature threshold and the deceleration duration is shorter than the first duration threshold, then the trajectory classification result is determined to be a global tour trajectory. If the trajectory curvature fluctuation index is between the first curvature threshold and the second curvature threshold and the deceleration duration is between the first duration threshold and the second duration threshold, then the trajectory classification result is determined to be a regional wandering trajectory. If the trajectory curvature fluctuation index is higher than the second curvature threshold and the deceleration duration exceeds the second duration threshold, then the trajectory classification result is determined to be a fixed-point approach trajectory.

3. The interaction method according to claim 1, characterized in that, The step of combining the trajectory classification results with weighted attenuation of the shortest distance sequence to determine the current appreciation focus area and its semantic partition type includes: A first distance attenuation coefficient, a second distance attenuation coefficient, and a third distance attenuation coefficient are respectively configured for the global tour trajectory, the regional wandering trajectory, and the fixed-point approach trajectory, wherein the third distance attenuation coefficient is greater than the second distance attenuation coefficient, and the second distance attenuation coefficient is greater than the first distance attenuation coefficient; Multiply the distance value corresponding to each semantic partition in the shortest distance sequence by the distance decay coefficient corresponding to the current trajectory classification result to obtain the decayed distance value of each semantic partition; The semantic partition with the smallest distance value after attenuation is selected as the hit partition. The geometric intersection area between the region boundary of the hit partition and the projection trajectory is determined as the appreciation focus area. The type label of the hit partition is determined as the semantic partition type.

4. The interaction method according to claim 1, characterized in that, The step of constructing a two-dimensional intent index key based on the trajectory classification result and the semantic partition type, and retrieving the appreciation response content package corresponding to the two-dimensional intent index key from a pre-set multi-level appreciation content library, includes: The trajectory classification result is encoded as a first-dimensional index value, the semantic partition type is encoded as a second-dimensional index value, and the first-dimensional index value and the second-dimensional index value are concatenated to generate the two-dimensional intent index key; Search the index table of the multi-level appreciation content library for content entries that match the two-dimensional intent index key. The content entries are associated with content priority identifiers and content presentation method identifiers. When the trajectory classification result is a fixed-point approach trajectory and the semantic partition type is a seal area, the signature and character identification analysis data in the appreciation response content package includes the seal outline extraction result, the character-by-character interpretation information of the seal characters, and the historical period information of the seal. When the trajectory classification result is a regional wandering trajectory and the semantic partition type is a brush and ink main area, the multi-point expert interpretation data in the appreciation response content package includes explanations of brush techniques and analysis of composition and structure corresponding to the appreciation focus area.

5. The interaction method according to claim 1, characterized in that, The process of performing multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data based on the visual focus offset includes: The absolute coordinates of the cropping center point in the ultra-high-definition original image layer data are determined based on the visual focus offset. Calculate the current screen scaling ratio based on the rendering level switching parameters, and determine the size range of the cropping window with the cropping center point as the geometric center according to the screen scaling ratio. Extract the layer pixel data within the cropping window from the ultra-high-definition original image layer data; The pixel data of the layer is subjected to hierarchical interpolation enhancement according to the screen scaling ratio. For each preset increment of the screen scaling ratio, an additional layer of pixel detail enhancement processing is added, so that the screen details gradually appear as the user's gesture approaches the action.

6. The interaction method according to claim 1, characterized in that, The calculation of the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area, and when the spatial deviation rate exceeds a preset intent switching threshold, step S2 is re-executed with the current screen state as the initial context, including: The Euclidean distance between the instantaneous projected coordinates of the subsequent gesture trajectory and the geometric center of the appreciation focus area is calculated according to a fixed sampling period. The spatial deviation rate is obtained by dividing the difference in Euclidean distance between adjacent sampling periods by the sampling period. When the spatial deviation rate exceeds the intent switching threshold for a consecutive preset number of sampling periods, it is confirmed that the user's appreciation intent has shifted. Record the rendering level and visual focus position of the current immersive appreciation and interactive screen as a snapshot of the screen state; Using the aforementioned snapshot of the screen state as the initial context, the rendering level is smoothly rolled back to the global view level, and step S2 is re-executed to maintain visual continuity during the intention switching process.

7. An interactive device for appreciating calligraphy and painting, used to implement the interactive method according to any one of claims 1 to 6, characterized in that, include: The interactive acquisition module is used to acquire the user's hand spatial motion data stream collected by the immersive digital experience cabin's body-sensing gesture interaction system, and load the ultra-high-definition original painting layer data of the calligraphy and painting works to be appreciated and the pre-annotated image semantic partition map. The image semantic partition map includes the boundary of the inscription area, the boundary of the seal area, the boundary of the brush and ink main body area, and the boundary of the colophon area. The trajectory analysis module is used to slide and segment the hand spatial motion data stream according to time windows, extract the spatial curvature sequence and velocity decay features of the gesture motion trajectory within each time window, and determine the trajectory classification result of the current gesture based on the periodic distribution pattern of the spatial curvature sequence and the convergence trend of the velocity decay features. The trajectory classification result includes global tour trajectory, regional wandering trajectory and fixed-point approach trajectory. The focus area determination module is used to project the spatial coordinates of the gesture movement trajectory onto the screen coordinate system of the ultra-high-definition original image layer data, calculate the shortest distance sequence between the projected trajectory and the boundaries of each region in the screen semantic partition map, and perform weighted attenuation on the shortest distance sequence in combination with the trajectory classification results to determine the current appreciation focus area and its semantic partition type. The content retrieval module is used to construct a two-dimensional intent index key based on the trajectory classification result and the semantic partition type, and to retrieve the appreciation response content package corresponding to the two-dimensional intent index key in the preset multi-level appreciation content library. The appreciation response content package includes original painting local enhanced rendering data, signature identification and character analysis data or multi-point expert interpretation data that match the appreciation focus area. The image rendering module is used to calculate the visual focus offset and rendering level switching parameters of the immersive display image based on the position range of the appreciation focus area in the image coordinate system and the content type of the appreciation response content package. Based on the visual focus offset, it performs multi-scale cropping and progressive detail enhancement on the ultra-high-definition original image layer data, and overlays and renders the appreciation response content package onto the enhanced image layer to generate an immersive appreciation interactive image. The interaction monitoring module is used to continuously collect subsequent hand spatial motion data streams during the immersive appreciation interaction screen output, calculate the spatial deviation rate between the subsequent gesture trajectory and the appreciation focus area, and when the spatial deviation rate exceeds the preset intention switching threshold, re-execute the trajectory analysis module with the current screen state as the initial context.

8. A calligraphy and painting appreciation experience cabin, characterized in that, The experience cabin includes: An immersive display system for displaying ultra-high-definition images of calligraphy and painting works; A motion-sensing gesture interaction system is used to collect data streams of spatial movement of the user's hands; The processing unit is connected to the immersive display system and the motion-sensing gesture interaction system, and the processing unit is used to perform the functions of the calligraphy and painting appreciation interactive device as described in claim 7.

Images