Gesture-driven bead lamp experience method fusing process rules

Abstract

The application belongs to the technical field of computers and relates to a gesture-driven bead lamp experience method fusing process rules. The method first establishes a three-dimensional digital model of a bead lamp, converts components such as colored beads, bead strings, lamp stands, treasure covers and tassels into a digital bead lamp model that can be disassembled and assembled; then configures a gesture recognition module, maps seven gestures such as grabbing, kneading, dragging and rotating into five process instructions of selecting beads, threading beads, assembling lamps, hanging tassels and lighting; then deploys a process rule engine, establishes five types of rule libraries of process sequence constraints, structure position constraints, color rhythm constraints, pattern matching constraints and completion constraints; then constructs a programmed bead string generation module; then establishes a real-time feedback subsystem; and finally integrates a cultural narrative triggering module. The application can realize process process visualization, intuitive transmission of structure relationship and deep transmission of cultural significance.

Description

Technical Field

[0001] This application belongs to the field of computer technology, specifically relating to a gesture-driven bead lamp experience method that integrates process rules. Background Technology

[0002] Intangible cultural heritage is a cultural treasure formed and passed down through the course of human social development. Traditional craft-related intangible cultural heritage, in particular, carries rich technical knowledge, aesthetic value, and cultural memory. With the rapid development of digital technology, digital protection and dissemination have become important pathways for the inheritance and innovation of intangible cultural heritage. Visualizing intangible cultural heritage resources through technologies such as 3D scanning, digital modeling, and virtual reality can overcome the limitations of traditional exhibitions in terms of time, space, and form, allowing precious artifacts and techniques to be disseminated more widely. However, existing digital displays of intangible cultural heritage mostly remain at the level of visual presentation. Effective technical solutions are still lacking in how to achieve deep interaction between the audience and the craft process through digital technology, and how to enable users to truly understand the inherent logic and cultural significance of traditional techniques.

[0003] The display and dissemination of traditional handicrafts has long faced the challenge of a superficial experiential level. Take traditional lanterns, such as those used in traditional halls, as an example. These products not only possess exquisite craftsmanship and unique artistic forms, but also embody rich cultural connotations and historical memories. However, current display methods only allow viewers to see the final product. They cannot perceive the complex production processes such as stringing beads, assembling the lantern body, hanging tassels, and lighting the lamp, nor can they understand the cultural significance and ritual function of the product in a specific spatial context. Traditional textual and pictorial explanations, limited by their expressive form, struggle to accurately convey the structural hierarchy and technological logic of the product, resulting in viewers' understanding of traditional craftsmanship remaining superficial and failing to establish a deeper cognitive and emotional connection.

[0004] In exploring the application of digital interactive technology to traditional craft experiences, existing solutions suffer from significant functional deficiencies and technical limitations. On the one hand, while gesture recognition technology has matured in consumer applications, it lacks semantic mapping design for craft actions in the realm of traditional craft experiences, failing to translate actions such as beading, assembly, and hanging into intuitive digital operations. On the other hand, traditional crafts have strict procedural sequences, spatial structures, and process specifications. Existing digital experience systems lack the support of a process rule engine, failing to reflect the inherent order of traditional crafts during interaction. This results in a lack of standardized constraints on user operations, making it difficult to truly understand the logical system of traditional crafts through physical participation. Due to these technical problems, existing digital display systems for traditional crafts cannot achieve the transformation from static viewing to dynamic participation, and from superficial appreciation to deep understanding. Therefore, a gesture-driven interaction method that integrates process rules is urgently needed to address these technical issues. Summary of the Invention

[0005] The purpose of this invention is to provide a gesture-driven bead lamp experience method that integrates process rules, which can effectively solve the problems of invisible process, unreadable structural relationships, and imperceptible cultural significance in existing traditional handicraft digital display systems.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A gesture-driven beaded lantern experience method integrating craft rules includes the following specific steps: Step 1: Establish a 3D digital model of the beaded lantern: By combining 3D scanning with manual modeling, the components of the Lu Zhai beaded lantern, such as colored beads, bead strings, lamp frames, canopies, tassels, patterns, light sources, and hall spaces, are transformed into a detachable, assembleable, and illuminateable digital beaded lantern model, forming a complete beaded lantern hierarchical structure dataset; Step 2: Configure a gesture recognition and mapping module: By acquiring depth images of the user's hand through a depth camera or infrared sensor, feature parameters such as palm position, five-finger posture, and movement trajectory are extracted, and seven basic gesture actions, such as grasping, pinching, dragging, rotating, closing, pulling down, and lifting, are mapped to craft operation instructions such as selecting beads, stringing beads, assembling lanterns, hanging tassels, and lighting lanterns; Step 3: Deploy a craft rule engine: Establish a five-category craft rule library including process sequence constraints, structural position constraints, color rhythm constraints, pattern matching constraints, and completion constraints. The system verifies in real time whether the user's operation conforms to traditional... The system's logic for making beaded lanterns involves identifying and correcting operations that violate the craft rules. Step 4 involves building a programmed bead string generation module: users set the starting point, ending point, and intermediate nodes of the bead string through key node operations. The system automatically generates a complete bead string based on parameters such as the number of beads, the spacing between beads, the color sequence, and the pattern rules, improving efficiency while preserving the sense of participation in the craft. Step 5 involves establishing a real-time feedback subsystem: correct operations trigger adsorption effects, light-emitting effects, particle effects, sound effects, and progress bar feedback; incorrect operations trigger node flashing effects, component rebound effects, and short prompts, guiding users to understand the craft logic and correct their operations. Step 6 involves integrating a cultural narrative trigger module: cultural content is dynamically unlocked as accompanying information for craft nodes. Completing bead stringing triggers a narrative of craftsmanship, completing lantern assembly triggers a narrative of structural order, completing tassel hanging triggers a narrative of auspicious symbolism, and completing lighting triggers a narrative of the history of the Lu Residence hall, realizing a transformation from viewer to lantern maker in the experience.

[0007] Preferably, in step 1, the three-dimensional scanning adopts a multi-angle surround acquisition scheme, the acquisition angle interval adopts a preset angle interval, the scanning point cloud density of a single bead lamp component adopts a preset density of three-dimensional coordinate points, and multi-view point cloud registration is performed by iterative nearest point algorithm. The manual modeling part adjusts the physical simulation parameters for the dynamic deformation of the bead string and the natural dangling of the tassel. The generated digital bead lamp model contains hierarchical structure relationship data with a preset number of layers.

[0008] Preferably, in step 2, the gesture recognition module uses a deep learning-based joint tracking algorithm to extract a preset number of three-dimensional coordinates of hand joints and a preset number of spatial orientation vectors of the palm area from the depth image. The gesture action determination period adopts a preset determination period, and the minimum duration of a single gesture adopts a preset minimum duration to prevent misidentification. The determination condition for grasping gesture is that the distance between the thumb and the other four fingers is less than a preset distance threshold and the holding time exceeds a preset holding time.

[0009] Preferably, in step 3, the process rule engine adopts a hybrid architecture that combines production rules and state machines. The process sequence constraints stipulate that bead selection must be completed before bead threading, bead threading must be completed before lamp assembly, lamp assembly must be completed before tassel hanging, and tassel hanging must be completed before lighting. If any process violates the sequence rules, the system locks the interaction permissions of subsequent processes until the current process is completed.

[0010] Preferably, in step 3, the structural position constraints stipulate that the bead string can only be hung on the preset hanging point of the lamp holder, the tassel can only be hung on the preset hanging point of the main lamp, and the canopy must cover the main lamp. Operations with a three-dimensional spatial position deviation exceeding a preset deviation threshold are judged as erroneous operations and trigger component rebound feedback.

[0011] Preferably, in step 4, the programmed bead string generation module uses a path reparameterization algorithm to fit the user-defined discrete nodes into a smooth curve, calculates the number of beads based on the curve length and bead diameter parameters, dynamically adjusts the bead spacing within a preset spacing range according to the pattern requirements, and gradually transitions the color sequence along the length of the bead string according to preset color gamut rules.

[0012] Preferably, the correct operation feedback of the real-time feedback subsystem in step 5 is completed within a preset response time. The adsorption effect is manifested as the component automatically moves to the target position and is accompanied by an elastic oscillation animation. The light emission effect is manifested as the brightness of the component surface increases from the initial value to the target value and is accompanied by a halo diffusion animation. The particle effect is manifested as a preset number of light particles being sprayed from the component surface and naturally falling along the direction of gravity.

[0013] Preferably, in step 5, the node blinking for error operation feedback adopts a preset blinking frequency, the blinking duration adopts a preset blinking duration, the elastic coefficient of component rebound adopts a preset elastic coefficient, the rebound displacement adopts a preset rebound displacement multiple, and the short prompt content is matched and selected from a preset prompt library according to the error type. The prompt library contains a preset number of guiding statements for different error scenarios.

[0014] Preferably, in step 6, the cultural narrative triggering module and the bead lamp completion detection module are linked in real time. When the completion of stringing beads reaches the preset completion threshold, the first stage of cultural content is unlocked. When the completion of lamp assembly reaches the preset completion threshold, the second stage of cultural content is unlocked. When the completion of hanging tassels reaches the preset completion threshold, the third stage of cultural content is unlocked. When the lamp lighting operation is triggered, the fourth stage of cultural content is unlocked and the scene is switched to the Lu Zhai Su Yong Tang three-dimensional digital space.

[0015] Preferably, the fourth stage of cultural content in step 6 includes narrative content in four dimensions: the historical origin of Lu Zhai Zhu Lantern, the spatial relationship of the hall, the lighting ceremony during festivals, and the story of intangible cultural heritage. The narrative content is presented in the form of floating text anchored in three-dimensional space and audio narration. The font size of the text is dynamically adjusted according to the user's viewing distance to ensure reading clarity.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. Visual presentation of the process By using 3D digital modeling, the components of the beaded lamp, such as the colored beads, bead strings, lamp frame, canopy, and tassels, are completely digitally disassembled. Combined with programmed bead string generation technology, users can complete the bead stringing process without operating each bead individually. The system automatically expands the user's single key node operation into a complete bead string that conforms to the process logic, effectively solving the problem that the process is not visible in traditional displays, allowing the audience to intuitively understand the complete production process from colored beads to the lamp.

[0017] 2. Structural relationships are conveyed intuitively. By establishing a process rule engine containing five types of constraints, the system verifies every user operation in real time. The process sequence constraint ensures that the user must complete the operation in the correct order of selecting beads, stringing beads, assembling lights, and hanging tassels. The structural position constraint ensures that the bead strings and tassels must be installed at the preset positions. Under the guidance of the rules, the user gradually understands the hierarchical structural relationship of the beaded lights, solving the problem that ordinary graphic descriptions cannot explain the structural logic between the canopy, main light, bead strings, and tassels.

[0018] 3. Deep Transmission of Cultural Significance By dynamically binding cultural content with craft nodes through the cultural narrative trigger module, users can understand the value of craftsmen's labor after completing the beading operation, the significance of structural order after completing the lamp assembly operation, the connotation of auspicious symbolism after completing the tassel hanging operation, and enter the digital space of Lu Zhai Hall after completing the lamp lighting operation and unlock the complete historical narrative. This realizes the transformation of cultural meaning from static textual explanation to dynamic physical participation, enabling users to establish a deep cognitive and emotional connection.

[0019] 4. The interactive experience is natural and smooth. Through the semantic mapping design of seven basic gestures and five process operation instructions, an intuitive correspondence is established between digital operations and real process actions. The real-time feedback subsystem responds to user operations within a preset response time. Correct operations receive immersive audiovisual feedback to enhance the sense of accomplishment, while incorrect operations receive immediate guidance to correct the operation path. The overall experience path is designed as a linear process of gesture calibration, lamp structure identification, bead selection, bead stringing, lamp assembly, tassel hanging, and lamp lighting. The logic is clear and natural, suitable for the interaction habits of users of different ages. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall technical solution architecture of the gesture-driven bead lamp experience method that integrates process rules proposed in this invention.

[0021] Figure 2 This is a schematic diagram of the core principle framework of the process rule engine in this invention.

[0022] Figure 3 This is a logical flowchart of the mapping between gesture recognition and process operation in this invention.

[0023] Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow between the user terminal and the system server in this invention.

[0024] Figure 5 This is a schematic diagram illustrating the principle of the phased unlocking mechanism of the Chinese narrative triggering module in this invention.

[0025] Figure 6 This is a schematic diagram of the technical framework of the programmed bead generation module in this invention. Detailed Implementation

[0026] Example 1

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0028] This invention proposes a gesture-driven bead lamp experience method that integrates process rules. Its overall technical architecture consists of six core modules coupled sequentially, forming a complete interactive closed loop from 3D digital modeling to cultural narrative triggering. In the specific implementation process, steps 1 to 6 are executed sequentially according to the data flow order. The output of the previous step serves as the input parameter for the subsequent step. The modules are seamlessly connected through standardized data interfaces, ensuring that the entire system performs collaborative calculations within a unified spatiotemporal coordinate system and rendering engine.

[0029] Step 1: Establish a 3D digital model of the beaded lamp This step utilizes a dual-track technical approach combining 3D scanning and manual modeling to transform the physical entity of the Lu Zhai beaded lamp into a precise digital twin model. This model needs to fully preserve the lamp's hierarchical structure, physical material properties, and spatial topology, providing a detachable and reassembleable digital base for subsequent gesture-based interactive operations.

[0030] In the 3D scanning phase, the acquisition equipment employs a multi-angle surround arrangement with a 15-degree interval between acquisition angles. This means that 24 scanning stations at different azimuth angles are required around the central axis of the beaded lamp. The raw point cloud data acquired at each scanning station contains a 3D coordinate grid of the visible surface from that viewpoint. The point cloud density parameter is set to 120 sampling points per square centimeter. The total number of sampling points for a single beaded lamp component depends on the surface area of ​​the component; a typical-sized colored bead component has approximately 500 to 800 sampling points, while the main body of the lamp frame has approximately 20,000 to 50,000 sampling points. The optical parameters of the acquisition equipment include a working distance range of 800 to 1500 mm, a depth of field of 200 mm, a depth accuracy of 0.1 mm, and an RGB resolution of 1920×1080 pixels.

[0031] Multi-view point cloud registration is achieved using an iterative nearest-point algorithm, and the registration process is divided into two stages: coarse registration and fine registration. In the coarse registration stage, an initial alignment method based on feature descriptors is used to extract FPFH features from the source and target point clouds, and the initial transformation matrix is ​​estimated through feature matching. In the fine registration stage, nearest-point iterative optimization is performed, with the objective function being the root mean square error of the point-to-point Euclidean distance. The convergence threshold is set to 0.001 meters, and the maximum number of iterations is set to 100. During each iteration, the algorithm dynamically calculates the translation and rotation components of the rigid body transformation matrix until the error value drops below the convergence threshold or the maximum number of iterations is reached. After registration, the point cloud data from each viewpoint are unified into the world coordinate system, forming a complete 3D point cloud model.

[0032] The manual modeling stage supplements the modeling of dynamic deformation features that are difficult to accurately capture through scanning. The dynamic deformation of the bead string is modeled using a flexible chain-like rigid body simulation method. The bead string is discretized into several mass nodes and connecting springs. The spring stiffness between adjacent nodes is set to 500 N / m, and the damping coefficient is set to 0.02. The node mass is calculated based on the density and volume of the bead material. During subsequent interactions, when the user drags the bead string, the spring physics system calculates the force state of each node in real time, driving the bead string to produce a natural dangling and swaying effect. The natural draping of the tassels is modeled using a particle rope simulation method. The tassels are discretized into several particle points, which are connected by distance constraints. The constraint stiffness is set to 800 N / m, simulating the natural stacking shape of the tassel fibers under gravity. Physical simulation parameters also include a gravitational acceleration of 9.81 m / s², an air drag coefficient of 0.05, and a collision detection mesh resolution of 5 mm.

[0033] The generated digital beaded lamp model contains multi-level structural data, with the hierarchy following the actual physical composition of the lamp. The top level is the hall space level, containing 3D model data of the exhibition hall walls, floor, ceiling, and wooden beams. The model accuracy is LOD2, and the number of polygons is controlled within 10,000. The second level is the canopy level, containing the canopy's skeletal structure, top decoration, and edge curtains. The skeletal structure is modeled using tubular curves, and the curtains using a mesh model. The third level is the main lamp level, containing the main frame of the lamp holder, the lampshade structure, the built-in light source, and the electrical wiring. The lamp holder is modeled using a rod structure with rectangular cross-sections. The fourth level is the bead string level, containing all the bead strings suspended on the lamp holder. Each bead string consists of several colored beads arranged along a specific path, and the strings are connected to the lamp holder via hanging points. The fifth level is the tassel level, containing the tassel components hanging below the main lamp. The tassel components consist of multiple layers of tassels of different lengths.

[0034] The spatial relationships between different levels are organized and managed using a transformation matrix tree structure. The root node is the world coordinate transformation matrix of the hall space, and the child nodes are, in order, the local coordinate transformation matrices of the canopy, main lamp, beaded string, and tassel. Each transformation matrix contains three components: a translation vector, a rotation quaternion, and a scaling factor, supporting linked transformations between levels. When a user moves a level using gestures, its child levels automatically transform synchronously with the parent level. The model data uses a hierarchical storage format, with hierarchical relationship data, structural geometry data, and material texture data stored in different data partitions, supporting on-demand loading and incremental updates.

[0035] Step 2: Configure the gesture recognition and mapping module This step involves acquiring depth images of the user's hand using a depth sensor, and then using a deep learning-based joint tracking algorithm to extract the three-dimensional coordinates of the hand joints and the spatial orientation vector of the palm area from the depth images. This maps seven basic hand gestures into five process operation commands, thus realizing the semantic association between the user's natural hand movements and the operation of the digital bead lamp.

[0036] The gesture recognition system uses an infrared depth sensor as its core acquisition device. The sensor is a structured light depth camera with an effective acquisition range of 0.3 to 0.8 meters, a depth image resolution of 640×480 pixels, a frame rate of 60 Hz, and adjustable infrared emission power. The depth measurement accuracy at the standard working distance is 0.5 mm. The sensor is mounted at the top center of the interactive terminal, with a tilt angle set to 15 degrees downwards to ensure complete capture of the user's frontal hand movements.

[0037] The joint tracking algorithm employs a deep learning network architecture, comprising two main components: a feature extraction backbone network and a keypoint regression head. The feature extraction backbone network uses a lightweight convolutional neural network with four feature pyramid levels, each containing 16, 32, 64, and 128 channels respectively. The convolutional kernel size is 3×3, and ReLU activation and batch normalization layers are used. The keypoint regression head contains 21 output nodes, corresponding to the 21 joints of the hand in 3D coordinates. Each output node contains x, y, and z coordinate components. The network model's input is a depth image tensor with a size of 128×128×1, and the network output is a 21×3 matrix of joint coordinates.

[0038] The definition of hand joints follows the 21-point hand model standard, which divides the hand structure into the wrist region, metacarpal region, and phalangeal region. The wrist region contains one joint as the base reference point of the palm. The metacarpal region contains five joints, located at the metacarpophalangeal joints of the thumb, index finger, middle finger, ring finger, and little finger. Each finger contains three joints, located in the middle of the proximal phalanx, middle phalanx, and distal phalanx. Therefore, the spatial distribution of the 21 joints can completely describe the morphological structure of the palm and the bending posture of each finger.

[0039] The spatial orientation vector of the palm region is calculated using the normal vector of the palm fitting plane. Specifically, six points are selected from 21 joints: the wrist joint and five metacarpophalangeal joints. The least squares method is used to fit the palm plane equation, and the plane normal vector is the palm orientation vector. Normalization of the orientation vector ensures a magnitude of 1, facilitating subsequent pose determination calculations. The angle between the palm normal vector and the principal optical axis of the depth camera is used to determine the palm's frontal or lateral orientation. An angle less than 30 degrees indicates the palm is facing the user directly, while an angle greater than 60 degrees indicates the palm is facing laterally.

[0040] Gesture recognition is achieved using a state machine method based on temporal features. The recognition period parameter is set to 16 milliseconds, corresponding to the reciprocal of the depth camera's frame rate of 60 Hz, ensuring that every frame can participate in the gesture recognition calculation. The minimum duration parameter for a single gesture is set to 200 milliseconds to filter out brief, unconscious hand movements and prevent misidentification. The recognition process first extracts the hand joint sequence from the depth images of the current frame and the previous N frames, with N set to 12, corresponding to a time window length of 192 milliseconds. Then, temporal features are calculated from the joint sequence, including joint movement speed, joint angle change rate, and palm orientation change rate. The temporal feature vector is used as input to a classifier, which employs a support vector machine model and outputs the category label of the current gesture.

[0041] The criteria for determining the seven basic hand gestures are defined as follows: The grasping gesture is defined as follows: the distance between the tips of the thumb and index finger is less than 40 mm and the duration exceeds 200 milliseconds; simultaneously, the minimum distance between the thumb and the other four fingers is less than 45 mm. This condition ensures sufficient robustness in determining the grasping gesture. The pinching gesture is defined as follows: the distance between the tips of the thumb and index finger is less than 25 mm and decreases rapidly at a rate greater than 100 mm per second. This condition distinguishes the pinching gesture from the grasping gesture. The dragging gesture is defined as follows: the overall displacement of the palm in three-dimensional space exceeds 50 mm and the trajectory is smooth. The displacement is calculated using the average position difference over five consecutive frames. The rotating gesture is defined as follows: the angular velocity of the palm around its central axis exceeds 30 degrees per second and the duration exceeds 150 milliseconds. The angular velocity is calculated through the rotational transformation of the palm's normal vector. The closing gesture is defined as follows: the area of ​​the region enclosed by the fingertips decreases rapidly at a rate greater than 5000 square millimeters per second. The criteria for a pull-down gesture are: the palm's speed in the direction opposite to gravity is greater than 200 millimeter per second and the duration is greater than 300 milliseconds. The direction of movement is determined based on the angle between the palm's normal vector and the gravity vector in the world coordinate system. The criteria for a lift gesture are: the palm's speed in the direction of gravity is greater than 150 millimeter per second and the angle between the palm's normal vector and the direction of gravity is less than 45 degrees. This condition ensures that the lift action has a correct palm-up posture.

[0042] The definitions of the five process operation commands and their mapping relationships with gestures are designed as follows: **Bead Selection:** The user selects a specific color and size bead from the digital bead lamp's bead library. The mapped gesture is a pinch gesture; the user selects the currently highlighted bead using this gesture, and the bead is then held in hand. **Bead Threading:** The user threads the selected bead into the bead string along a specific path. The mapped gesture is a drag gesture; the user moves the held bead to the corresponding position on the bead string path, and the system automatically completes the bead threading animation and the bead string's geometry update. **Lamp Assembly:** The user installs the bead string onto the preset hanging points on the lamp holder. The mapped gesture is a rotation gesture; the user adjusts the hanging angle of the bead string to align with the hanging point. When the bead string approaches the hanging point, the system triggers an adsorption effect to complete the installation. **Tassel Hanging:** The user hangs the tassel component below the main lamp. The mapped gesture is a closing gesture; the user aligns the tassel component with the tassel hanging ring of the main lamp using this closing gesture, and the system automatically completes the hanging animation. The lighting operation command corresponds to the user activating the light source component of the beaded lamp to make it emit light. The mapped gesture is a lifting gesture. The user lifts their palm to the bottom of the lamp body by lifting the gesture. After the system determines that the lifting posture is correct, it triggers the lighting sequence animation.

[0043] Step 3: Deploy the process rule engine This step establishes a process rule engine to ensure that every user operation conforms to the traditional bead lamp making logic. The process rule engine adopts a hybrid architecture combining production rules and state machines, with five types of process rules working together to achieve real-time verification and error guidance for user operations.

[0044] The design considerations for the hybrid architecture of production rules and state machines are as follows: Production rules are suitable for expressing complex conditional judgment logic and causal reasoning relationships, and can flexibly describe the dependency constraints between processes, the positional constraints between components, and the matching rules of colors and patterns; State machines are suitable for expressing process-oriented sequential logic and stage-based state transitions, and can clearly define the user's current work stage and the set of executable operations. Organically combining the two architectures can fully leverage their respective advantages: the state machine is responsible for managing the macro-level process progress and stage states, while production rules are responsible for handling micro-level operation verification and constraint judgment.

[0045] The state machine defines seven working states and two special states. Working states include the initial state, bead selection state, bead stringing state, lamp assembly state, tassel hanging state, lamp lighting state, and completion state. The initial state is the default state after the user enters the system; at this time, the digital bead lamp has not yet begun production, and all process operations are unavailable. The bead selection state indicates that the user is selecting colored beads; in this state, only a pinching gesture is responded to as a bead selection operation command. The bead stringing state indicates that the user is stringing beads; in this state, a dragging gesture is responded to as a bead stringing operation command. The lamp assembly state indicates that the user is assembling the lamp body; in this state, a rotating gesture is responded to as a lamp assembly operation command. The tassel hanging state indicates that the user is hanging tassels; in this state, a closing gesture is responded to as a tassel hanging operation command. The lamp lighting state indicates that the user is performing a lamp lighting ceremony; in this state, a lifting gesture is responded to as a lamp lighting operation command. The completion state indicates that the entire bead lamp production process is complete; in this state, the user can view the production record or start a new production process. The two special states are error state and lock state. Error state means that the system has detected that the user's operation violates the process rules, and lock state means that the system temporarily freezes the user's interaction permissions.

[0046] State transition rules define the transition conditions and actions between states. Transition conditions include forward transition conditions and rollback transition conditions. A forward transition condition is that the process operation corresponding to the current state is successfully completed. The criterion for successful completion is that all process rule checks for the operation pass. A rollback transition condition is that the user executes an operation that violates the process rules in the current state and the number of corrections exceeds a threshold, or the user actively initiates a rollback command. Forward transition actions include updating the current state variables of the state machine, triggering the initialization operation of the next state, recording the operation completion log, and activating the interaction interface corresponding to the next state. Rollback transition actions include rolling back the model changes caused by the current operation, playing the operation rollback animation, outputting error messages, and activating error correction guidance.

[0047] The detailed definitions of the five categories of process rule bases are as follows. The first category is process sequence constraint rules, which employ a dual constraint mechanism combining implicit state machine constraints and explicit production constraints. State machine constraints are implemented by locking states, preventing users from entering the working state of subsequent processes without completing the preceding process. A production constraint rule entry is represented as "IF current state equals bead selection completed AND operation instruction equals bead threading operation THEN allow execution ELSE lock interaction," ensuring that bead selection must be completed before bead threading. Similar rule entries also constrain bead threading to be completed before lamp assembly, lamp assembly to be completed before tassel hanging, and tassel hanging to be completed before lighting. The second category is structural position constraint rules, which define the spatial installation position relationships of the various components of the beaded lamp. Bead strings can only be suspended on preset hanging points on the lamp holder. The hanging point positions are represented by spherical markers in the 3D model, with a sphere radius of 20 mm. The determination of three-dimensional spatial position deviation uses Euclidean distance measurement. When the distance between the user-operated bead string hanging point and the preset hanging point is less than 15 mm, an adsorption effect is triggered. When the distance is greater than 15 mm but less than 50 mm, a magnetic attraction effect is triggered. When the distance is greater than 50 mm, it is judged as an incorrect operation and a component rebound effect is triggered. Tassels can only be hung on the preset tassel hanging rings of the main light. The position of the hanging rings is indicated by a ring mark, and the hanging judgment conditions are the same as those for the bead string hanging points. The canopy must cover the main light, the distance between the bottom surface of the canopy and the top surface of the main light must be less than 30 mm, and the projection of the bottom surface of the canopy must completely include the projection of the top surface of the main light; otherwise, a position correction prompt will be triggered.

[0048] The third category is the color rhythm constraint rule, which defines the aesthetic principles and sequence rules for the color matching of beads. The traditional color scheme of Lu Zhai beaded lanterns follows color theory and the requirements of a festive atmosphere, primarily using five colors: red, yellow, blue, green, and white, each corresponding to a different auspicious meaning. The color sequence rule stipulates that the color scheme of a single bead string must follow a gradual transition principle, the hue difference between adjacent beads must not exceed a hue angle of 60 degrees, and the direction of hue gradation must be consistent with the direction of the bead string. The overall lantern color scheme rule stipulates that the upper bead string should be predominantly warm-toned, the middle bead string predominantly neutral-toned, and the lower bead string predominantly cool-toned, with the color distribution conforming to the aesthetic requirement of a stable visual center of gravity. When the user selects a bead color that violates the color scheme rule, the system outputs a prompt message guiding the user to reselect.

[0049] The fourth type is pattern matching constraint rules, which define the pattern composition and arrangement rules of the patterns on the surface of beaded lanterns. The traditional patterns of Lu Residence beaded lanterns include five main pattern types: cloud pattern, dragon pattern, phoenix pattern, peony pattern and fu character pattern, and each pattern is realized by a specific arrangement sequence of colored beads. The pattern matching rule stipulates that when the user strings the beads, the color of the colored beads must be selected according to the preset pattern template sequence. The template sequence is stored in the form of a finite state machine, and the state transition condition is that the user successfully threads a colored bead. When the color of the colored bead selected by the user does not match the color expected by the current template state, the system outputs pattern prompt information. The pattern template library contains five preset templates, and the user can select the pattern type before the production starts, and the pattern type cannot be changed during the production process.

[0050] The fifth type is completion degree constraint rules, which define the judgment criteria and thresholds for the completion of each process. The beading completion degree is defined as the ratio of the number of colored beads already threaded into the bead string to the preset total number of colored beads, and the beading process is judged to be completed when the ratio reaches 0.95. The lantern assembly completion degree is defined as the ratio of the number of bead strings that have been installed in place to the preset total number of bead strings, and the lantern assembly process is judged to be completed when the ratio reaches 1.0. The tassel hanging completion degree is defined as the ratio of the number of tassel components that have been hung in place to the preset total number of tassel components, and the tassel hanging process is judged to be completed when the ratio reaches 1.0. The trigger condition for the lighting operation is that the beading completion degree, the lantern assembly completion degree and the tassel hanging completion degree all reach 1.0, and if any item does not meet the standard, the lighting ceremony cannot be activated.

[0051] The real-time verification process of the process rule engine is designed as follows. The system traverses all valid rule entries at a period of 16 milliseconds, obtains the type and parameters of the current gesture action from the user interaction module, obtains the type and state of the current operation object from the digital model module, and matches the applicable rule conditions from the rule library for verification calculation. The verification results are divided into three types: allowed execution, warning execution and prohibited execution. Allowed execution means that the operation conforms to the process rules, and the system performs corresponding model update and feedback response. Warning execution means that the operation basically conforms to the process rules but there is room for optimization, and the system performs corresponding model update and outputs suggestion information. Prohibited execution means that the operation violates the process rules, and the system refuses to perform the model update and triggers a correction guidance process. The correction guidance process includes three stages: error type judgment, guidance strategy selection and guidance content output. The error type judgment determines the severity of the error according to the category of the violated rule, the guidance strategy selection determines the guidance mode according to the error severity, and the guidance content output selects the corresponding guidance text from the preset prompt library according to the guidance mode.

[0052] Step 4: Construct a programmed bead string generation module This step implements a programmatic bead generation technology. Users set the path shape and basic parameters of the bead string through key node operations. The system automatically calculates and renders the complete geometric shape of the bead string based on these parameters, which greatly improves the interaction efficiency while retaining the sense of participation in the craft.

[0053] The input parameters for the procedural bead string generation module include five categories: path node sequence, bead diameter, bead spacing, color sequence, and pattern rule. The path node sequence is determined by the user through gestures in 3D space, with a minimum of 3 and a maximum of 20 nodes. The bead diameter parameter is determined based on the actual specifications of the Lu Zhai bead lamp, typically set at 12 mm; this parameter affects the bead count calculation and collision detection range. The bead spacing parameter is defined as the straight-line distance between the centers of adjacent beads, typically set at 16 mm, with a range of 14 to 20 mm. The color sequence parameter defines the color arrangement rule of each bead along the length of the string, and can be a single-color sequence, an alternating sequence, or a gradient sequence. The pattern rule parameter references the pattern template defined in the pattern matching constraint rules of step 3.

[0054] The path reparameterization algorithm fits user-defined discrete nodes into a smooth curve, using Catmull-Rom spline curves as the basic algorithm for path fitting. The advantage of Catmull-Rom spline curves lies in automatically generating tangent directions through shape points, ensuring the first-order continuity of the curve at each shape point, resulting in a natural and smooth beaded path. The algorithm's implementation includes three stages: node preprocessing, parameterization calculation, and curve sampling. In the node preprocessing stage, redundancy detection and noise filtering are performed on the input nodes, removing adjacent nodes with excessively small spacing and abnormal nodes with excessive positional jitter. In the parameterization calculation stage, a cumulative chord length parameter value is assigned to each node, defined as the total path length from the starting node to the current node. In the curve sampling stage, N curve point coordinates are uniformly sampled along the parameter interval. The value of N is calculated based on the total path length and a preset sampling density, typically 200 sampling points per meter.

[0055] The number of beads is calculated based on the curve length and the geometric parameters of the beads. The mathematical expression for the calculation is as follows:

[0056] in Indicates the number of beads. Indicates the total length of the path curve. Indicates the diameter of the bead. This represents the bead spacing parameter. The rounding down operation in the formula ensures that the calculation result is an integer, and the number of beads is at least 1. The distribution position of the beads along the path is determined using the equal arc length parameterization method. This method ensures that the arc length between adjacent beads is equal, thereby achieving a uniform distribution effect.

[0057] The dynamic adjustment mechanism for bead spacing adaptively calculates based on pattern requirements and spatial constraints. When the pattern requires a bead at a specific location to contrast with adjacent beads, the system appropriately increases the spacing near that location to highlight the visual effect of that bead, with the adjustment not exceeding 30% of the original spacing. When the bead string path has a large curvature, the system automatically reduces the spacing of inner beads to avoid bead stacking, while appropriately increasing the spacing of outer beads to maintain overall visual balance. The spacing adjustment algorithm uses a curvature threshold and pattern priority as input parameters. A typical curvature threshold is 0.5 radians per meter; exceeding this threshold triggers the spacing adjustment logic.

[0058] The gradient transition of the color sequence is achieved using linear interpolation in the HSL color space. The color sequence parameters specify the HSL values ​​of the starting and ending colors. The colors of each bead along the length of the string are calculated through linear interpolation, with the interpolation coefficient being the ratio of the bead number to the total number of beads. The smoothness of the gradient transition is controlled by the interpolation order; first-order linear interpolation produces a linear gradient effect, while higher-order interpolation produces a smoother gradient transition. The alternating sequence is implemented using color switching logic: color A is used when the bead number is odd, and color B is used when the bead number is even. The color sequence parameters can also specify intermediate transition nodes; the system performs gradient interpolation calculations between each transition node to achieve multi-segment color gradient effects.

[0059] The bead geometry model is rendered using an instantiation rendering technique to ensure rendering efficiency. The bead geometry is stored only once in GPU memory, and the position, orientation, and color attributes of each bead are specified during rendering via instantiation parameters. The position attribute is the 3D coordinates of the bead's center in the model coordinate system; the orientation attribute is the rotation quaternion of the bead's local coordinate system relative to the model coordinate system; and the color attribute is the diffuse color value of the bead's material. The LOD level of the bead geometry dynamically switches based on the screen space projection area, and simplified geometry is used for distant beads to reduce rendering load.

[0060] Step 5: Establish a real-time feedback subsystem This step implements a real-time feedback subsystem, providing immediate audio-visual feedback to the user's actions, reinforcing the user's awareness of correct operation and guiding the correction of errors. The feedback subsystem includes two main categories: correct operation feedback and incorrect operation feedback, each corresponding to different feedback strategies and content.

[0061] Correct operation feedback is triggered after the user completes a legal operation that conforms to the process rules. The system completes the rendering and playback of all feedback effects within a preset response time of 100 milliseconds. The feedback response time is counted from the moment the operation is successfully determined, including the time spent on model update calculation, rendering command submission, and display refresh waiting. The goal is to control the end-to-end feedback latency within 100 milliseconds to ensure the immediate perception of feedback.

[0062] The adsorption effect is manifested as the component automatically moving to the target position accompanied by an elastic oscillation animation. Adsorption is triggered when the object enters an effective adsorption radius of 15 mm within the target position. The adsorption process uses a easing function to control position interpolation, employing an elastic easing algorithm with parameters including an elastic coefficient of 2.0 and 3 oscillations. The displacement amplitude of the elastic oscillation animation decays from an initial value of 10 mm with a decay coefficient of 0.7, and the animation duration is 400 milliseconds. The adsorption effect simultaneously triggers audio feedback, with a mechanical clicking sound. The volume adjusts according to the adsorption distance, increasing as the distance increases.

[0063] The luminous effect is manifested as the brightness of the component surface increasing from an initial value to a target value, accompanied by a halo diffusion animation. The brightness increase process uses an exponential easing function, with the initial brightness set to 0.2. The target brightness is determined based on the component type: 0.9 for bead-type components, 0.7 for lamp holder-type components, and 1.0 for light source-type components. The brightness change duration is 300 milliseconds. The halo diffusion animation is achieved using volumetric lighting rendering technology, expanding outwards from the component's center at a speed of 0.5 meters per second, with a maximum diffusion radius twice the component's size. The halo color is determined based on the component's color attributes, maintaining the hue while reducing saturation by 20% and increasing brightness by 30%.

[0064] The particle effect is achieved by ejecting a preset number of light-point particles from the surface of a component, which then drift naturally down along the direction of gravity. The particle emission points are random sampling points on the component's surface; the number of sampling points is determined by the component's surface area, typically ranging from 20 to 50. The initial particle velocity is 3 to 5 meters per second vertically upwards, the particle mass is 0.001 grams, and the air resistance coefficient is 0.1. The particle lifespan is 1.5 to 2.5 seconds, randomly determined at emission. The particle size decreases linearly with lifespan, starting at 5 millimeters and ending at 1 millimeter. The particle color uses a brightness-enhanced variant of the component's color attribute, with a 50% increase. Particle rendering employs point sprite technology, rendering each particle as a single luminous point in screen space.

[0065] The progress bar is presented as a 2D UI element on the interactive terminal screen, displaying the completion progress of the current process. The progress bar is a horizontal filled bar, filling from left to right, with a gold gradient to orange color. Below the progress bar is the ratio of the currently completed workpieces to the target number of workpieces, for example, "12 / 20 beaded strings have been attached". The progress bar updates immediately after each successful operation, using smooth interpolation with an interpolation time of 200 milliseconds for the update animation.

[0066] Error feedback is triggered when a user performs an illegal operation that violates process rules. The system immediately outputs an error message and guides the user to correct the operation. Error determination is based on the verification result of the process rule engine in step 3. Error feedback is triggered when the verification result indicates that execution is prohibited.

[0067] The node blinking effect serves as the first layer of error feedback, with a blinking frequency set to 5 Hz, or 5 blinks per second. The blinking color is the warning color with RGB values ​​(255, 80, 0). The blinking duration is 600 milliseconds, or 3 complete cycles. The blinking target is the component object currently being operated on, and the blinking area covers the entire geometric area of ​​the component. The blinking effect is accompanied by a short, high-frequency sound effect at 80% of normal volume and a pitch of 800 Hz.

[0068] The component rebound effect serves as a secondary error feedback mechanism, executing after the node blinks. The rebound is triggered when the user's manipulated object is in an incorrect position and the system determines it needs to be removed. The rebound animation is implemented using an elastic physics simulation algorithm, with an elasticity coefficient set to 500 N / m and a damping coefficient set to 0.08. The rebound direction is from the current position to the component's default storage location, with a rebound displacement of 1.5 times the target distance. The rebound animation duration is 350 milliseconds, and the animation curve uses a damped vibration model. After rebounding, the component automatically returns to its default storage location.

[0069] Short prompts serve as the third layer of error feedback, executing after node blinking and component bounce. The prompt content is selected from a pre-defined prompt library based on the error type. This library contains 50 guiding statements for different error scenarios. The library is stored using a key-value mapping between error codes and prompt text. Each prompt text includes an error description and a suggested correction. The prompt text is displayed as a floating box in the center of the screen with a semi-transparent black background and white text. The font size dynamically adjusts according to the screen resolution to ensure readability. The prompt box automatically fades out after 3 seconds.

[0070] The following is an example of the correspondence between error types and their corresponding prompts. When the error type is a violation of the process sequence, the prompt is "The preceding process has not been completed; please complete the current stage first," and the suggested correction is "Return to the previous step and continue production." When the error type is excessive positional deviation, the prompt is "This part should be installed in another location," and the suggested correction is "Try moving the part near the marked location." When the error type is a color scheme violation, the prompt is "This color does not coordinate with the overall color scheme," and the suggested correction is "Select the next color that conforms to the gradient rules."

[0071] Step 6: Integrate the cultural narrative trigger module This step implements the cultural narrative trigger module, which dynamically binds cultural content and craft nodes related to Lu Zhai beaded lanterns, allowing users to gradually unlock cultural knowledge as they complete the craft operations, thus transforming their experience from that of a viewer to a lantern maker.

[0072] The cultural narrative trigger module works in real time with the bead lamp completion detection module in step 3. The completion detection module calculates the current completion value of each process every 16 milliseconds. The completion value is represented in floating-point form, with a value range of 0.0 to 1.0. The trigger module subscribes to completion change events. When the completion value changes from below the threshold to above or equal to the threshold, the corresponding cultural content is unlocked.

[0073] The phased unlocking mechanism for cultural content is designed as follows. The first phase unlocks narratives of artisan labor, triggered by achieving a bead-threading completion rate of 0.95. This phase includes the historical origins of bead-threading, artisans' techniques and rhymes, the cultural significance of the colored beads, and the physical exertion involved in the bead-threading process. It is presented as a floating text box on the side of the screen, occupying 20% ​​of the screen width, with the text displayed in a vertical calligraphy style. A 90-second audio narration is played simultaneously, with the audio content matching the text. When a user first sees the first phase content after completing the bead-threading process, the system automatically pauses the interaction, resuming interaction after the user finishes reading or clicks the skip button.

[0074] The second phase unlocks a narrative of structural order, triggered when the lamp assembly reaches a preset completion threshold of 1.0. This phase includes the symbolic meaning of the lamp's hierarchical structure, the cultural connotations of the canopy, the spatial function of the main lamp, the correspondence between the beaded strings and the hall's beams, and the overall structural mechanical balance principles. Presented as floating labels anchored in three-dimensional space, these labels move with the corresponding lamp components. When the user looks at a component, the label automatically expands to display detailed text descriptions. Perspective projection is used to ensure the labels' accurate positioning in three-dimensional space, and the depth test threshold for the label content is set to within 5 meters of the camera.

[0075] The third stage unlocks a narrative with auspicious symbolism, triggered by achieving a tassel completion rate of 1.0. This stage includes the auspicious symbolism of tassels, cultural interpretations of five main patterns, the festive uses of Lu Zhai beaded lanterns, the cultural connections between beaded lanterns and other traditional crafts, and the family lineage of beaded lantern making techniques. Presented as narrative cards with mixed text and images, each 800×600 pixels with a Xuan paper textured background, the cards contain a title, illustration, and explanatory text. Users can switch between different themed narrative cards by swiping, and each card supports zooming to view details.

[0076] The fourth stage unlocks the historical narrative of the Lu Residence's main hall, triggered by the lighting operation. This operation is initiated by the user through a lifting gesture. Once the system determines the lifting posture is correct and the overall completion level of the lamp reaches 1.0, it executes the lighting sequence animation. The lighting sequence animation lasts 8 seconds and includes three stages: a power-on animation, a gradual brightening animation, and a light diffusion animation. The power-on animation lasts 1 second, depicting the mechanical action of a switch being toggled; the gradual brightening animation lasts 3 seconds, using an exponential easing function to control the brightness increase from 0 to 1; and the light diffusion animation lasts 4 seconds, with light spreading outwards from the center of the lamp to illuminate the entire hall space.

[0077] The fourth stage unlocks after the lighting sequence animation is completed, triggering a scene transition to the Suyong Hall 3D digital space of the Lu Residence. The scene transition uses a fade-in / fade-out transition effect with a duration of 2 seconds. The Suyong Hall 3D space model is at LOD2 level precision, and its dimensions are modeled 1:1 according to the actual architectural dimensions of Suyong Hall. The space includes four main architectural areas: the entrance hall, front hall, central hall, and rear hall, as well as the beams, column bases, plaques, couplets, and furnishings within the halls. Beaded lamps hang in the main hall of the central hall, consistent with the actual layout of Suyong Hall.

[0078] The fourth phase of cultural content encompasses four narrative themes. The first dimension is the historical origins of the Lu Residence's beaded lanterns, including the construction history of the residence, the creation date of the lanterns, the evolution of their status within the residence, and their restoration and transmission. The second dimension is the spatial relationships within the halls, including the architectural style of the Suyong Hall, the functional zoning of the halls, the correspondence between the lanterns and the hall spaces, and the role of the lanterns in hall ceremonies. The third dimension is the festive lantern-lighting ceremony, including the traditional festive customs of the Lu Residence, the timing and process of the lantern-lighting ceremony, the participants, and the cultural significance of the ceremony. The fourth dimension is the story of intangible cultural heritage transmission, including the lineage of beaded lantern making techniques, the craftsmanship characteristics of representative inheritors, the endangered status of beaded lantern making techniques, and contemporary protection and transmission measures.

[0079] The narrative content is presented as floating text anchored in three-dimensional space and accompanied by audio narration. The floating text is anchored in specific locations within the Suyong Hall space; for example, text about historical origins is anchored in the entrance hall area, text about hall spaces is anchored in the main hall area, text about festivals and ceremonies is anchored near the beaded lamps, and text about heritage stories is anchored below the plaques. A semi-transparent background frame ensures readability under different lighting conditions, with a background transparency set to 60%. The text font size dynamically adjusts based on the user's viewing distance, with an adjustment factor of 0.05 times the viewing distance. The minimum font size is 24 pixels, and the maximum font size is 48 pixels, ensuring clear reading regardless of distance. Audio narration plays synchronously with the floating text, with a total audio duration of 180 seconds. Users can pause the audio or adjust the playback progress by clicking on the text box.

[0080] Example 2 This embodiment describes another specific application scenario of the gesture-driven bead lamp experience method that integrates process rules. This scenario is aimed at the youth study tour education user group, and adaptive adjustments have been made in terms of technical configuration and interaction design.

[0081] The system configuration adjustments for the youth study tour scenario are as follows: The gesture recognition module has undergone parameter calibration for the hand size characteristics of youth users, and the training dataset for the joint tracking algorithm includes image samples of youth hands. The grasping distance threshold in the gesture judgment parameters has been adjusted to 35 mm, and the lifting speed threshold has been adjusted to 120 mm per second. The gesture recognition module has also integrated a child safety mode, in which the system automatically blocks adult users' gestures to prevent misrecognition, and monitors and reminds minors of their operation time.

[0082] The process rule engine has simplified the rules to suit the cognitive level of teenage users. The process sequence constraints remain unchanged to cultivate users' process awareness, the deviation threshold for structural position constraints has been relaxed to 25 mm to reduce the difficulty of operation, the color rhythm constraints and pattern matching constraints have been adjusted to suggestive rules rather than mandatory rules, and the completion constraint threshold has been uniformly adjusted to 0.85 to speed up the experience.

[0083] The cultural narrative trigger module has been restructured to suit the learning characteristics of teenage users. The content of the first to fourth stages has been adapted into four themes: the story of the little bead-stringing craftsman, exploring the structure of beaded lanterns, the code of auspicious patterns, and the legend of the Lu family. Interactive Q&A sessions have been added to the content presentation. After each narrative stage, two to three multiple-choice questions related to the content of that stage will pop up. Users can unlock the next narrative stage or receive a virtual badge reward after answering correctly.

[0084] The real-time feedback subsystem has been enhanced to better suit the feedback preferences of teenage users. The number of particle effects for correct operation feedback has been doubled, the glowing effect has been updated with a flashing animation sequence, and the sound effects have been enhanced with upbeat background music clips. The prompts for incorrect operation feedback have been changed to an encouraging tone, and short prompts have been given voice broadcast functionality to help users with limited literacy understand the prompts.

[0085] The procedural bead generation module has been optimized for the operation characteristics of teenage users. Users can input the bead length parameter via voice commands, and the system will automatically calculate and render the complete bead string, reducing the complexity of gesture operations. Users can also select a target difficulty level from 5 preset difficulty levels, which affects the preset number of path nodes and the adjustment range of bead spacing.

[0086] The typical user experience flow in this application scenario is designed as follows. After entering the system, the user first selects the difficulty level and color theme. The system then loads the corresponding pattern templates and color schemes based on the user's selection. The user then completes gesture calibration, during which the system demonstrates the key points of seven basic gestures through animation. After entering the digital workshop space, the system introduces the basic structure of the beaded lamp and the overall goal of this project through voice explanation and animation demonstration.

[0087] Users complete each step in the following order: selecting beads, stringing beads, assembling lamps, and attaching tassels. After each step is completed, the system triggers corresponding narrative content and interactive Q&A. After activating the lighting ceremony with a lifting gesture, the system switches to the three-dimensional space of the Suyong Hall. Users explore the hall space and view anchored narrative text using rotation and zoom gestures. Finally, the system generates a lamp-making record card for the user, including a review of the user's steps, a list of earned virtual badges, and recommended follow-up learning content.

[0088] This invention also provides other variant implementations besides the two embodiments described above, all of which are within the protection scope of this invention. Any equivalent transformations and combinations made according to the technical solutions of this invention should be considered as specific applications of the technical solutions of this invention within the scope of this application.

[0089] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for experiencing gesture-driven beaded lamps that integrates process rules, comprising the following steps: Step 1: Establish a 3D digital model of the beaded lantern. Through a combination of 3D scanning and manual modeling, the components of the Lu Residence beaded lantern, including colored beads, bead strings, lantern frames, canopies, tassels, patterns, light sources, and hall spaces, are transformed into a detachable, assembleable, and illuminateable digital beaded lantern model, forming a complete hierarchical dataset of the beaded lantern. Step 2: Configure a gesture recognition and mapping module. Using a depth camera or infrared sensor, depth images of the user's hand are collected, and feature parameters such as palm position, finger posture, and movement trajectory are extracted. Seven basic gesture actions—grasping, pinching, dragging, rotating, closing, pulling down, and lifting—are mapped to five process operation commands: selecting beads, stringing beads, assembling lanterns, hanging tassels, and lighting the lanterns. Step 3: Deploy a process rule engine. Establish a five-category process rule library containing constraints on process sequence, structural position, color rhythm, pattern matching, and completion. The system performs real-time corrections. Step 1 verifies whether user operations conform to the traditional logic of bead lamp making, and judges and triggers correction guidance for operations that violate the process rules; Step 4 builds a programmatic bead string generation module, where users set the starting point, ending point and intermediate nodes of the bead string through key node operations, and the system automatically generates a complete bead string based on parameters such as the number of beads, the spacing between beads, the color sequence parameters and the pattern rules parameters; Step 5 establishes a real-time feedback subsystem, where correct operations trigger adsorption effects, light-emitting effects, particle effects, sound effects and progress bar feedback, and incorrect operations trigger node flashing effects, component rebound effects and short sentence prompt feedback; Step 6 integrates a cultural narrative trigger module, where cultural content is dynamically unlocked as accompanying information of the process nodes, completing bead stringing triggers the craftsman's labor narrative, completing lamp assembly triggers the structural order narrative, completing tassel hanging triggers the auspicious meaning narrative, and completing lamp lighting triggers the historical narrative of the Lu Residence hall.

2. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 1, the 3D scanning adopts a multi-angle surround acquisition scheme, the acquisition angle interval adopts a preset angle interval, the scanning point cloud density of a single bead lamp component adopts a preset density of 3D coordinate points, and multi-view point cloud registration is performed by iterative nearest point algorithm.

3. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 1, the manual modeling part adjusts the physical simulation parameters for the dynamic deformation of the bead string and the natural draping of the tassel. The bead string is discretized into several mass nodes and connecting springs for flexible chain rigid body simulation, and the tassel is discretized into several particle points for particle rope simulation. The generated digital bead lamp model contains hierarchical structural relationship data with a preset number of layers.

4. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 2, the gesture recognition module uses a deep learning-based joint tracking algorithm to extract the three-dimensional coordinates of a preset number of hand joints and the spatial orientation vectors of a preset number of palm regions from the depth image. The judgment period for gesture actions is a preset judgment period, and the minimum duration of a single gesture is a preset minimum duration to prevent misrecognition.

5. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 2, the conditions for determining the grasping gesture are that the distance between the thumb and the other four fingers is less than a preset distance threshold and the holding time exceeds a preset holding time. The conditions for determining the pinching gesture are that the distance between the fingertips of the thumb and the index finger is less than a preset pinching distance threshold and decreases rapidly. The conditions for determining the dragging gesture are that the overall displacement of the palm in three-dimensional space exceeds a preset displacement threshold and the movement trajectory is smooth.

6. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 3, the process rule engine adopts a hybrid architecture that combines production rules and state machines. The state machine manages the macro-level process progress and stage status, while the production rules are responsible for micro-level operation verification and constraint determination. State transitions include forward transition conditions and backtracking transition conditions.

7. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 3, the process sequence constraint stipulates that bead selection must be completed before bead threading, bead threading must be completed before lamp assembly, lamp assembly must be completed before tassel hanging, and tassel hanging must be completed before lighting. If any process violates the sequence rule, the system will lock the interaction permissions of subsequent processes until the current process is completed.

8. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 3, the structural position constraints stipulate that the bead string can only be hung on the preset hanging point of the lamp holder, the tassel can only be hung on the preset hanging point of the main lamp, and the canopy must cover the main lamp. Operations with a three-dimensional spatial position deviation exceeding the preset deviation threshold are judged as erroneous operations and trigger component rebound feedback.

9. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 3, the color rhythm constraint stipulates that the color matching of a single bead string follows the principle of gradual transition, and the hue difference between adjacent colored beads must not exceed the preset hue difference threshold. The pattern matching constraint stipulates that when stringing beads, users must select the colored bead colors according to the preset pattern template sequence.

10. The gesture-driven bead lamp experience method based on the fusion process rules according to claim 1, characterized in that: In step 4, the programmed bead string generation module uses a path reparameterization algorithm to fit the user-defined discrete nodes into a smooth curve. Based on the curve length and bead diameter parameters, the number of beads is calculated. The bead spacing is dynamically adjusted within a preset spacing range according to the pattern requirements. The color sequence gradually transitions along the length of the bead string according to preset color gamut rules.

Images