An interactive education projection display method based on gesture control

Abstract

The application discloses an interactive education projection display method based on gesture control and relates to the technical field of data processing, which comprises the following steps: acquiring original gesture trajectory data in a continuous time sequence, obtaining first gesture trajectory data through space mapping processing, continuously distributing the first gesture trajectory data, generating attention distribution data, and extracting an interactive focus; determining a knowledge entity set based on the attention distribution data, correcting a space correlation relationship through a gravitational radius mechanism, and obtaining mapping relationship data; analyzing the topological relationship of the knowledge entity, checking and adding a logical attribute label according to a preset subject logic rule, and updating coordinate data by using a filtering algorithm; and finally adjusting visual parameters according to the logical attribute label and the entity type, and synchronously outputting corresponding interactive content. Through filtering denoising, coordinate adsorption, logical checking and hierarchical feedback, the application effectively improves the accuracy, smoothness and teaching intelligent level of gesture interaction.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an interactive educational projection display method based on gesture control. Background Technology

[0002] In recent years, with the development of smart education, interactive projection technology has been widely used in classroom teaching. Existing interaction methods mainly rely on laser pointers, infrared styluses, or direct gesture recognition. Among these, gesture recognition technology, due to its lack of external devices and natural interaction, has become a research hotspot in the field of interactive education. In teaching scenarios, teachers use gestures to rotate, move, or scale the projected knowledge entities to achieve intuitive demonstrations.

[0003] However, existing gesture-controlled projection display methods have the following limitations in practical teaching applications: First, due to limitations in sensor accuracy and ambient light interference, the captured raw gesture coordinate trajectories often exhibit high-frequency jitter. During precise operations, these subtle coordinate fluctuations can cause visual flickering or accidental touches on the projected image, affecting the continuity of the teaching demonstration. Secondly, most existing systems only implement physical mapping between coordinates, lacking the ability to determine the semantic logic between user intentions and knowledge entities. When a user's gesture approaches multiple densely arranged entities, the system struggles to accurately identify the user's target object, resulting in interactive feedback that does not conform to the teaching logic. In addition, the existing feedback methods are relatively simple, failing to automatically verify the correctness of the operation based on the subject logic, and failing to trigger in-depth multimedia teaching content simultaneously during gesture interaction, resulting in a low level of immersion and intelligence in human-computer interaction. Summary of the Invention

[0004] The technical problem solved by this invention is that existing gesture recognition technologies in interactive educational projection scenarios suffer from problems such as high-frequency jitter, visual flicker, and accidental touches in gesture trajectories due to sensor accuracy and environmental interference. Furthermore, the system lacks the ability to determine the semantic logic between gesture operation intentions and knowledge entities, making it difficult to accurately identify target objects when densely packed entities are arranged. At the same time, the feedback is limited and cannot automatically verify the correctness of operations or trigger in-depth multimedia content based on subject logic, resulting in low stability, intelligence level, and immersion in human-computer interaction.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: an interactive educational projection display method based on gesture control, comprising the following steps: Step S1: Obtain the original gesture trajectory data generated by the interactive subject in a continuous time series, perform spatial mapping processing on the original gesture trajectory data to obtain the first gesture trajectory data, and convert the first gesture trajectory data to the pixel coordinate system corresponding to the projection display canvas. Step S2: Perform continuous distribution processing on the first gesture trajectory data to obtain attention distribution data, and perform time evolution update on the attention distribution data to extract the interaction focus; Step S3: Based on the attention distribution data, determine the set of knowledge entities corresponding to the interaction focus, and correct the spatial relationship between the interaction focus and the set of knowledge entities to obtain the mapping relationship data; Step S4: Perform topological relationship parsing on the set of knowledge entities at the interaction focus in the mapping relationship data, verify the topological association data according to the preset subject logic rules, add corresponding logical attribute labels to the mapping relationship data, update the pixel coordinate values ​​in the mapping relationship data using a filtering algorithm, and output the updated mapping relationship data. Step S5: Obtain the updated mapping relationship data, extract graphical interface instructions based on logical attribute labels and knowledge entity type identifiers, adjust the visual parameters of the corresponding knowledge entities in the projection display canvas, and synchronously output the interactive content corresponding to the logical attribute labels.

[0006] As a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S1 specifically includes: Step S11: Capture key point location information of the interactive subject in a continuous time series using a visual sensor to generate raw gesture trajectory data; The interactive subject includes a human hand, a human torso, and a guiding medium carried by the human body to perform teaching demonstration actions; The guidance medium carried by the human body includes a pointer with a fixed outline, a marker with reflective properties, and a laser pointer that emits light signals. The key point location information includes the physical coordinates of the interactive subject in the three-dimensional sensing space and the recognition confidence value; The original gesture trajectory data includes spatial coordinates, timestamps corresponding to the spatial coordinates, and key point type identifiers corresponding to the spatial coordinates. The key point type identifiers corresponding to the spatial coordinates include semantic tags corresponding to the fingertips, the center of the palm, and the joint points in the human hand; Step S12: Establish the spatial mapping relationship between the projection display canvas and the three-dimensional sensing space. Use the homography transformation matrix to project the original gesture trajectory data onto the two-dimensional coordinate space corresponding to the projection display canvas to obtain the projection trajectory coordinates. Step S13: Normalize the coordinates of the projection trajectory and match them to the pixel coordinate system corresponding to the projection display canvas to obtain the first gesture trajectory data.

[0007] As a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S12 specifically includes: Step S121: Obtain the intrinsic and extrinsic parameter matrices of the visual sensor, and establish the perspective projection relationship from the three-dimensional sensing space to the two-dimensional camera coordinate system based on the intrinsic and extrinsic parameter matrices. Step S122: By sampling the coordinates of the four corner points of the projection display canvas, the homography transformation matrix between the two-dimensional camera coordinate system and the coordinate space of the projection display canvas is calculated using the homography transformation algorithm. Step S123: Use the homography transformation matrix to perform a linear transformation on the physical coordinates in the original gesture trajectory data to obtain the mapped coordinates of the original gesture trajectory data in the two-dimensional coordinate space.

[0008] As a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S2 specifically includes: Step S21: Extract the pixel coordinates and corresponding recognition confidence values ​​from the first gesture trajectory data. Use a Gaussian kernel function to perform energy mapping on each pixel coordinate point to generate a continuous initial attention potential energy distribution. The calculation formula is as follows: ; in, This represents the initial attention potential energy value at any coordinate point in a continuous initial attention potential energy distribution. This represents the total number of keypoints within the current sampling time. This indicates the index number of the key point within the current sampling time. Indicates the first The confidence score of the identification of each key point Indicates the first The key point is the x-coordinate of the pixel coordinates in the coordinate space of the projected display canvas. Indicates the first The key point is the vertical coordinate of the pixel in the coordinate space of the projected display canvas. This represents the x-coordinate of any pixel in the coordinate space of the projected display canvas. This represents the ordinate of any pixel in the coordinate space of the projected display canvas. Represents the spatial diffusion operator; Step S22: Read the attention distribution data generated at the previous sampling time from the preset cache space as the historical attention distribution, introduce a time decay mechanism, and iteratively update the historical attention distribution to obtain the attention distribution data at the current time. The calculation formula is as follows: ; in, This represents the coordinates in the attention distribution data at the current moment. The potential energy value at that location. This represents the corresponding coordinate in the attention distribution data generated at the previous sampling time. The potential energy value at that location. Represents the natural constant. Indicates the time decay coefficient. This represents the time interval between the current sampling time and the previous sampling time. Indicates the current moment; Step S23: Store the attention distribution data at the current moment into a preset cache space, and perform a local maximum search on the attention distribution data at the current moment to extract the coordinates corresponding to the potential energy peak as the interaction focus.

[0009] As a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S23 specifically includes: Step S231: Traverse all pixel coordinates in the attention distribution data, filter out pixel coordinates with potential values ​​greater than a preset threshold, and construct an active interaction region coordinate set. Step S232: Within the coordinate set of the active interaction region, use a search algorithm to find the local potential energy maxima. The local potential energy maxima must satisfy the discriminant: ; in, To represent any one, Indicates The pre-defined neighborhood space centered on the center Represents any coordinate within the preset neighborhood space at the current time. The potential energy value at that location; Step S233: Based on the potential energy values ​​of each coordinate point within the active interaction area coordinate set, the centroid weight is calculated to obtain the pixel coordinates corresponding to the interaction focus. The calculation formula is as follows: ; in, This represents the x-coordinate of the pixel coordinates corresponding to the focus point. The y-coordinate represents the pixel coordinates corresponding to the focus of the interaction. This represents the coordinate set of the active interaction area.

[0010] As a preferred embodiment of the gesture-controlled interactive educational projection display method described in this invention, step S3 includes: Step S31: Based on the pixel coordinates corresponding to the interactive focus, retrieve the set of knowledge entities in the projection display canvas, and obtain the center pixel coordinates, type identifier, and preset gravitational radius corresponding to the type identifier of each knowledge entity; The knowledge entity set includes text objects, geometric figures, and interactive components; The type identifiers include text object identifiers, geometric shape identifiers, and interactive component identifiers; Step S32: Calculate the Euclidean distance between the pixel coordinates corresponding to the interactive focus and the center pixel coordinates of each knowledge entity, and determine whether the Euclidean distance satisfies the following conditions. ; in, Indicates the preset gravitational radius. Represents Euclidean distance; If the conditions are met, it is determined that there is a semantic relationship between the interaction focus and the corresponding knowledge entity, and the knowledge entity is identified as the target knowledge entity. Step S33: Based on semantic association, the pixel coordinates corresponding to the interaction focus are corrected using the adsorption vector formula to generate mapping relationship data. The calculation formula is as follows: ; in, This represents the pixel coordinates corresponding to the mapping relationship data. This indicates the pixel coordinates corresponding to the focus of the interaction. This represents the adsorption weight coefficient. Represents the center pixel coordinates of each knowledge entity.

[0011] In a preferred embodiment of the gesture-based interactive educational projection display method described in this invention, the processing logic for the adsorption weight coefficient includes: Get the potential energy value at the pixel coordinates corresponding to the interactive focus and the highest potential energy value in the entire projected display canvas. Based on the potential energy value, the maximum potential energy value, and the preset sensitivity adjustment factor, the adsorption weight coefficient is calculated using the following formula: ; in, Represents the pixel coordinates of the interactive focus. The potential energy value at that location. This indicates the highest potential energy value across the entire projected display canvas. This indicates the preset anti-lock threshold. This represents the preset sensitivity adjustment factor, with a value range of [value range missing]. , This indicates the minimum operator; Based on the type identifier in each knowledge entity, extract the value corresponding to the type identifier from the preset mapping table as the preset gravitational radius; The preset gravitational radius of the text object identifier is smaller than that of the preset gravitational radius of the interactive component identifier.

[0012] In a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S4 specifically includes: Step S41: Extract the pixel coordinates of each group in the mapping relationship data according to the timestamp order, construct the dynamic behavior trajectory sequence associated with each knowledge entity, and process the dynamic behavior trajectory sequence using the moving average filtering algorithm to obtain the updated mapping relationship data. Step S42: Calculate the topological association data between the dynamic behavior trajectory sequence and the geometric boundary of the corresponding knowledge entity; The topological association data includes containment relationships, intersection relationships, and tangency relationships; Step S43: Based on the type identifier of each knowledge entity, retrieve matching verification rules from the preset subject logic rule base, and input the topological association data into the verification rules for comparison; If the topological association data meets the validation rules, then add a first logical attribute label to the mapping relationship data; If the topological association data does not meet the validation rules, a second logical attribute label is added to the mapping relationship data.

[0013] As a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S41 specifically includes: Step S411: Within a preset sliding time window, extract the pixel coordinates from the mapping relationship data corresponding to each sampling time. Step S412: Based on semantic association, pixel coordinates of objects with the same association are grouped into the same data cluster; Step S413: For each data cluster, the pixel coordinates of objects with the same association are linearly arranged according to the order of timestamps to generate a dynamic behavior trajectory sequence that corresponds one-to-one with the target knowledge entity. Step S414: The moving average filtering algorithm is used to calculate the mean of the coordinate components in the dynamic behavior trajectory sequence, and the pixel coordinate values ​​of each coordinate point in the dynamic behavior trajectory sequence are updated to obtain the updated mapping relationship data.

[0014] As a preferred embodiment of the gesture-controlled interactive educational projection display method of the present invention, step S5 specifically includes: Step S51: Obtain the updated mapping relationship data, extract the logical attribute tags and type identifiers corresponding to each knowledge entity carried by the updated mapping relationship data, and extract the corresponding graphical interface instructions from the preset interactive feedback database. The logical attribute tags include a first logical attribute tag and a second logical attribute tag; Step S52: Modify the visual parameters of the corresponding knowledge entity in the projection display canvas according to the graphical interface instructions. The visual parameters include color values, transparency values, and line width values; Step S53: If the logical attribute tag is the first logical attribute tag, then the preset multimedia file associated with the knowledge entity is triggered. If the logical attribute label is the second logical attribute label, then the preset guide label will be output in the projection display canvas according to the graphical interface instructions; Step S54: Refresh the layer buffer and output the modified visual parameters and the interactive content corresponding to the logical attribute labels in the projection display canvas. The interactive content includes a preset multimedia file corresponding to the first logical attribute tag and preset guidance identifier data corresponding to the second logical attribute tag.

[0015] The beneficial effects of this invention: This invention innovatively proposes a gesture interaction control logic that integrates physical correction, semantic verification and hierarchical feedback. First, a moving average filtering algorithm is used to average the coordinate components in the dynamic behavior trajectory sequence, filtering out high-frequency noise signals caused by sensor limitations or stray light from the ground up. Combined with a dynamic update mechanism for pixel coordinate values, this significantly improves the smoothness of the knowledge entity's movement trajectory and effectively eliminates visual flicker and accidental touches. Second, by setting a gravitational radius and constructing a mapping relationship between gesture interaction focus and knowledge entity, automatic coordinate snapping correction is achieved, completing the automatic calibration from physical coordinates to logical positions, significantly reducing the operational difficulty of non-contact interaction. On this basis, by extracting type identifiers and performing topological association data verification, the system is given the ability to recognize user operation intentions. Logical attribute tags are used to make interactive feedback conform to preset educational semantics, realizing the transformation from a simple visual follow-up to a teaching auxiliary terminal with logical judgment capabilities. Finally, a hierarchical feedback mechanism is constructed using logical attribute tags. When the judgment result is correct, it triggers associated multimedia files, while when the judgment result is incorrect, it outputs guidance identifier data. Through the differentiated combination of visual parameters and interactive content, real-time evaluation and accurate correction in the teaching process are ensured, greatly improving the intelligence level and immersive experience of human-computer interaction. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the steps of an interactive educational projection display method based on gesture control, as provided in one embodiment of the present invention. Detailed Implementation

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] Example, refer to Figure 1 This paper provides an interactive educational projection display method based on gesture control, which includes the following steps: Step S1: Obtain the original gesture trajectory data generated by the interactive subject in a continuous time series, perform spatial mapping processing on the original gesture trajectory data to obtain the first gesture trajectory data, and convert the first gesture trajectory data to the pixel coordinate system corresponding to the projection display canvas. Step S2: Perform continuous distribution processing on the first gesture trajectory data to obtain attention distribution data, and perform time evolution update on the attention distribution data to extract the interaction focus; Step S3: Based on the attention distribution data, determine the set of knowledge entities corresponding to the interaction focus, and correct the spatial relationship between the interaction focus and the set of knowledge entities to obtain the mapping relationship data; Step S4: Perform topological relationship parsing on the set of knowledge entities at the interaction focus in the mapping relationship data, verify the topological association data according to the preset subject logic rules, add corresponding logical attribute labels to the mapping relationship data, update the pixel coordinate values ​​in the mapping relationship data using a filtering algorithm, and output the updated mapping relationship data. Step S5: Obtain the updated mapping relationship data, extract graphical interface instructions based on logical attribute labels and knowledge entity type identifiers, adjust the visual parameters of the corresponding knowledge entities in the projection display canvas, and synchronously output the interactive content corresponding to the logical attribute labels.

[0019] In specific implementation, step S1 includes: Step S11: Capture key point location information of the interactive subject in a continuous time series using a visual sensor to generate raw gesture trajectory data; The interactive elements include the human hands, torso, and guiding medium carried by the human body to perform the teaching demonstration actions; Human-carried guidance media include pointers with fixed outlines, markers with reflective properties, and laser pointers that emit light signals; Key point location information includes the physical coordinates of the interactive subject in the three-dimensional sensing space and the recognition confidence value; The raw gesture trajectory data includes spatial coordinates, the timestamps corresponding to the spatial coordinates, and the key point type identifiers corresponding to the spatial coordinates; The key point type identifiers corresponding to spatial coordinates include semantic tags corresponding to the fingertips, the center of the palm, and the joints in the human hand; Step S12: Establish the spatial mapping relationship between the projection display canvas and the three-dimensional sensing space. Use the homography transformation matrix to project the original gesture trajectory data onto the two-dimensional coordinate space corresponding to the projection display canvas to obtain the projection trajectory coordinates. Step S13: Normalize the coordinates of the projection trajectory and match them to the pixel coordinate system corresponding to the projection display canvas to obtain the first gesture trajectory data.

[0020] Specifically, the system captures key point location information of the interactive subject within a continuous time series using a visual sensor to generate raw gesture trajectory data. This process relies on a depth camera, which synchronously acquires RGB images and depth information at a fixed frame rate. In each frame, a key point detection algorithm identifies preset key parts on the interactive subject and outputs their coordinates in the two-dimensional image. Then, combining the depth value corresponding to that pixel, a back-projection calculation is performed using the camera's intrinsic parameter matrix, thereby converting the two-dimensional image coordinates into physical coordinates in the three-dimensional sensing space. The system records each coordinate point along with its corresponding timestamp, key point type identifier, and recognition confidence score, arranging them chronologically to form the raw gesture trajectory data.

[0021] Step S12 mainly transforms the three-dimensional sensing space coordinates into a two-dimensional projected coordinate system and eliminates viewpoint distortion. Step S13 further processes these two-dimensional coordinates to adapt them to the actual display allocation rate. Through normalization and resolution matching, the coordinates can be correctly mapped to the pixel positions on the screen.

[0022] After obtaining the original gesture trajectory data, a spatial mapping relationship between the projection display canvas and the three-dimensional sensing space is established, and the original gesture trajectory data is projected onto the two-dimensional coordinate space corresponding to the projection display canvas using a homography transformation matrix. After projection, the trajectory coordinates need to be normalized for easier subsequent processing. Specifically, the system finds the maximum and minimum values ​​of all projected coordinates in the X and Y directions, and scales the coordinate values ​​to a standard range, such as the [0,1] interval, through a linear transformation. This step aims to eliminate scale differences in the original data. Finally, the system matches the normalized coordinates to the pixel coordinate system corresponding to the resolution in the projection display canvas. This resolution refers to the logical canvas size defined internally by the system and matched to the final display device, such as 1920 pixels × 1080 pixels. The matching process involves multiplying the X and Y components of the normalized coordinates by the width and height values ​​corresponding to the canvas resolution, thereby mapping the continuous normalized coordinates to discrete screen pixel positions, obtaining the first gesture trajectory data that can be directly used for interface interaction. The purpose of preset resolution is to provide a stable and unified absolute reference system for the entire interactive space, ensuring that gesture coordinates can be precisely aligned with the knowledge entities laid out on the screen. This is the technical foundation for ensuring interaction determinism and visual consistency.

[0023] In specific implementation, step S12 includes: Step S121: Obtain the intrinsic and extrinsic parameter matrices of the visual sensor, and establish the perspective projection relationship from the three-dimensional sensing space to the two-dimensional camera coordinate system based on the intrinsic and extrinsic parameter matrices. Step S122: By sampling the coordinates of the four corner points of the projection display canvas, the homography transformation matrix between the two-dimensional camera coordinate system and the coordinate space of the projection display canvas is calculated using the homography transformation algorithm. Step S123: Use the homography transformation matrix to perform a linear transformation on the physical coordinates in the original gesture trajectory data to obtain the mapped coordinates of the original gesture trajectory data in the two-dimensional coordinate space.

[0024] Specifically, first, the system needs to know the position of the gesture within the camera's "eye." This is achieved by establishing a perspective projection relationship. Depth cameras have their own intrinsic parameters (intrinsic matrix, containing information such as focal length and principal point) and their position and pose in the world (extrinsic matrix, containing rotation and translation). When a gesture keypoint is captured with its physical coordinates in 3D sensing space, the system uses the extrinsic matrix to transform this point into the camera's own coordinate system, and then performs perspective projection using the intrinsic matrix. This projection process simulates the pinhole camera principle, flattening the 3D point onto the camera's 2D imaging plane, outputting a 2D pixel coordinate. This coordinate precisely describes which pixel the gesture point appears on in the camera's view, but it carries perspective distortion from the camera's perspective.

[0025] The image coordinates seen by the camera cannot be directly used for projection interaction because the projector projects the image onto the canvas from a different angle. To correct this perspective deviation, the system needs to perform a coordinate system calibration, i.e., calculate the homography transformation matrix. Specifically, the system controls the projector to project clear markers, such as light spots, at the four corners of the canvas, while simultaneously capturing an image containing these four markers with the camera. Through image recognition, the system can simultaneously determine the ideal coordinates of these four points in the canvas coordinate system defined by the projector, as well as their actual pixel coordinates in the image captured by the camera. With these four sets of one-to-one corresponding coordinate points, the system can use the homography transformation algorithm to solve for a 3x3 transformation matrix. This matrix contains all the geometric correction information needed to project from the camera image plane to the projection canvas plane, including rotation, scaling, shearing, and perspective transformations. Its function is to correct the "distorted" image from the camera's perspective back to the orthogonal perspective of the projection canvas.

[0026] By concatenating the two transformations described above, the mapping from physical space to the interactive canvas is completed. For each original gesture trajectory data point with its three-dimensional physical coordinates, the system first calculates its two-dimensional coordinates in the camera image through perspective projection. Then, it multiplies these two-dimensional coordinates with the previously obtained homography transformation matrix. In homogeneous coordinates, this multiplication operation performs a linear transformation on the coordinates, automatically corrects the viewing angle, and outputs the final mapped coordinates of the gesture point in the unified coordinate space of the projection display canvas. Thus, a teacher's gesture at a certain position in the air is stably and accurately converted into the position of a specific pixel on the large screen, laying a precise spatial foundation for subsequent judgment of gesture intent. The stability and accuracy of this mapping relationship are the technological cornerstone for the smooth and seamless operation of the entire gesture interaction system.

[0027] This application significantly improves the accuracy of gesture interaction in practical teaching scenarios by constructing a rigorous geometric mapping logic. Compared to traditional techniques that rely solely on simple linear scaling or manual alignment, this application establishes the perspective projection relationship from the 3D sensing space to the 2D camera coordinate system by obtaining the intrinsic and extrinsic parameter matrices of the visual sensor. The technical advantage of this process lies in its ability to compensate for geometric distortions caused by sensor installation position and angular deviations through mathematical modeling, ensuring accurate restoration of the physical position of gestures even under non-parallel viewpoints. Based on this, a homography transformation matrix is ​​calculated by sampling the coordinates of the corner points of the projection canvas. Its core benefit is the establishment of an automated correction mechanism that eliminates rotation, scaling, and perspective distortion between the camera image plane and the projection plane, enabling the system to quickly adapt to different hardware deployment environments. Finally, the homography transformation matrix is ​​used to perform a linear transformation on the original physical coordinates, converting the complex spatial mapping into efficient matrix operations and directly outputting the mapped coordinates in the 2D coordinate space. This design not only solves the problem of inaccurate gesture landing points caused by perspective shift in traditional solutions, but also provides a stable and unified absolute reference system, laying a solid technical foundation for subsequent precise gravitational radius adsorption, topological relationship analysis, and discipline logic verification, greatly enhancing the determinism and visual consistency of non-contact interaction.

[0028] In specific implementation, step S2 includes: Step S21: Extract the pixel coordinates and corresponding recognition confidence values ​​from the first gesture trajectory data. Use a Gaussian kernel function to perform energy mapping on each pixel coordinate point to generate a continuous initial attention potential energy distribution. The calculation formula is as follows: ; in, This represents the initial attention potential energy value at any coordinate point in a continuous initial attention potential energy distribution. This represents the total number of keypoints within the current sampling time. This indicates the index number of the key point within the current sampling time. Indicates the first The confidence score of the identification of each key point Indicates the first The key point is the x-coordinate of the pixel coordinates in the coordinate space of the projected display canvas. Indicates the first The key point is the vertical coordinate of the pixel in the coordinate space of the projected display canvas. This represents the x-coordinate of any pixel in the coordinate space of the projected display canvas. This represents the ordinate of any pixel in the coordinate space of the projected display canvas. Represents the spatial diffusion operator; Step S22: Read the attention distribution data generated at the previous sampling time from the preset cache space as the historical attention distribution, introduce a time decay mechanism, and iteratively update the historical attention distribution to obtain the attention distribution data at the current time. The calculation formula is as follows: ; in, This represents the coordinates in the attention distribution data at the current moment. The potential energy value at that location. This represents the corresponding coordinate in the attention distribution data generated at the previous sampling time. The potential energy value at that location. Represents the natural constant. Indicates the time decay coefficient. This represents the time interval between the current sampling time and the previous sampling time. Indicates the current moment; Step S23: Store the attention distribution data at the current moment into a preset cache space, and perform a local maximum search on the attention distribution data at the current moment to extract the coordinates corresponding to the potential energy peak as the interaction focus.

[0029] Specifically, the operation of "reading attention distribution data generated at the previous sampling moment from a preset cache space" is the core memory mechanism that enables the entire attention evolution model to run. This "preset cache space" physically refers to a dedicated area in system memory for storing attention distribution data. Logically, it's a two-dimensional array, its size corresponding to the resolution of the projection canvas. Each array element stores the attention potential value of a point on the canvas in the previous calculation cycle. Its operation is a sophisticated loop: when calculating the attention distribution at the current moment T, the system quickly reads the stored data matrix representing the complete attention state at moment T-1 from this fixed area, using it as the historical basis. Then, the system multiplies this historical data by a time decay factor based on the natural constant e, and then adds the new potential distribution generated in real-time based on the gesture in the current frame. The resulting new result is immediately written back to the same cache area, overwriting the old data, thus preparing for the calculation of the next frame. The fundamental reason for designing such a cache mechanism with state memory is to combat the "instantaneous noise" problem that traditional interaction schemes cannot overcome. Traditional gesture focus determination mostly adopts a "stateless" model, which only analyzes the gesture coordinates of the current frame to determine where to point. This method treats each frame as an isolated event, and its fatal flaw is that single-frame data jumps caused by sensor noise, natural hand tremors, or light interference will directly and completely manifest as abrupt jittering or jumping of the cursor on the screen, severely damaging the smoothness of the interaction and the reliability of the pointing. This invention introduces this cache space and iterative update formula, which essentially endows the system with short-term memory and physical inertia. It makes "attention" no longer a series of discrete snapshots, but an energy field that evolves continuously on the timeline. Historical data is partially preserved through a decay factor and fused with the current signal. This means that even if the data of a certain frame deviates due to interference, the strong temporal continuity will smooth it out, giving the calculated interaction focus a stable trajectory and inertia. This not only eliminates visual flicker at the algorithmic level, but also elevates the interaction logic from "recognizing gesture positions" to "understanding attention trends," providing a stable and reliable temporal context for subsequent semantic absorption and logical verification. This is a crucial leap towards achieving a natural and realistic interactive experience.

[0030] In specific implementation, step S23 includes: Step S231: Traverse all pixel coordinates in the attention distribution data, filter out pixel coordinates with potential values ​​greater than a preset threshold, and construct an active interaction region coordinate set. Step S232: Within the coordinate set of the active interaction region, use a search algorithm to find the local potential energy maxima. The local potential energy maxima must satisfy the discriminant: ; in, To represent any one, Indicates The pre-defined neighborhood space centered on the center Represents any coordinate within the preset neighborhood space at the current time. The potential energy value at that location; Step S233: Based on the potential energy values ​​of each coordinate point within the active interaction area coordinate set, the centroid weight is calculated to obtain the pixel coordinates corresponding to the interaction focus. The calculation formula is as follows: ; in, This represents the x-coordinate of the pixel coordinates corresponding to the focus point. The y-coordinate represents the pixel coordinates corresponding to the focus of the interaction. This represents the coordinate set of the active interaction area.

[0031] Specifically, by traversing all pixel coordinates in the attention distribution data, the potential energy value of each pixel is compared with a preset threshold. This filtering process aims to identify regions with sufficient energy intensity from the continuous attention potential energy distribution, thereby constructing a set of active interaction region coordinates. Since the attention distribution data is generated by superimposing energy mappings of multiple keypoints using a Gaussian kernel function, the entire canvas often contains low-potential-energy backgrounds caused by environmental interference or non-target actions. Through filtering with the preset threshold, the system can eliminate these invalid weak signals, locking subsequent search calculations onto local regions with significant interaction characteristics, thus ensuring the initial accuracy of interaction intent recognition.

[0032] After constructing the coordinate set of active interaction regions, a search algorithm is used to find local potential energy maxima within this set. The predefined neighborhood space is a local computational window defined centered on the target pixel, such as a pixel matrix. In practice, the system uses a discriminant to verify whether the potential energy value of the center point is greater than the potential energy value of any coordinate point within its neighborhood space. The neighborhood space discriminant is introduced because the attention distribution generated by interaction mathematically presents a peak-like shape; only through this local numerical comparison can the peak vertices with the most concentrated energy be accurately located. This effectively solves the problem that when multiple knowledge entities are densely arranged, the system may struggle to determine the core focus due to energy overlap.

[0033] After identifying a local maximum point, the system does not directly use it as the final interaction focus. Instead, it calculates the centroid weight based on the potential energy values ​​of each coordinate point within the active interaction area's coordinate set. This process uses the centroid weight formula to multiply the pixel coordinates of all points within the area by their corresponding potential energy values ​​and then performs a weighted average to finally calculate the pixel coordinates corresponding to the interaction focus. The necessity of this calculation method lies in its ability to make the interaction focus behave more smoothly and physically inertially during the evolution over time. Compared to the pixel-level jitter that may occur when directly taking the maximum point, the centroid algorithm integrates the energy trend of the entire active area, which can smooth out the positioning deviation caused by slight hand tremors at the root of the algorithm, thus providing a stable and reliable coordinate benchmark for the semantic association retrieval with knowledge entities in the subsequent step S3.

[0034] This application significantly improves the stability and anti-interference capability of gesture control by introducing an active interaction region filtering and centroid weighting mechanism. Compared with traditional techniques that rely solely on single-frame coordinate determination, this application constructs an active interaction region coordinate set through a preset threshold, effectively filtering low-potential background noise from the algorithm's core and eliminating invalid signals caused by ambient light interference or sensor inaccuracies. Based on this, it uses a preset neighborhood space to search for local potential maxima, accurately pinpointing the peaks of energy distribution through discriminative methods, effectively solving the directional ambiguity problem when multiple knowledge entities are densely arranged. Finally, the system uses a centroid weighting formula to weight the coordinates within the region. Its core advantage lies in giving the interaction focus stronger physical inertia and temporal continuity, smoothing out coordinate jitter caused by hand tremors at the algorithm's root. This design ensures that the output interaction focus pixel coordinates are smooth and detailed, providing a stable and reliable data benchmark for high-precision gravitational radius adsorption in subsequent step S3, greatly enhancing the determinism and immersion of human-computer interaction.

[0035] In specific implementation, step S3 includes: Step S31: Based on the pixel coordinates corresponding to the interactive focus, retrieve the set of knowledge entities in the projection display canvas, and obtain the center pixel coordinates, type identifier, and preset gravitational radius corresponding to the type identifier of each knowledge entity; The knowledge entity set includes text objects, geometric figures, and interactive components; Type identifiers include text object identifiers, geometric shape identifiers, and interactive component identifiers; Step S32: Calculate the Euclidean distance between the pixel coordinates corresponding to the interactive focus and the center pixel coordinates of each knowledge entity, and determine whether the Euclidean distance satisfies the following conditions. ; in, Indicates the preset gravitational radius. Represents Euclidean distance; If the conditions are met, it is determined that there is a semantic relationship between the interaction focus and the corresponding knowledge entity, and the knowledge entity is identified as the target knowledge entity. Step S33: Based on semantic association, the pixel coordinates corresponding to the interaction focus are corrected using the adsorption vector formula to generate mapping relationship data. The calculation formula is as follows: ; in, This represents the pixel coordinates corresponding to the mapping relationship data. This indicates the pixel coordinates corresponding to the focus of the interaction. This represents the adsorption weight coefficient. Represents the center pixel coordinates of each knowledge entity.

[0036] Specifically, when text objects, geometric shapes, and interactive components are loaded into the system page, their center pixel coordinates, type identifiers, and corresponding preset gravitational radii are already stored in memory as structured data. The retrieval process essentially extracts the parameter information of all active entities from the backend database or the entity list of the current view based on the current pixel coordinate position.

[0037] The preset gravity radius refers to the logical sensing range defined for different types of knowledge entities, determining how close the interactive focus needs to be to trigger the automatic attraction effect. In practice, the system pre-sets differentiated values ​​for entities of varying importance or clickability. For example, interactive components (buttons) are typically given a larger gravity radius for easier triggering, while dense text objects are given a smaller radius to prevent accidental touches. The core reason for setting the gravity radius is to eliminate the "spatial drift" feeling in non-contact gesture interaction. Since it is difficult for users to achieve pixel-level precision when hovering gestures in the air, by setting the gravity radius, the system can actively "capture" the user's intent and pull slightly off-center coordinates back to the entity's center, thereby significantly reducing the difficulty of fine-grained operations.

[0038] When calculating the Euclidean distance, the system extracts the pixel coordinates corresponding to the interactive focus. Center pixel coordinates of the knowledge entity The calculation formula is as follows: The numerical value obtained through this formula It will immediately match the corresponding preset gravitational radius Perform a comparison; if the conditions are met... If the system determines that there is a semantic relationship between the two, it initiates a position correction procedure. Compared to traditional methods that rely solely on absolute position matching, this design achieves automatic alignment from "physical coordinates" to "logical intent." Through real-time Euclidean distance discrimination and vector snapping correction, the system can automatically compensate for minor deviations captured by the sensors, ensuring that even if the teacher's gesture does not completely cover the knowledge entity, the cursor or feedback on the large screen can accurately and stably snap to the target center. This not only solves the visual flicker problem during fine-grained operations but also utilizes differentiated gravity radius settings to make interactive feedback more consistent with teaching logic and subject rules, greatly improving the intelligence level and operational certainty of human-computer interaction.

[0039] In specific implementation, the processing logic for the adsorption weight coefficient includes: Get the potential energy value at the pixel coordinates corresponding to the interactive focus and the highest potential energy value in the entire projected display canvas. Based on the potential energy value, the maximum potential energy value, and the preset sensitivity adjustment factor, the adsorption weight coefficient is calculated using the following formula: ; in, Represents the pixel coordinates of the interactive focus. The potential energy value at that location. This indicates the highest potential energy value across the entire projected display canvas. This indicates the preset anti-lock threshold. This represents the preset sensitivity adjustment factor, with a value range of [value range missing]. , This indicates the minimum operator; Based on the type identifier in each knowledge entity, extract the value corresponding to the type identifier from the preset mapping table as the preset gravitational radius; The preset gravitational radius of the text object identifier is smaller than that of the preset gravitational radius of the interactive component identifier.

[0040] Specifically, during position correction processing, the system first extracts the potential energy value at the pixel coordinates of the interactive focus from the attention distribution data, and simultaneously retrieves the highest potential energy value across the entire projection display canvas. Then, the system introduces a preset sensitivity adjustment factor and calculates the adsorption weight coefficient based on the adsorption weight coefficient formula. The significance of this calculation logic is that by comparing the potential energy of the current focus with the global highest potential energy, the "strength" of the adsorption is dynamically adjusted. When the user's gesture intent is extremely strong (potential energy close to the peak), the weight coefficient approaches 1, achieving fast and accurate center adsorption; while when the gesture signal is weak or in a transitional phase, the weight decreases, thus ensuring the naturalness of the interaction and preventing accidental touches. The core reason for setting the sensitivity adjustment factor is to achieve a balance between the "determinism" and "smoothness" of the interaction. First, non-contact gesture interaction inherently exhibits signal fluctuations. Without a sensitivity adjustment factor, relying solely on the original potential energy ratio, the adsorption strength may fluctuate drastically with subtle tremors of the gesture, causing frequent jumps in the focus on the projected image. Second, different users have different interaction habits. By adjusting the sensitivity adjustment factor, the system can adapt to different "intent thresholds": in scenarios requiring rapid, wide-area clicks, the sensitivity adjustment factor is decreased to increase sensitivity, allowing the interactive focus to quickly "boom" and attach to the target; while in explanatory scenarios requiring precise selection or prolonged focus, the sensitivity adjustment factor is increased to raise the attachment threshold, preventing accidental attachment due to unintentional swipes. This design greatly enhances the robustness of human-computer interaction, ensuring that the system can accurately capture and calibrate the user's true operational intent even in complex teaching environments.

[0041] It is worth noting that when calculating the adsorption weight coefficient, an anti-lock threshold is introduced, with a value range of [0.7, 0.9], establishing a dynamic adsorption mechanism with geometric margin. The technical principle is that when the interaction focus moves towards the center of the knowledge entity, the adsorption intensity will not reach absolute saturation, thus maintaining a certain physical deviation between the corrected pixel coordinates and the geometric center of the knowledge entity. The core purpose of this design is to preserve the true motion features for the topology analysis in step S4. By preventing the coordinates from being "completely locked" at the center point, the system can ensure that when the user performs fine operations such as edge sliding and cutting, the trajectory sequence can still carry relative displacement information relative to the knowledge entity boundary, thereby avoiding the problem of topological feature loss due to sudden coordinate changes.

[0042] Based on the type identifier of each knowledge entity retrieved, the corresponding value is extracted from the preset mapping table as the preset gravitational radius; Table 1 is a preset mapping table; Table 1 ; In the above mapping relationship, the preset gravitational radius of the text object identifier is set to be smaller than the preset gravitational radius of the interactive component identifier. The necessity of this differentiated design is that text objects are usually arranged relatively closely on the teaching page, and a smaller gravitational radius can prevent frequent "jumps" or accidental selections when gestures switch between adjacent text; while interactive components (buttons) usually have clear functions and relatively independent positions, and setting a larger gravitational radius can expand their sensing area, allowing users to more easily "hit" the target in non-contact operation, greatly improving the convenience of operation.

[0043] Compared to traditional technologies that use fixed adsorption strength or uniform sensing radius, the advantages of this application are primarily reflected in the intelligent allocation of adsorption strength. Traditional solutions often result in a rigid adsorption feel, making it difficult to smoothly switch targets with gestures. This application, however, calculates the ratio of the real-time potential energy of the interactive focus to the highest potential energy across the entire area, and combines this with a sensitivity adjustment factor, allowing the adsorption weight coefficient to dynamically evolve according to the strength of the user's intention. When the user's pointing intention is clear, the system enhances the adsorption weight to achieve precise locking; when the gesture signal is weak or in a rapid movement state, the weight automatically decreases, ensuring the naturalness of the interaction and preventing accidental touches. This dynamic adjustment mechanism eliminates the "drift" of coordinates at the algorithmic level, significantly improving the stability of refined operations. Simultaneously, this application uses a preset mapping table to set differentiated gravitational radii for different knowledge entities such as text objects, geometric shapes, and interactive components. The brilliance of this design lies in its alignment with the practical logic of teaching scenarios: setting a smaller radius of attraction for densely packed text objects effectively filters out interference between adjacent characters, preventing frequent misselections; while setting a larger radius for functionally critical and relatively independently distributed interactive components (buttons) expands the sensing area, greatly enhancing the convenience of contactless operation. This differentiated processing based on physical characteristics, combined with a dynamic adsorption algorithm, constitutes the core technical guarantee for ensuring the deterministic nature and visual consistency of the projected display interaction.

[0044] In specific implementation, step S4 includes: Step S41: Extract the pixel coordinates of each group in the mapping relationship data according to the timestamp order, construct the dynamic behavior trajectory sequence associated with each knowledge entity, and process the dynamic behavior trajectory sequence using the moving average filtering algorithm to obtain the updated mapping relationship data. Step S42: Calculate the topological association data between the dynamic behavior trajectory sequence and the geometric boundary of the corresponding knowledge entity; Topological association data includes containment, intersection, and tangency relationships; Step S43: Based on the type identifier of each knowledge entity, retrieve matching verification rules from the preset subject logic rule base, and input the topological association data into the verification rules for comparison; If the topological association data meets the validation rules, then add a first logical attribute label to the mapping relationship data; If the topological association data does not meet the validation rules, a second logical attribute label is added to the mapping relationship data.

[0045] Specifically, in step S4, the system resolves the "logical deadlock" problem that might occur during "semantic adsorption" in step S3 through temporal reconstruction. Since step S3 has already corrected the centripetal adsorption of the interaction focus, the resulting mapping relationship data, while spatially more aligned with the knowledge entity center, may cause non-physical instantaneous jumps in the originally continuous trajectory at geometric boundaries, thus interfering with topological parsing. Therefore, the execution of step S42 (calculating topological association data) must be based on the updated mapping relationship data output from step S414 (filtering update). By performing a moving average filter on the adsorbed trajectory sequence before performing geometric topological determinations of inclusion, intersection, or tangency, the system evenly distributes the adsorption displacement generated in S3 across the time series. The technical advantage of this processing logic is that it preserves the "intent adsorption" effect of S3 while reconstructing a smooth trajectory curve that conforms to physical inertia, ensuring that step S42 can accurately identify the minute displacement features of gestures at the edges of knowledge entities, thereby avoiding the failure of subject logic determination due to instantaneous coordinate shifts.

[0046] The pre-defined subject-specific logic rule base is a database storing specific teaching logic and interaction judgment criteria, specifying which topological actions different types of knowledge entities are considered "correct" or "valid" when performing them. For example, in mathematics and geometry, the rule base might stipulate that "when a gesture trajectory satisfies the inclusion relationship with the center point of a circle and the intersection relationship with the circumference, it is judged as a radius measurement action"; in chemistry, the rule base might stipulate that "when two entities representing chemical reagents are in a tangential relationship, the reaction logic is triggered." The necessity of establishing this rule base lies in elevating the physical-level "coordinate movement" to the subject-level "intent understanding." Traditional gesture control can only achieve simple clicks or drags, and cannot determine whether the user's operation conforms to the teaching logic. Through the pre-defined subject-specific logic rule base, the system can use topological association data (inclusion, intersection, tangency) as input, automatically compare whether the user's trajectory conforms to the subject standards, and then provide a decision basis for the subsequent step S5 to output graded teaching feedback by adding first (conforming) or second (non-conforming) logical attribute labels, such as animated rewards for correct actions or guidance and correction for incorrect actions. This design transforms projection display devices from simple display terminals into interactive auxiliary tools with logical evaluation capabilities, greatly enhancing the rigor and intelligence of teaching interaction.

[0047] By transforming physical coordinates into topological relationships and verifying them using subject-specific logical rules, this application achieves accurate extraction and discrimination of complex user interaction intentions. Compared to the limitations of traditional technologies that can only identify single-location triggers, this application ensures that interactive feedback not only conforms to visual intuition but also to the inherent logic of education and teaching through dynamic trajectory modeling and logical label classification. This allows for customized interactive feedback to be output for different teaching content, greatly improving teaching efficiency in smart education scenarios.

[0048] In specific implementation, step S41 includes: Step S411: Within a preset sliding time window, extract the pixel coordinates from the mapping relationship data corresponding to each sampling time. Step S412: Based on semantic association, pixel coordinates of objects with the same association are grouped into the same data cluster; Step S413: For each data cluster, the pixel coordinates of objects with the same association are linearly arranged according to the order of timestamps to generate a dynamic behavior trajectory sequence that corresponds one-to-one with the target knowledge entity. Step S414: The moving average filtering algorithm is used to calculate the mean of the coordinate components in the dynamic behavior trajectory sequence, and the pixel coordinate values ​​of each coordinate point in the dynamic behavior trajectory sequence are updated to obtain the updated mapping relationship data.

[0049] Specifically, in step S414, the moving average filtering algorithm is used not only to eliminate high-frequency noise at the hardware level, but also as an "attraction buffer." Within a preset sliding time window, such as 500 milliseconds, the system calculates the weighted average of the pixel coordinates of objects with the same association (i.e., target entities already attracted by S3). This processing method transforms the static pixels after attraction correction back into a dynamic behavioral trajectory sequence. By updating the pixel coordinate values ​​of each coordinate point in the dynamic behavioral trajectory sequence, the system completes the motion trend of each "attraction point" within the sliding window. This step ensures that the mapping relationship data not only contains the static intention of "where to attract," but also retains the dynamic process of "how to move." This data input with "motion inertia" provides a high-confidence logical benchmark for step S43 to retrieve and match verification rules from the preset subject rule base, ensuring that the interactive feedback is both accurate and in line with physical intuition.

[0050] The preset sliding time window length is set within the range of 200 to 800 milliseconds, with 500 milliseconds being the preferred value. Converted to a common sampling frequency of 30 fps, this corresponds to approximately 15 frames of data samples. The fundamental reason for this setting is to resolve the inherent contradiction between real-time performance and smoothness in contactless interaction through a precise match between physical characteristics and algorithm performance. First, setting the window within this range ensures the immediacy of the operation while filtering out noise. If the window is shorter than 200 milliseconds, too few data points are used in the sliding average calculation, which cannot effectively offset the random white noise generated by the sensor and the physiological tremors of the hand, resulting in visible coordinate jumps in the projected image. Conversely, if the window is longer than 800 milliseconds, although the trajectory will be extremely smooth, it will lead to severe "interaction lag," causing a noticeable trailing effect between the teacher's gestures and the screen feedback. Secondly, a duration of approximately 500 milliseconds can fully cover the physical execution cycle of a typical teaching interaction action (such as pointing, drawing a short line, or clicking). This ensures that the system's "recent memory" at any sampling moment contains a motion vector with directionality and shape trends, providing coherent and semantically meaningful data input for subsequent topological relationship determination, and avoiding misjudging continuous gestures as isolated noise due to insufficient data span. Finally, this setting ensures that the generated dynamic behavior trajectory sequence not only conforms to the teacher's physical operation intuition but also provides a stable and high-quality decision benchmark for verifying the subject's logical rule base, thereby achieving a "precise and fast" control experience in complex classroom interaction environments.

[0051] In specific implementation, step S5 includes: Step S51: Obtain the updated mapping relationship data, extract the logical attribute tags and type identifiers corresponding to each knowledge entity carried by the updated mapping relationship data, and extract the corresponding graphical interface instructions from the preset interactive feedback database. Logical attribute tags include first logical attribute tags and second logical attribute tags; Step S52: Modify the visual parameters of the corresponding knowledge entity in the projection display canvas according to the graphical interface instructions. Visual parameters include color values, transparency values, and line width values; Step S53: If the logical attribute tag is the first logical attribute tag, then the preset multimedia file associated with the knowledge entity is triggered. If the logical attribute label is the second logical attribute label, then the preset guide label will be output in the projection display canvas according to the graphical interface instructions; Step S54: Refresh the layer buffer and output the modified visual parameters and the interactive content corresponding to the logical attribute labels in the projection display canvas. The interactive content includes preset multimedia files corresponding to the first logical attribute tag and preset guidance identifier data corresponding to the second logical attribute tag.

[0052] Specifically, the pre-defined interactive feedback database is a structured rule base that stores the correspondence between "logical attributes - knowledge entity types - graphical interface commands." It exists in the system backend as an XML configuration file, associating logical attribute tags with specific interface operation commands, such as highlighting, color changing, or vibration prompts. The reason for establishing this database is to achieve standardization and decoupling of feedback, enabling the system to quickly and automatically retrieve the most suitable interaction method for the current entity type based on different subject-specific logical verification results (first or second tag), thereby ensuring that the system provides intuitive responses when dealing with text, geometric shapes, or components.

[0053] The specific operation of modifying visual parameters involves the system dynamically adjusting the rendering attributes of the target knowledge entity through the Canvas engine after receiving instructions from the graphical interface. The system modifies the color value of the corresponding entity in memory (changing the stroke from black to green), the transparency value (adjusting the alpha channel of the selected target to 1.0 for emphasis), or the line width value (thickening the geometric lines being manipulated). The significance of this process is to provide users with immediate visual confirmation, using strong visual contrast to inform them which entities have been successfully locked or identified by the system, thereby eliminating the uncertainty of non-contact operations.

[0054] Pre-set multimedia files refer to digital teaching resources deeply associated with knowledge entities, including but not limited to correct prompts, analytical videos, 3D model animations, or subject experiment simulation programs. This is achieved by reserving resource link addresses (URIs) in the metadata of each entity. When the system determines that the logical attribute is the "first logical attribute tag" (correct operation or logical match), it will automatically load and play these files. The purpose of this is to enhance teaching effectiveness through multi-dimensional information output, transforming tedious gesture operations into vivid teaching feedback, and turning the projection device from a simple display into an interactive terminal with intelligent and inspiring capabilities.

[0055] Pre-defined guidance indicators are auxiliary visual cues output by the system when it determines that the operation is a "second logical attribute label" (operation logic mismatch). These cues may include a flashing red dashed box, an arrow pointing in the correct direction, or floating text descriptions. Based on the current topological association data, the generation position of the guidance indicator is calculated in the graphics layer buffer and then rendered. The reason for setting up guidance indicators is to provide non-intrusive "corrective guidance" when user operations deviate from expectations or subject rules, preventing teachers from interrupting demonstrations due to operational errors during teaching, and also providing students with clear error correction examples, significantly improving the system's usability and fault tolerance.

[0056] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0057] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. An interactive educational projection display method based on gesture control, characterized in that, Includes the following steps: Step S1: Obtain the original gesture trajectory data generated by the interactive subject in a continuous time series, perform spatial mapping processing on the original gesture trajectory data to obtain the first gesture trajectory data, and convert the first gesture trajectory data to the pixel coordinate system corresponding to the projection display canvas. Step S2 involves continuously distributing the first gesture trajectory data to obtain attention distribution data, and then performing a time evolution update on the attention distribution data to extract the interaction focus. Specifically, this includes: Step S21: Extract the pixel coordinates and corresponding recognition confidence values ​​from the first gesture trajectory data. Use a Gaussian kernel function to perform energy mapping on each pixel coordinate point to generate a continuous initial attention potential energy distribution. The calculation formula is as follows: ； in, This represents the initial attention potential energy value at any coordinate point in a continuous initial attention potential energy distribution. This represents the total number of keypoints within the current sampling time. This indicates the index number of the key point within the current sampling time. Indicates the first The confidence score of the identification of each key point Indicates the first The key point is the x-coordinate of the pixel coordinates in the coordinate space of the projected display canvas. Indicates the first The key point is the vertical coordinate of the pixel in the coordinate space of the projected display canvas. This represents the x-coordinate of any pixel in the coordinate space of the projected display canvas. This represents the ordinate of any pixel in the coordinate space of the projected display canvas. Represents the spatial diffusion operator; Step S22: Read the attention distribution data generated at the previous sampling time from the preset cache space as the historical attention distribution, introduce a time decay mechanism, and iteratively update the historical attention distribution to obtain the attention distribution data at the current time. The calculation formula is as follows: ； in, This represents the coordinates in the attention distribution data at the current moment. The potential energy value at that location. This represents the corresponding coordinate in the attention distribution data generated at the previous sampling time. The potential energy value at that point represents the natural constant. Indicates the time decay coefficient. This represents the time interval between the current sampling time and the previous sampling time. Indicates the current time; Step S23: Store the attention distribution data at the current moment into a preset cache space, and perform a local maximum search on the attention distribution data at the current moment to extract the coordinates corresponding to the potential energy peak as the interaction focus. Specifically, this includes: Step S231: Traverse all pixel coordinates in the attention distribution data, filter out pixel coordinates with potential energy values ​​greater than a preset threshold, and construct an active interaction region coordinate set. Step S232: Within the coordinate set of the active interaction region, use a search algorithm to find the local potential energy maxima. The local potential energy maxima must satisfy the discriminant: ； in, To represent any one, Indicates The pre-defined neighborhood space centered on, Represents any coordinate within the preset neighborhood space at the current time. The potential energy value at that location; Step S233: Based on the potential energy values ​​of each coordinate point within the active interaction area coordinate set, the centroid weight is calculated to obtain the pixel coordinates corresponding to the interaction focus. The calculation formula is as follows: ， ； in, This represents the x-coordinate of the pixel coordinates corresponding to the focus point. The y-coordinate represents the pixel coordinates corresponding to the focus of the interaction. Represents the coordinate set of the active interaction area; Step S3: Based on the attention distribution data, determine the set of knowledge entities corresponding to the interaction focus, and correct the spatial relationship between the interaction focus and the set of knowledge entities to obtain mapping relationship data, specifically including: Step S31: Based on the pixel coordinates corresponding to the interactive focus, retrieve the set of knowledge entities in the projection display canvas, and obtain the center pixel coordinates, type identifier, and preset gravitational radius corresponding to the type identifier of each knowledge entity; The knowledge entity set includes text objects, geometric figures, and interactive components; The type identifiers include text object identifiers, geometric shape identifiers, and interactive component identifiers; Step S32: Calculate the Euclidean distance between the pixel coordinates corresponding to the interactive focus and the center pixel coordinates of each knowledge entity, and determine whether the Euclidean distance satisfies the following conditions. ; in, Indicates the preset gravitational radius. Represents Euclidean distance; If the conditions are met, it is determined that there is a semantic relationship between the interaction focus and the corresponding knowledge entity, and the knowledge entity is identified as the target knowledge entity. Step S33: Based on semantic association, the pixel coordinates corresponding to the interaction focus are corrected using the adsorption vector formula to generate mapping relationship data. The calculation formula is as follows: ； in, This represents the pixel coordinates corresponding to the mapping relationship data. This indicates the pixel coordinates corresponding to the focus of the interaction. This represents the adsorption weight coefficient. Represents the center pixel coordinates of each knowledge entity; The processing logic for the adsorption weight coefficient includes: Get the potential energy value at the pixel coordinates corresponding to the interactive focus and the highest potential energy value in the entire projected display canvas. Based on the potential energy value, the maximum potential energy value, and the preset sensitivity adjustment factor, the adsorption weight coefficient is calculated using the following formula: ； in, Represents the pixel coordinates of the interactive focus. The potential energy value at that location. This indicates the highest potential energy value across the entire projected display canvas. This indicates the preset anti-lock threshold. This represents the preset sensitivity adjustment factor, with a value range of [value range missing]. , This indicates the minimum operator; Based on the type identifier in each knowledge entity, extract the value corresponding to the type identifier from the preset mapping table as the preset gravitational radius; Among them, the preset gravitational radius of the text object identifier is smaller than the preset gravitational radius of the interactive component identifier; Step S4: Perform topological relationship parsing on the set of knowledge entities at the interaction focus in the mapping relationship data, verify the topological association data according to the preset subject logic rules, add corresponding logical attribute labels to the mapping relationship data, update the pixel coordinate values ​​in the mapping relationship data using a filtering algorithm, and output the updated mapping relationship data. Step S5: Obtain the updated mapping relationship data, extract graphical interface instructions based on logical attribute labels and knowledge entity type identifiers, adjust the visual parameters of the corresponding knowledge entities in the projection display canvas, and synchronously output the interactive content corresponding to the logical attribute labels.

2. The interactive educational projection display method based on gesture control as described in claim 1, characterized in that, Step S1 specifically includes: Step S11: Capture key point location information of the interactive subject in a continuous time series using a visual sensor to generate raw gesture trajectory data; The interactive subject includes a human hand, a human torso, and a guiding medium carried by the human body to perform teaching demonstration actions; The guidance medium carried by the human body includes a pointer with a fixed outline, a marker with reflective properties, and a laser pointer that emits light signals. The key point location information includes the physical coordinates of the interactive subject in the three-dimensional sensing space and the recognition confidence value; The original gesture trajectory data includes spatial coordinates, timestamps corresponding to the spatial coordinates, and key point type identifiers corresponding to the spatial coordinates. The key point type identifiers corresponding to the spatial coordinates include semantic tags corresponding to the fingertips, the center of the palm, and the joint points in the human hand; Step S12: Establish the spatial mapping relationship between the projection display canvas and the three-dimensional sensing space. Use the homography transformation matrix to project the original gesture trajectory data onto the two-dimensional coordinate space corresponding to the projection display canvas to obtain the projection trajectory coordinates. Step S13: Normalize the coordinates of the projection trajectory and match them to the pixel coordinate system corresponding to the projection display canvas to obtain the first gesture trajectory data.

3. The interactive educational projection display method based on gesture control as described in claim 2, characterized in that, Step S12 specifically includes: Step S121: Obtain the intrinsic and extrinsic parameter matrices of the visual sensor, and establish the perspective projection relationship from the three-dimensional sensing space to the two-dimensional camera coordinate system based on the intrinsic and extrinsic parameter matrices. Step S122: By sampling the coordinates of the four corner points of the projection display canvas, the homography transformation matrix between the two-dimensional camera coordinate system and the coordinate space of the projection display canvas is calculated using the homography transformation algorithm. Step S123: Use the homography transformation matrix to perform a linear transformation on the physical coordinates in the original gesture trajectory data to obtain the mapped coordinates of the original gesture trajectory data in the two-dimensional coordinate space.

4. The interactive educational projection display method based on gesture control as described in claim 3, characterized in that, Step S4 specifically includes: Step S41: Extract the pixel coordinates of each group in the mapping relationship data according to the timestamp order, construct the dynamic behavior trajectory sequence associated with each knowledge entity, and process the dynamic behavior trajectory sequence using the moving average filtering algorithm to obtain the updated mapping relationship data. Step S42: Calculate the topological association data between the dynamic behavior trajectory sequence and the geometric boundary of the corresponding knowledge entity; The topological association data includes containment relationships, intersection relationships, and tangency relationships; Step S43: Based on the type identifier of each knowledge entity, retrieve matching verification rules from the preset subject logic rule base, and input the topological association data into the verification rules for comparison; If the topological association data meets the validation rules, then add a first logical attribute label to the mapping relationship data; If the topological association data does not meet the validation rules, a second logical attribute label is added to the mapping relationship data.

5. The interactive educational projection display method based on gesture control as described in claim 4, characterized in that, Step S41 specifically includes: Step S411: Within a preset sliding time window, extract the pixel coordinates from the mapping relationship data corresponding to each sampling time. Step S412: Based on semantic association, pixel coordinates of objects with the same association are grouped into the same data cluster; Step S413: For each data cluster, the pixel coordinates of the same associated objects are linearly arranged according to the order of timestamps to generate a dynamic behavior trajectory sequence that corresponds one-to-one with the target knowledge entity. Step S414: The moving average filtering algorithm is used to calculate the mean of the coordinate components in the dynamic behavior trajectory sequence, and the pixel coordinate values ​​of each coordinate point in the dynamic behavior trajectory sequence are updated to obtain the updated mapping relationship data.

6. The interactive educational projection display method based on gesture control as described in claim 5, characterized in that, Step S5 specifically includes: Step S51: Obtain the updated mapping relationship data, extract the logical attribute tags and type identifiers corresponding to each knowledge entity carried by the updated mapping relationship data, and extract the corresponding graphical interface instructions from the preset interactive feedback database. The logical attribute tags include a first logical attribute tag and a second logical attribute tag; Step S52: Modify the visual parameters of the corresponding knowledge entity in the projection display canvas according to the graphical interface instructions. The visual parameters include color values, transparency values, and line width values; Step S53: If the logical attribute tag is the first logical attribute tag, then the preset multimedia file associated with the knowledge entity is triggered. If the logical attribute label is the second logical attribute label, then the preset guide label will be output in the projection display canvas according to the graphical interface instructions; Step S54: Refresh the layer buffer and output the modified visual parameters and the interactive content corresponding to the logical attribute labels in the projection display canvas. The interactive content includes a preset multimedia file corresponding to the first logical attribute tag and preset guidance identifier data corresponding to the second logical attribute tag.

Images