Fingertip extraction touch method and system based on convex hull algorithm

Abstract

The invention relates to the technical field of fingertip touch, and discloses a fingertip extraction touch method and system based on a convex hull algorithm, and the method comprises the steps: collecting an original hand image, and extracting a hand layered sparse point set; performing feature space mapping on the hand layered sparse point set to obtain a hand feature enhanced point set; generating a hand contour convex hull according to the hand feature enhancement point set; performing point angle analysis on the hand contour convex hull to obtain a hand fingertip candidate point set; performing inter-finger relation constraint analysis on the hand fingertip candidate point set to obtain a hand fingertip target point set; and executing critical angular velocity touch control judgment on the hand fingertip target point set to generate a touch control trigger signal. According to the method, the integrity and continuity of a convex hull result are ensured, the problem of incoherence caused by region segmentation is avoided, and the stability in a rapid gesture conversion process is enhanced.

Description

Technical Field

[0001] This invention relates to the field of fingertip touch technology, and in particular to a fingertip touch extraction method and system based on convex hull algorithm. Background Technology

[0002] Traditional fingertip detection methods primarily rely on contour extraction and geometric feature analysis after hand segmentation. Common techniques include convex point detection based on curvature analysis, template matching based on shape features, and end-to-end recognition based on machine learning. However, these methods have several shortcomings in practical applications: curvature analysis-based methods are sensitive to noise and prone to generating false detections in complex backgrounds; template matching-based methods struggle to adapt to multi-angle and deformable situations; and while machine learning-based methods offer better robustness, their high computational complexity makes it difficult to meet the low-latency requirements of real-time interaction. Summary of the Invention

[0003] This invention provides a fingertip extraction touch method and system based on convex hull algorithm. This invention ensures the integrity and continuity of convex hull results, avoids the inconsistency problem caused by region segmentation, and enhances stability in the process of rapid gesture transition.

[0004] In a first aspect, the present invention provides a fingertip extraction touch method based on a convex hull algorithm, the fingertip extraction touch method based on a convex hull algorithm comprising: Acquire raw images of the hand and extract layered sparse point sets of the hand; The hand segment sparse point set is mapped in the feature space to obtain the hand feature enhancement point set; A convex hull of the hand contour is generated based on the hand feature enhancement point set; A point angle analysis is performed on the convex hull of the hand contour to obtain a set of candidate points for the fingertips; The fingertip candidate point set is subjected to interfinite relationship constraint analysis to obtain the fingertip target point set; A critical angular velocity touch judgment is performed on the target point set of the hand fingertips to generate a touch trigger signal.

[0005] Secondly, the present invention provides a fingertip extraction touch system based on a convex hull algorithm, the fingertip extraction touch system based on the convex hull algorithm comprising: The layered sparse sampling module is used to acquire the original image of the hand and extract the layered sparse point set of the hand. The feature space mapping module is used to perform feature space mapping on the layered sparse point set of the hand to obtain the hand feature enhancement point set; A semi-periodic convex hull iteration module is used to generate a hand contour convex hull based on the hand feature enhancement point set; The point angle analysis module is used to perform point angle analysis on the convex hull of the hand contour to obtain a set of candidate points for the fingertips. The finger-interval relationship constraint analysis module is used to perform finger-interval relationship constraint analysis on the candidate fingertip point set of the hand to obtain the target fingertip point set of the hand; The touch detection module is used to perform critical angular velocity touch detection on the target point set of the hand fingertips and generate a touch trigger signal.

[0006] The technical solution provided by this invention employs a hierarchical sparse sampling strategy to perform non-uniform downsampling of the hand contour. This retains high-density feature points in the fingertip region while reducing redundant points in non-critical areas, significantly reducing the computational load of subsequent processing. By identifying changing and static regions, convex hull calculation is performed only on regions within the field of view that are predicted to change, while historical results are directly reused for static regions, greatly reducing the computational overhead of static regions and improving system response speed. By constructing a fingertip relationship network represented by vertices, edges, and weights, and using the graph Laplacian operator to quantify the physical correlation between fingertips, false detection points that violate hand anatomy are automatically identified and filtered, improving detection accuracy. Combining a fingertip deceleration function and an adaptive threshold, the dynamic features of the user's touch intent are accurately captured, effectively distinguishing different operation intents such as hovering, swiping, and clicking, significantly reducing the system's false touch rate. By mapping the contour point set to a sparse feature space through core equations, the feature representation of the fingertip region is strengthened, improving the recognition accuracy of fingertip candidate points and enhancing the system's ability to handle complex gestures. By periodically analyzing touch success rate and false touch rate, and dynamically updating angular velocity and deceleration threshold parameters, the system can adapt to different user operating habits and environmental conditions, improving overall system robustness. Seamless connection is achieved between real-time convex hulls of changing regions and historical convex hulls of static regions, ensuring the integrity and continuity of the final convex hull result, avoiding inconsistencies caused by region segmentation, and enhancing system stability during rapid gesture transitions. Attached Figure Description

[0007] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 This is a schematic diagram illustrating the steps of the fingertip touch extraction method based on the convex hull algorithm in an embodiment of the present invention; Figure 2 This is a schematic diagram of the fingertip extraction touch system based on the convex hull algorithm in an embodiment of the present invention. Detailed Implementation

[0009] This invention provides a fingertip extraction touch method and system based on a convex hull algorithm. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0010] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the fingertip extraction touch method based on the convex hull algorithm in this invention includes: Step S1: Acquire the original image of the hand and extract the layered sparse point set of the hand; It is understood that the executing entity of this invention can be a fingertip extraction touch system based on the convex hull algorithm, or it can be a terminal or a server; the specific implementation is not limited here. This embodiment of the invention will be described using a server as an example.

[0011] Specifically, raw image data containing the complete hand is acquired using an image sensor, and depth image segmentation is performed on the image. Depth information is used to effectively separate the foreground hand region from the background region, extracting a continuous closed initial hand contour point set, denoted by , where each point corresponds to an edge feature point in the image. A structured importance assessment is performed on the initial contour point set. Based on the spatial distribution and geometric characteristics of the points in the hand contour, the entire hand region is divided into multiple regions of different importance, and a hierarchical region model of the hand contour is established, divided into the fingertip potential region, the finger trunk region, and the palm interior region, each assigned different processing priorities and geometric importance. Different sampling rates are set for each region according to this hierarchical model. A density control function is constructed to achieve a progressively decreasing distribution of the sampling rate, ensuring that high-importance regions, such as the fingertip region, receive the densest sampling points, while low-importance regions, such as the palm interior region, receive the sparsest point set, forming a hierarchical sampling parameter set. A sampling density distribution model is established to perform local curvature analysis on each point in the initial hand contour point set. The curvature value reflects the degree of edge curvature at that point; the greater the curvature, the more drastic the change at that point, and the more likely it is to be an important structure such as the fingertip or joint. All curvature values ​​are summarized to generate a hand contour curvature distribution map. Combining the hierarchical sampling parameter set and the curvature distribution map, a non-uniform downsampling operation is performed on the initial point set. More points are preferentially retained in areas with greater curvature, while redundant points are removed in areas with less curvature. This generates a hierarchical sparse point set of the hand that retains structural features while significantly compressing the data volume.

[0012] Step S2: Perform feature space mapping on the sparse point set of the hand layer to obtain the hand feature enhancement point set; Specifically, based on a layered sparse point set of the hand, local structural analysis is performed on each point. The point density of each point in its neighborhood is evaluated by statistically analyzing the number and compactness of neighboring points within a certain range, calculating a local density weight that reflects the geometric concentration of the point within the current hand structure. Regions with higher density correspond to areas of concentrated structural boundaries or contour changes, especially points near the fingertips, which are more likely to have high-density features. Building upon the density analysis, a regional importance response is calculated for each point. This involves setting a kernel function centered on the point to measure its spatial influence range and intensity on surrounding points, automatically adjusting the influence range based on the point's contour level. This gives points in the fingertip region stronger spatial perception capabilities, while points in the palm region exhibit a more relaxed coverage range, thus constructing a feature representation that better conforms to the human hand structure during the mapping stage. A feature enhancement factor is introduced as another strengthening mechanism, evaluating the degree of geometric shape change of each point, i.e., the curvature of the point's location, and assigning it a structural enhancement weight based on the distance relationship between the point and the hand boundary. Points located near the edges and with greater curvature are amplified in spatial representation, while those in the center of the palm or in areas with slow structural changes are weakened. A point set projection model is constructed by integrating three core attributes—density weight, region importance response, and feature enhancement factor—into a feature mapping mechanism. This model projects the sparse, layered point set of the hand onto a high-dimensional feature space. In this space, points with high structural information are concentrated in local extrema, while ordinary points are distributed in lower response regions. A threshold is set to extract salient feature regions, and points meeting the criteria are grouped into the hand feature enhancement point set.

[0013] Step S3: Generate the convex hull of the hand contour based on the enhanced point set of hand features; Specifically, a dynamic frame buffer with historical data storage function is constructed internally to continuously store hand feature enhancement point sets at multiple time points. The buffer is set to a fixed frame length t to retain sufficient spatial change information for subsequent judgment within each time period. When a new hand image frame arrives, the system stores its corresponding feature enhancement point set in the buffer and updates the queue content, thus maintaining a point set sequence with a constant length but continuously updated content. After completing continuous frame storage, a preset half-cycle iteration strategy is used to determine whether to perform convex hull calculation for the current frame. The core mechanism of this strategy is to limit the calculation time to two trigger scenarios: first, the current frame is an odd-numbered frame; second, the rate of change between the feature enhancement point set of the current frame and the previous frame exceeds a certain preset threshold ε. Once either of the above conditions is met, a calculation trigger signal is generated, indicating that the current frame has sufficient geometric changes or temporal triggering reasons to warrant a new round of convex hull construction. After receiving the calculation trigger signal, the system performs differential analysis on the feature enhancement point set of the current frame and divides it into two subsets: the changed subset that needs to be recalculated and the static subset that retains historical calculation results. The changing subset contains newly added, moved, or locally deformed points. The contour structure of the regions containing these points exhibits a changing trend, thus requiring real-time convex hull updates. For the changing subset, the classic Graham scan method is used to construct the convex hull. This process involves polar angle sorting starting from a reference point, followed by stack operations combined with geometric calculations to determine whether the current point should remain on the boundary path, thereby progressively constructing the real-time convex hull of the changing region. Simultaneously, to avoid redundant calculations of the boundary structure of the static region, no new calculations are performed on the static subset; instead, the convex hull results obtained in the previous calculation cycle are directly reused. Topological boundary stitching is performed on the real-time convex hull of the changing region and the historical convex hull of the static region. By checking for gaps, overlaps, or topological breaks at the junction of the two convex hull segments, strategies such as boundary sequence connection and angle consistency repair are used to stitch the boundary paths of the two parts into a complete, continuous, and unambiguous hand contour convex hull.

[0014] Step S4: Perform point angle analysis on the convex hull of the hand contour to obtain the candidate point set of the fingertips; Specifically, all boundary points constituting the convex hull of the hand contour are traversed sequentially, and the corresponding interior angle value is calculated for each point. The interior angle of each convex hull point is determined by the angle formed between that point and its two adjacent points, and its geometric meaning is the degree of inflection of the current point in the entire contour curve. By calculating the interior angle of each point and comparing it with a preset angle threshold, all points with interior angles smaller than the threshold are marked as candidate points with sharp turning characteristics, i.e., initial fingertip feature points, forming an initial fingertip candidate point set. These points correspond to the fingertip, joint, or protruding edge structure in the contour geometry and are important geometric landmarks of the fingertip. Local geometric structure analysis is performed on each initial fingertip feature point in the initial fingertip candidate point set. By fitting each initial candidate point and several neighboring convex hull points before and after it, a continuous boundary arc is formed to evaluate the curvature characteristics at that point. By fitting this boundary arc and determining its minimum radius of curvature, the curvature characteristics of the point are quantified. The greater the curvature, the more tightly bent the arc, and the more likely it is to represent the true fingertip position. Curvature feature analysis can effectively help determine whether a sharp angle is merely a random inflection point or a fingertip with biological structural significance. Corner response value analysis is performed on each initial candidate point. A fixed-size local region window is selected centered on this point, and a structural tensor is constructed to evaluate the directionality and stability of grayscale changes in that region. By analyzing the feature distribution of this structural tensor, it is determined whether the point possesses obvious corner attributes, i.e., whether it is a feature point with drastic grayscale changes and prominent directionality in the image. A comprehensive scoring model is constructed for all initial candidate points based on the obtained curvature features and corner response values. This model forms a score reflecting the probability of the point being a fingertip by weighted fusion of curvature features and corner response values. A higher score indicates that the point possesses both structural sharpness and feature saliency in the image. After ranking the scores of all candidate points, the points with the highest scores are selected as the high-confidence fingertip candidate point set for the hand.

[0015] Step S5: Perform interfinite relationship constraint analysis on the candidate point set of the hand fingertips to obtain the target point set of the hand fingertips; Specifically, all points in the initial candidate fingertip points are used as vertices in the graph structure to construct an initial topological graph reflecting the spatial relationships between fingertips. During this process, based on a preset distance threshold range, the geometric distance between point pairs is used as the connection condition to establish two types of edges: when the distance between two points is between the first and second distance thresholds, a first-type edge is established, indicating a clear proximity relationship between fingers; when the distance between two points is between the third and fourth distance thresholds, a second-type edge is established, representing a possible indirect structural correlation. This hierarchical edge construction method ensures a refined representation of the finger structure and constitutes the initial finger relationship topological graph. Based on this initial graph structure, the multidimensional feature contribution values ​​between each point are calculated, specifically including Euclidean distance contribution values, angular constraint contribution values, and neighborhood similarity contribution values. Euclidean distance reflects the geometric compactness between points in space, angular constraints describe the consistency between the direction of the line connecting point pairs and the direction of the reference finger arrangement, and neighborhood similarity assesses whether adjacent points belong to the same finger anatomical structure by comparing the distribution characteristics of local regions. These three features are integrated into a multidimensional feature-weighted interfinite relationship graph. The multidimensional feature-weighted interfinite relationship graph undergoes matrix processing to construct the graph's degree matrix, and then the graph Laplacian operator is calculated. Stable eigenvectors are extracted through orthogonal decomposition, forming principal eigenvector groups and secondary eigenvector groups. Principal eigenvectors represent the most structurally significant connection relationships, corresponding to the patterns with the strongest interfinite coherence, while secondary eigenvectors reveal weaker but discriminative connection trends. Based on this, a first-class coherence score is calculated for each candidate point based on the principal eigenvector group, reflecting its consistency across a wide range of finger structures. Simultaneously, a second-class coherence score is calculated based on the secondary eigenvector group, quantifying its integration into local or peripheral structures. By fusing these two types of coherence scores, a hierarchical fingertip coherence score set is formed, providing an overall evaluation of the organizational consistency of all candidate points in the spatial topology. After structural scoring, a multidimensional anatomical constraint factor is introduced to improve the physiological rationality of the recognition. Specifically, biological structural standard parameters such as the finger distance ratio coefficient, effective range of the included angle, relative position offset tolerance, and arrangement consistency coefficient are set. The distribution patterns of candidate points are compared and analyzed with the conventional anatomical features of the human hand. An anatomical constraint score is calculated for each candidate point, forming a multi-dimensional constraint score set to reflect whether the point conforms to the natural characteristics of real finger arrangement, position, and orientation. These score data are used as evaluation indicators, comprehensively considering coherence scores and anatomical matching, to perform hierarchical screening of the initial fingertip candidate point set. Points with high scores in both structural coherence and anatomical rationality are prioritized for retention, while outliers with low scores or isolated structures are excluded. Finally, a set of target fingertip points with reasonable spatial distribution, strong structural consistency, and conformity to human structural characteristics is obtained.

[0016] Step S6: Perform critical angular velocity touch judgment on the target point set of the hand's fingertips and generate a touch trigger signal.

[0017] Specifically, position tracking is performed on each fingertip point across consecutive frames to establish its spatial trajectory information over time, resulting in a set of fingertip motion trajectories. By tracking the pixel-level coordinate changes of each fingertip point in consecutive frames, its displacement between each frame is calculated. Furthermore, the instantaneous velocity vector at each moment is estimated by combining the frame time interval. Simultaneously, based on the changes in velocity direction between consecutive frames, the corresponding angular velocity value is derived, forming a velocity feature dataset encompassing the dynamic behavior of all fingertips throughout the entire observation time window. This dataset records the spatial movement direction and speed of each point and reflects key dynamic features of the fingertips during hand movements, such as acceleration, deceleration, and turning. Based on the velocity feature dataset, the deceleration trend of each fingertip point is analyzed. By comparing the change in velocity magnitude between the current frame and the previous three frames, it is determined whether the current fingertip is undergoing a significant velocity decrease, i.e., identifying the occurrence and degree of deceleration. Based on the different degrees of deceleration, the fingertip action state is divided into three stages: the rapid movement stage, the deceleration transition stage, and the touch preparation stage. The rapid movement phase represents a fingertip still in a high-energy state, lacking the potential for touch control; the deceleration transition phase shows a decrease in fingertip speed but not yet at a critical point; while the touch preparation phase is strongly correlated with an impending touch, corresponding to a low fingertip speed and frequent directional changes. These phase divisions collectively constitute fingertip deceleration trend data, used in subsequent touch behavior determination logic. To characterize touch behavior under different speed conditions, an adaptive threshold parameter related to the initial speed level is dynamically set for each fingertip. The dynamic threshold is a weighted combination based on the speed magnitude measured in the first three frames and a set of preset adjustment coefficients, with a basic offset term added to form a fingertip touch determination threshold table. The speed feature dataset, deceleration trend data, and touch determination threshold table are jointly analyzed to implement multi-condition determination logic. The logic sets three touch detection conditions that must be met simultaneously: First, the current deceleration trend value of the fingertip must exceed a preset target value, indicating that the fingertip is rapidly decelerating and approaching a standstill; second, the angular velocity calculated in the current frame must exceed a first angular velocity threshold, indicating that the fingertip has exhibited a significant change in direction, matching the condition of "the fingertip approaching and preparing to contact the surface"; third, the current instantaneous velocity must be less than a second velocity threshold, indicating that the fingertip has essentially stopped its high-speed movement and is in a potential contact state. Only when all three conditions are met within the same time frame will the system confirm a valid touch action and generate a corresponding touch trigger signal for subsequent interface interaction or system response.

[0018] In one specific embodiment, the process of performing step S1 may specifically include the following steps: Acquire the original hand image, perform depth image segmentation on the original hand image to obtain the initial hand contour point set, and evaluate the importance of the initial hand contour point set to obtain the hand contour hierarchical region. Sampling rates are assigned to the hand contour layer regions. Based on a preset density control function, the hand contour layer regions are divided into layers one through m. The first layer corresponds to the fingertip potential region and is assigned a sampling rate R1. The second through m-1 layers correspond to the finger regions and are assigned decreasing sampling rates R2 through R... m-1 The minimum sampling rate R is assigned to the inner region of the palm corresponding to the m-th level. m Satisfying R1>R2>...>R m The decreasing sampling rate relationship is used to obtain the hierarchical sampling parameter set; Calculate the local curvature value of each contour point in the initial hand contour point set to obtain the hand contour curvature distribution map; The initial hand contour point set is subjected to non-uniform downsampling based on the hierarchical sampling parameter set and the hand contour curvature distribution map to obtain a hierarchical sparse point set of the hand.

[0019] Specifically, the image acquisition module obtains raw image data containing the complete shape of the hand. Depth image segmentation technology is used to extract the hand region from the image. This step relies not only on the edge texture of the RGB image but also on depth information to semantically isolate the foreground hand from the background region, constructing a high-confidence hand region mask. Based on this, edge contours are extracted as the initial hand contour point set. This point set exists as a sequence of pixel coordinates continuously distributed along the hand boundary in the image, representing the complete geometric contours of the palm, fingers, and fingertips. To improve downsampling efficiency and preserve the information density of key areas, an importance assessment is performed on the initial hand contour point set, thereby establishing a spatial hierarchy of the hand structure and dividing the contour points into several regions with different processing priorities. The evaluation criteria are mainly based on the position and shape characteristics of the contour points in the geometric structure. Points near the fingertips or with large curvature are assigned higher importance levels, while areas near the center of the palm or with gentle geometric deformation are classified as low-importance regions. The entire hand contour point set is divided into multi-level regions, each level representing an importance gradient, arranged sequentially from the high-feature region of the fingertip to the low-feature region of the palm, forming the hierarchical region structure of the hand contour. Sampling rates are assigned to each level region based on a preset density control function to establish a point density distribution model that conforms to structural priority. All levels are labeled as level 1 to level m, where level 1 corresponds to the potential fingertip region, considered the most information-dense and detail-retaining location, and therefore assigned the highest sampling rate R1; while levels 2 to (m-1) correspond to the root joint region and the middle region of the finger respectively, with progressively decreasing sampling density, and sampling rates set to R2 to R... m-1The m-th level corresponds to low-feature regions such as the center of the palm, where structural changes are gradual and information density requirements are lowest. Therefore, only the lowest sampling rate R is assigned. m The density control function is a decreasing nonlinear form to ensure higher data retention accuracy in higher-level regions, while lower-level regions primarily retain contour trends, thus maximizing the preservation of key morphological features while compressing the overall point set size. The system constructs a hierarchical sampling parameter set by combining each level with its corresponding sampling rate. This parameter set forms the control benchmark for subsequent non-uniform downsampling. Local curvature is calculated for each point in the initial contour point set. The curvature value quantifies the degree of bending of the point on the boundary curve; points with high curvature represent fingertips, joints, or sharp contour turns, which are geometrically critical nodes of the hand structure. By statistically analyzing the curvature values ​​of all points and mapping them onto the contour map, a hand contour curvature distribution map is constructed. This map depicts the bending trend and density variation of the hand at different locations, providing structural information constraints for downsampling. Non-uniform downsampling is then performed on the initial hand contour point set based on the hierarchical sampling parameter set and the hand contour curvature distribution map. During this process, each initial contour point is affected not only by the sampling rate corresponding to its level but also by the adjustment effect of its local curvature value. In high-level regions, even points with low curvature are more likely to be preserved; while in low-level regions, only points with curvature values ​​exceeding a set threshold are included in the sampling point set. This dual-screening mechanism effectively preserves details in structurally complex areas such as fingertips, finger gaps, and finger edges, while significantly compressing point density in structurally flat areas such as the palm and wrist. The downsampling result is the layered sparse point set of the hand.

[0020] In one specific embodiment, the process of performing step S2 may specifically include the following steps: Calculate the local density weight and regional importance kernel function of each point in the sparse point set of the hand segment to obtain the point set density weight and importance kernel function values; The feature enhancement factor of each point in the sparse point set of the hand is calculated, and the feature weight is assigned by combining the curvature value of the point with the distance from the point to the hand boundary, so as to obtain the feature enhancement factor value of the point set. A core equation for feature mapping is constructed and combined with point set density weights, importance kernel function values, and feature enhancement factor values ​​to perform feature space coordinate mapping on points in the sparse point set of the hand layer, thereby obtaining the hand feature enhancement point set.

[0021] Specifically, a local perceptual neighborhood is established on the two-dimensional image plane, centered on each sparse point. Statistical analysis is performed on other sparse points within a certain range around this neighborhood. By statistically analyzing the number and distance distribution of each point's neighbors within its neighborhood, a local density weight is calculated. This weight reflects the degree of structural aggregation at the point's location; higher density indicates it is located in a densely populated edge region or a key geometric corner, and therefore should occupy a higher weight in spatial feature representation. Simultaneously, to enhance the point's positional sensitivity within the global structure, a regional importance kernel function is constructed for each point. This function, centered on the point, adaptively adjusts its response range based on its affiliation at different levels (e.g., fingertip, finger, palm), making the kernel function response sharper in the fingertip region and smoother in the palm region. This results in significant structural differences in response values ​​within the same distance, highlighting the feature representation of highly important regions from the global structure. The feature enhancement factor for each point is calculated. This factor is a dual model of the intensity of local geometric changes and the relationship between the position and the boundary. First, the system extracts the curvature value based on the local contour fitting result of each sparse point. This value can effectively quantify the degree of contour curvature at the point. Structures such as fingertips, finger gaps, or knuckles have significant curvature values ​​that are significantly greater than the gentler areas at the edge of the palm due to obvious geometric turning. On the other hand, the shortest distance from the point to the overall hand boundary is measured. Since points closer to the edge are more likely to participate in the morphological features of the outer contour or finger area, this distance index is introduced as another key factor into the feature enhancement weight calculation, forming an enhancement mode of "curvature multiplied by boundary sensitivity". Combining these two factors, a feature enhancement factor value is assigned to each point, thereby strengthening the spatial representation of key points with drastic edge structure changes and close to the outer contour, and improving the accuracy of local geometric feature recognition. The core equation of feature mapping is constructed and combined with the point set density weight, importance kernel function value, and feature enhancement factor value to perform feature space coordinate mapping on the points in the sparse point set of the hand layer. This comprehensive mapping mechanism, based on the current image spatial coordinates, reprojects each sparse point into the feature space. During the mapping process, the final response value of each point is jointly determined by three attributes: density weight reflects the degree of structural crowding, kernel function response reflects the regional importance weight distribution, and feature enhancement factor reflects the intensity of geometric boundary changes. These three attributes together determine the strength and significance of the point's representation in the feature space. The system constructs a spatially continuous response field from the mapping results of all points. In this field, key structures such as fingertips, knuckles, and edge corners form feature peak regions, while other flat regions exhibit response troughs, forming a natural feature distribution gradient. To enhance the robustness and controllability of the model, the response field is normalized, and meaningless weak response points are removed based on a set response intensity threshold, retaining points with clear structural representation and significant responses to form the final hand feature enhancement point set.

[0022] In one specific embodiment, the process of performing step S3 may specifically include the following steps: Construct a dynamic frame buffer and store a set of hand feature enhancement points for t consecutive frames, where t is a preset number of frames; Half-cycle iterative control is performed on the hand feature enhancement point set, and the convex hull calculation cycle is triggered only in odd frames or when the rate of change of the point set is detected to be greater than the preset threshold ε, so as to obtain the calculation trigger signal; Based on the calculation trigger signal, the set of hand feature enhancement points in the current frame is divided into a variable subset that needs to be calculated and a static subset that does not need to be calculated; Graham's convex hull scanning algorithm is performed only on the changing subset, and the points are sorted according to their polar angles and the real-time convex hull of the changing region is constructed by stack operations and cross product judgment. By directly reusing the convex hull result of the previous calculation cycle for the static subset without performing any new calculations, the historical convex hull of the static region is obtained. Topological boundary stitching is performed on the real-time convex hull of the changing region and the historical convex hull of the static region to obtain the convex hull of the hand contour.

[0023] Specifically, a frame buffer structure with real-time update capability is established. This structure stores the hand feature enhancement point set of consecutive frames t in a circular manner, where t is a preset buffer length. Its size is adjusted according to the frame rate, computation latency, and gesture response time, ensuring sufficient historical information without excessive pressure on memory or computing resources. During the image stream entering the system, each newly generated feature enhancement point set is written to this buffer, while older data is dequeued in chronological order, achieving continuous time-series data maintenance. After completing the storage of consecutive frames, a semi-cycle convex hull iteration control judgment is performed on each frame entering the processing flow. This involves determining whether to trigger a new round of convex hull calculation based on the current frame number and the magnitude of change between the current frame and the previous frame. This control logic employs a dual-condition judgment strategy: firstly, a frame number constraint in the time dimension, where calculation is only triggered when the current frame is an odd-numbered frame; secondly, a change judgment in the data dimension, where the geometric change of the enhancement point set in the current frame compared to the previous frame determines whether the calculation threshold is met. The system uses indicators such as spatial distribution differences between point sets, changes in the number of feature points, or mean coordinate drift as the basis for the rate of change. By comparing this rate of change with a preset change threshold ε, if the change exceeds this threshold, a convex hull update process is forcibly triggered even if the frame number is even, thus achieving a balance between computation and stability. Once the trigger condition is met, the system enters the point set partitioning stage, dividing all feature enhancement points in the current frame into two sets based on their similarity to the previous frame. The changing subset includes newly added points, points with significant displacement, or points that have undergone topological changes in structure, while the static subset includes points with stable positions and unchanged boundaries. This partitioning ensures that computational resources are concentrated on structurally active regions, while reusing resources in unchanged regions reduces redundant overhead. After acquiring the changing subset, the Graham scan algorithm is executed to construct the convex hull. This algorithm uses the point with the smallest y-coordinate and the smallest x-coordinate as the polar angle reference point. Then, all points in the changing subset are sorted according to their polar angle with that point. Next, the boundary path is constructed step by step through a stack structure. Before each push operation, the cross product operation is performed on the vector formed by the current point and the top two points of the stack to determine whether a valid boundary is formed in the counterclockwise direction. If it is a right turn, the stack is popped; if it is a left turn, the stack is pushed, thus constructing a real-time convex hull of the changing region containing all valid boundary points. At the same time, for static subsets that have not changed, the convex hull result corresponding to that subset in the previous cycle is directly called and its boundary path is reused. This history reuse mechanism reduces the repetitive calculation burden of the system at high frame rates, which is especially effective when the hand is in a stable or slightly moving state, allowing the system to save energy and resources while maintaining real-time performance. After the convex hull results of the two subsets are generated, the topology stitching stage begins. In this stage, the degree of overlap and tangent direction of the real-time convex hull of the changing region and the historical convex hull of the static region at the boundary connection are compared. Operations such as point sorting, boundary splicing, and overlap fusion are then performed to merge the two into a whole continuous closed convex hull boundary.During the stitching process, overlapping points are removed and gaps are filled to ensure that the final hand contour convex hull is not only geometrically closed but also topologically seamless. The stitched hand contour convex hull is the output boundary data for this processing cycle. This data has complete fingertip morphology, finger structure, and palm edge contour, ensuring computational efficiency without sacrificing recognition accuracy.

[0024] In one specific embodiment, the process of performing the Graham scan convex hull algorithm only on the changing subset, and constructing the real-time convex hull of the changing region by sorting the points according to their polar angles and using stack operations and cross product to determine the convex hull, can specifically include the following steps: Perform coordinate analysis on all points in the variation subset, find the point with the smallest y-coordinate as the reference point, and if there are multiple points with the smallest y-coordinate, select the point with the smallest x-coordinate to obtain the starting reference point of the convex hull. Polar angles are calculated for the remaining points in the variation subset based on the starting reference point of the convex hull, resulting in a polar angle sorting sequence for the point set. Push the starting reference point of the convex hull and the two points with the smallest polar angles in the polar angle sorting sequence of the point set onto the convex hull point stack as the initial point stack of the convex hull; The remaining points in the polar angle sorting sequence of the point set are processed by stack operations in ascending order of polar angle. The turning direction is determined by calculating the cross product of the two vectors formed by the current point and the top two points of the stack. When the cross product is negative, it means that a right turn is formed. When the cross product is positive, it means that a left turn is formed. The current point is pushed onto the stack to obtain the convex hull candidate point stack. Connect all points in the candidate stack of the convex hull in clockwise order according to the order they were pushed onto the stack to obtain the real-time convex hull of the changing region.

[0025] Specifically, coordinate analysis is performed on all points in the variation subset to determine the geometric starting point of the entire convex hull. During the analysis, the two-dimensional coordinate values ​​of each point are read sequentially. The y-coordinate values ​​are compared first, and the point with the smallest y-coordinate is selected as the reference. If multiple points have the same minimum y-coordinate value, the x-coordinates are compared, and the point with the smallest x-coordinate is selected as the final starting reference point for the convex hull. This selection ensures that the starting point is in the lower left of the entire point set, giving subsequent polar angle sorting and path connections a consistent direction and logical starting point, thus constructing a stable boundary topology. After selecting the starting reference point, it is used as the reference origin for polar angle calculation. Polar angle calculation is performed on all other points in the variation subset except for this point. The polar angle is the angle between the vector formed by the reference point and all target points relative to the positive horizontal direction. This angle establishes the directional sorting system for the starting point set. Using these polar angle values, all non-reference points in the point set are arranged in ascending order, constructing a polar angle-increasing sorting sequence. This sequence provides a clear processing order for subsequent convex hull stack construction. The reference point and the two points with the smallest polar angles in the sorted sequence are pushed sequentially onto a stack structure. This stack is used to dynamically store the confirmed convex hull points on the current boundary path, serving as the initial convex hull point stack for the entire Graham scan method. This initial stack ensures the consistency of the path's starting direction and provides sufficient vector basis for subsequent direction determination. The remaining points are traversed one by one in polar angle sorting order, and a push-on judgment operation is performed on each new point. This process combines the relative direction relationship between the two points at the top of the stack and the current candidate point to make a decision. The system calculates the cross product between the two vectors formed by these three points. The sign of the cross product reflects the turning direction of the path. When the cross product is negative, it indicates that the current path forms a right turn, which does not meet the convex hull enclosing direction rule. Therefore, the stack vertices need to be popped, thus backtracking one step to adjust the boundary path. When the cross product is positive, it indicates that the three points form a left turn, which meets the geometric condition for constructing a counterclockwise convex hull. At this time, the system pushes the current point onto the stack, indicating that it is a point that can be retained on the current boundary. This process continues until all points have been processed. The contents of the stack constitute the candidate convex hull point set of the current changing subset, which geometrically forms a set of boundary points enclosing the outermost part of the structure. After the candidate point stack is constructed, the boundary is reconstructed according to the order in which the points are pushed onto the stack. A closed contour path is formed by connecting each point clockwise, ultimately obtaining the real-time convex hull of the changing region. Since the order in which the points are pushed onto the stack ensures the geometric consistency of the boundary, the system only needs to connect every two adjacent points sequentially to construct a complete boundary contour curve. This curve completely encloses all points of the changing subset in space and does not contain any concave regions or path reversals, thus exhibiting extremely high geometric stability and structural clarity.This process not only updates the boundaries of the currently changing region in real time, but also significantly reduces the involvement of redundant points through dynamic judgment of the stack structure, improving the efficiency and accuracy of boundary construction. The final convex hull result serves as an important input for subsequent fingertip extraction and region analysis, providing a geometric foundation for the system to complete touch intent recognition, structural coherence analysis, and behavior prediction.

[0026] In one specific embodiment, the process of performing step S4 may specifically include the following steps: Calculate the interior angle of each convex hull point in the hand contour convex hull, and mark the convex hull points with interior angles less than a preset angle threshold as initial fingertip feature points to obtain the initial fingertip candidate point set; Calculate the local curvature features of each initial fingertip feature point in the initial fingertip candidate point set, and determine the curvature radius by fitting the initial fingertip feature points and their neighboring convex hull points to obtain the candidate point curvature feature set; Calculate the corner response value of each candidate point in the initial fingertip candidate point set, and analyze the corner saliency based on the structure tensor centered on the candidate point to obtain the candidate point corner response value set; A fingertip probability scoring function is constructed, and a comprehensive score is calculated by combining the candidate point curvature feature set and the candidate point corner response value set to obtain the candidate point score result. At the same time, the candidate point score results are sorted, and the hand fingertip candidate point set with the highest score is selected.

[0027] Specifically, for each boundary point on the convex hull of the hand contour, the interior angle is calculated based on its adjacency relationship on the contour curve. These interior angles are then compared with preset angle thresholds to identify points with sharp turning features in the curve structure. When three adjacent convex hull points are connected sequentially in space to form a polygonal line, the interior angle at the midpoint of the polygonal line reflects the curvature of the curve at that point. The system traverses the entire convex hull boundary, marking all points with interior angles smaller than the threshold as initial fingertip feature points, forming an initial set of candidate fingertip points. In this process, the selection of the threshold must consider both the physical characteristics of the fingertip being typically located at the convex position of the contour and the possible slight curvature differences in the fingers, to avoid misclassifying knuckles or outer edge imperfections as potential fingertips. For each point in the initial set of candidate fingertip points, local geometric curvature analysis is performed to obtain more refined structural information. Centered on each candidate point, several neighboring convex hull points are extracted and a continuous local boundary arc is fitted. The corresponding radius of curvature is calculated using the fitted arc, thereby quantifying the curvature of the boundary curve at that point. A smaller radius of curvature indicates a greater curvature at that point, making it more likely to be a true fingertip. Conversely, points with relatively straight or limited curvature correspond to larger radii of curvature and are therefore assigned lower geometric saliency. The radius of curvature for each initial candidate point is recorded, forming a candidate point curvature feature set. The corner saliency of each candidate point is evaluated at the image grayscale or depth map feature level to identify which candidate points have the strongest corner features in visual or depth information. For this purpose, a fixed-size local neighborhood window is taken centered on each candidate point. The structure tensor within this window is calculated, and its eigenvalue distribution is analyzed to measure the intensity of grayscale gradient changes in different directions. A greater difference in the eigenvalues ​​of the structure tensor indicates that the region where the point is located has more significant corner properties; conversely, if the grayscale or depth changes are relatively uniform, the corner response value will be lower. The corner response values ​​calculated for each candidate point within its local neighborhood are stored in the candidate point corner response value set to reflect the prominence of the candidate point in the image feature space. A fingertip probability scoring function is constructed, which weights and fuses the curvature features and corner response values ​​of candidate points according to preset weights to generate a comprehensive score. The scoring function considers two factors: the greater the curvature, the higher the probability of a fingertip; and the higher the corner response value, the more structurally significant the point. The influence of these two factors is balanced through weighting coefficients to obtain a comprehensive index that best reflects the true fingertip location. After the comprehensive scoring of all candidate points is completed, these points are sorted from highest to lowest score. Based on the number of fingers or the number of previously detected fingertips, the highest-scoring points in the sorted results are selected as the final set of candidate fingertip points.

[0028] In one specific embodiment, the process of performing step S5 may specifically include the following steps: The points in the candidate point set of the fingertips are used as vertices of the graph. First-class edge connections are established between point pairs that satisfy the first distance threshold and the second distance threshold, and second-class edge connections are established between point pairs that satisfy the third distance threshold and the fourth distance threshold, so as to obtain the initial finger relationship topology graph. Based on the initial finger relationship topology graph, the contribution values ​​of Euclidean distance between vertices, the contribution values ​​of angular constraints between vertices, and the contribution values ​​of neighborhood similarity between vertices are calculated to obtain a multi-dimensional feature-weighted finger relationship graph. Perform orthogonal decomposition to calculate the degree matrix and graph Laplacian operator of the multidimensional feature weighted inter-exponential relationship graph, and extract stable feature vectors to obtain the principal feature vector group and the secondary feature vector group; The first type of coherence score is calculated based on the main feature vector group, the second type of coherence score is calculated based on the secondary feature vector group, and the first type of coherence score and the second type of coherence score are fused to obtain a hierarchical fingertip coherence score set. By setting the interfinite distance ratio coefficient, effective range of included angle, relative position offset tolerance, and fingertip alignment consistency coefficient, the matching degree between candidate points and standard hand anatomical features is evaluated, resulting in a multidimensional anatomical constraint score set. Based on the candidate point set of the fingertips, the candidate points are graded and screened by combining the hierarchical fingertip coherence score set and the multidimensional anatomical constraint score set to obtain the target point set of the fingertips.

[0029] Specifically, each point in the initially screened candidate fingertip points is abstracted as a vertex in a graph structure, and an initial topological connection graph is constructed using the spatial relationships between these vertices. In this process, based on the Euclidean distance between point pairs, a distance judgment is made for each pair of candidate points. If the distance is between a first and a second distance threshold, the two points are considered to have a direct structural relationship in space, and a first-type edge connection is established between them to represent potentially adjacent fingertips. If the distance is between a third and a fourth distance threshold, they are considered to be point pairs that are relatively far apart but still have the potential for structural relationship, and a second-type edge connection is established between these points, thereby expanding the expressible range of the topological graph and forming an initial finger-to-finger relationship topological graph containing information on nearest and next-nearest neighbors. Multidimensional feature analysis is performed on all edge connections in the initial finger-to-finger relationship topological graph to enhance its ability to express structural relationships. For each edge connecting two endpoints, the Euclidean distance contribution is calculated, reflecting the impact of the geometric distance between the two points on the stability of the graph structure. The angular constraint contribution between each pair of points is also calculated, by measuring the angle between the connecting line and the horizontal direction or the main axis of the hand, and referencing the geometric rules of normal fingertip arrangement to determine if the angle is within a reasonable range. The neighborhood morphology of adjacent point pairs is compared, such as the directional distribution, point density, or curvature trend contained in their local feature space, to derive the neighborhood similarity contribution. These three types of indicators are integrated into the edge weighting function, ensuring that each edge not only carries geometric distance information between point pairs but also reflects directional consistency and structural matching degree, thus constructing a multidimensional feature-weighted inter-finger relationship graph. Spectral analysis is performed on this multidimensional weighted graph to calculate the graph's degree matrix, i.e., by summing the weights of the edges connected to each vertex and constructing a diagonal matrix. Based on the degree matrix and weight matrix, matrix subtraction is performed to obtain the Laplacian operator matrix of the graph, which reflects the connectivity between each point and its adjacent structures and the topological features of the graph. After obtaining the Laplacian matrix, eigenvalue decomposition is performed to obtain a series of eigenvalues ​​and their corresponding eigenvectors, which are then sorted according to their magnitude. After sorting, the eigenvectors corresponding to the k smallest eigenvalues ​​form the principal eigenvector group. These vectors represent the main connectivity patterns in the graph structure, reflecting large-scale coherence trends. The eigenvectors corresponding to the (k+1)th to 2kth eigenvalues ​​are classified as the secondary eigenvector group, used to express fine-grained or edge region structural changes in the graph. A coherence score is calculated for each candidate point in the graph based on these two sets of eigenvectors. The principal eigenvector group is used to calculate the first type of coherence score, which reflects the centrality and stability of the candidate point in the overall structural connectivity; while the secondary eigenvector group is used to calculate the second type of coherence score, focusing on local dissimilarity and anomaly detection capabilities.These two types of scoring results are merged and weighted to form a comprehensive hierarchical fingertip coherence scoring set, used to judge the organizational rationality and topological coherence of each candidate point in the hand structure diagram. To make the judgment results more consistent with the actual human anatomy, several physiological structural constraint indicators are introduced. A finger distance ratio coefficient is set to determine whether the distance between adjacent fingertips is within the normal finger opening ratio range; an effective angle range is set to measure whether the angle formed by the fingers in natural bending or opening conditions conforms to physiological laws; a relative position offset tolerance is defined to allow for small offsets caused by changes in shooting angle and grip posture in actual application; and a fingertip arrangement consistency coefficient is configured to measure the consistency and sequential regularity of multiple fingertip points in the arrangement direction. The above structural constraints are merged into a multidimensional anatomical constraint scoring set to score the degree of fit between the spatial position of each candidate point and its natural hand shape model. Based on the constructed fingertip candidate point set, and combining its structural stability in the coherence scoring set and its physiological rationality in the anatomical scoring set, a multidimensional hierarchical screening operation is performed. Points with high scores in both structural connectivity and physiological compatibility are prioritized for retention, while candidate points that do not meet the standards or exhibit structural isolation or abnormal arrangement are eliminated, ultimately determining the target point set for the fingertips.

[0030] In one specific embodiment, the process of performing orthogonal decomposition to calculate the degree matrix and graph Laplacian operator of the multidimensional feature-weighted inter-exponential relationship graph and extracting stable feature vectors to obtain the principal feature vector group and the secondary feature vector group can specifically include the following steps: The multidimensional feature-weighted inter-finite relationship graph is processed into a matrix representation to obtain the graph weight relationship matrix; The degree value of each vertex is calculated based on the graph weight relation matrix, and the degree values ​​of all vertices are organized into a graph degree diagonal matrix. Perform matrix subtraction based on the graph weight relation matrix and the graph degree diagonal matrix to obtain the graph Laplacian operator matrix; Perform eigenvalue decomposition and eigenvalue sorting on the graph Laplacian operator matrix to obtain the sorted eigenvalue sequence and the corresponding eigenvector sequence; Spectral clustering is performed based on the sorted feature value sequence and the corresponding feature vector sequence. The feature vectors corresponding to the smallest k feature values ​​are divided into the primary feature vector group, and the feature vectors corresponding to the second smallest k+1 to 2k feature values ​​are divided into the secondary feature vector group.

[0031] Specifically, the multi-dimensional feature-weighted finger-point relationship graph is processed using matrix representation. In the graph structure, vertices represent candidate fingertip points, and the weighted values ​​of edges comprehensively describe the fusion relationship of multi-dimensional features such as Euclidean distance, angular constraints, and neighborhood similarity between two points. The weights of all edges in the graph are represented as a two-dimensional array, forming the graph's weight relationship matrix. Each matrix element represents the weighted connection strength between two vertices; points with no connection are set to zero, thus ensuring the sparsity and symmetry of the matrix. After constructing the weight matrix, the degree value of each vertex in the graph is calculated, which is the sum of the connection weights between a point and all other points. The degree value reflects the connectivity activity of the vertex in the entire graph structure; a larger degree value indicates a more critical structural position in the topological network. All degree values ​​are uniformly organized into a diagonal matrix, where the degree values ​​of each point are on the main diagonal, and the remaining elements are zero, forming the graph's degree diagonal matrix, providing structural constraints for subsequent graph operator calculations. According to the definition of the Laplacian operator in spectral graph theory, the standard Laplacian operator matrix of a graph is obtained by performing matrix subtraction on the graph degree diagonal matrix and the weight relation matrix. This matrix, centered on structural differences, integrates the differences in connection strength between vertices and the degree of node aggregation within a unified framework, used to analyze connectivity, clustering trends, and relative stability between nodes in the graph structure. The Laplacian matrix has a zero row sum, and its symmetry and positive semi-definite properties ensure effective eigenvalue decomposition. Eigenvalue decomposition is performed on the Laplacian matrix to solve for all eigenvalues ​​and their corresponding eigenvectors. Eigenvalues ​​reveal connection patterns of different frequencies in the graph structure; smaller eigenvalues ​​represent strong connectivity and slow-changing main structural patterns, while larger eigenvalues ​​reflect structural tensions between marginal, local, or outlier points. Arranging all eigenvalues ​​in ascending order forms a spectral sequence from low-frequency to high-frequency patterns, and the corresponding eigenvectors constitute the orthogonal basis of the graph structure space. Based on the sorted spectral structure information, a spectral clustering strategy is used for structural connectivity analysis. The eigenvectors corresponding to the k smallest eigenvalues ​​are selected to form the principal eigenvector group. These vectors describe the strongest connected components and core structures in the graph, representing the main arrangement patterns of the fingers and the main tissue paths of the fingertips. Subsequently, the eigenvectors corresponding to the (k+1)th to 2kth eigenvalues ​​are selected to form the secondary eigenvector group. This group is used to analyze structural points located at the margins, unstable states, or potentially anomalous states. This part of the vectors helps to capture atypical arrangements or local outliers. The principal and secondary eigenvector groups will be used as inputs for fingertip coherence scoring and anatomical rationality determination, respectively, providing multi-level spectral analysis support for the system to select the final fingertip target points, ensuring both structural stability and spatial consistency are achieved.

[0032] In one specific embodiment, the process of performing step S6 may specifically include the following steps: By tracking the position of each fingertip point in the target point set of the hand's fingertips in consecutive frames, a set of fingertip motion trajectories is obtained; The instantaneous velocity vector and angular velocity value of each fingertip are calculated based on the fingertip motion trajectory set to obtain the fingertip velocity feature dataset; The fingertip deceleration trend is calculated by comparing the speed magnitude difference between the current frame and the previous three frames. Based on the degree of deceleration, the fingertip action is divided into a rapid movement phase, a deceleration transition phase, and a touch preparation phase to obtain fingertip deceleration trend data. For each fingertip point, set an initial speed-related dynamic threshold parameter. At the same time, based on the dynamic threshold parameter, multiply the speed values ​​of the first three frames by a preset coefficient and add a basic offset value to obtain a fingertip touch judgment threshold table. Based on the fingertip touch judgment threshold table, multi-condition touch judgment analysis is performed on the fingertip speed feature dataset and fingertip deceleration trend data. When the three conditions of fingertip deceleration trend being greater than the preset target value, angular velocity being greater than the first angular velocity threshold and instantaneous velocity being less than the second velocity threshold are met simultaneously, a touch trigger signal is obtained.

[0033] Specifically, the system tracks the position of each fingertip in a target point set across consecutive frames and constructs a time-sliding window to store the two-dimensional coordinates of each fingertip in the most recent few frames. The length of this sliding window is determined by the system frame rate, the expected finger movement frequency, and the response latency tolerance, and is set to 3 to 5 frames. Whenever a new frame enters the processing flow, the system establishes a mapping relationship between the position of each fingertip in the current frame and its position in the previous frame, and updates the current coordinates into the motion trajectory history of that point, thus forming a set of fingertip motion trajectories. Based on this trajectory set, the spatial displacement of each fingertip between consecutive frames is processed. The position difference is divided by the time interval to obtain its instantaneous velocity vector, which contains direction information and quantifies the magnitude of the velocity. The angular velocity value is calculated by comparing the angle of change of direction between two adjacent velocity vectors and dividing by the corresponding frame interval to obtain the rate of change of the fingertip's motion direction. This process generates a fingertip velocity feature dataset for each fingertip, including speed and direction change characteristics. This dataset contains both a temporal sequence of velocity magnitude and the evolution trend of angular velocity, used to dynamically analyze whether the fingertip is in a stable movement, suddenly decelerating, or about to touch. After velocity feature extraction, the velocity magnitude of each fingertip in the current frame is compared with that of the previous three frames, and the deceleration trend of the fingertip during this period is calculated. By analyzing the monotonically decreasing velocity trend, average change magnitude, and slope of the velocity change curve across consecutive frames, it is determined whether the current point is rapidly losing speed. Based on the magnitude of the deceleration trend value, each fingertip action is divided into three stages: a rapid movement stage (high speed, weak deceleration), a deceleration transition stage (speed begins to decrease but not yet critical), and a touch preparation stage (speed continues to decrease, direction tends to stabilize), forming a motion state label set based on velocity changes, i.e., fingertip deceleration trend data. This label provides a prerequisite state judgment for touch detection, preventing accidental touches during high-speed movement or premature triggering before the action is completed. To improve the adaptability of touch detection and its compatibility with individual dynamic characteristics, an initial velocity-related dynamic threshold parameter is set for each fingertip. This dynamic threshold parameter is automatically derived based on the point's velocity performance in historical trajectories. The system calculates the average velocity value of each fingertip in the first three frames and introduces multiple adjustable preset coefficients to dynamically scale this average velocity. A fixed offset is also added to accommodate sensitive detection at low speeds, generating multiple detection boundaries for the current moment, including touch velocity threshold, deceleration trend threshold, and angular velocity critical threshold, thus forming a fingertip touch detection threshold table. In each frame, the aforementioned velocity feature dataset, deceleration trend data, and threshold table are used in combination to execute a multi-condition touch detection logic.The logic is set so that the following three conditions must be met simultaneously to trigger a valid touch signal: First, the deceleration trend of the fingertip in the current frame must be greater than a preset target value, meaning the fingertip is significantly decelerating and showing a tendency to stop; second, the angular velocity of the fingertip is greater than a first angular velocity threshold, indicating a significant change in the fingertip's direction, which is usually a turning point feature in touch actions; third, the instantaneous velocity is less than a second velocity threshold, indicating that the finger speed has dropped to a near-stationary level, meeting the physical conditions for touch to occur. When these three conditions are simultaneously met by the fingertip, the system determines that the fingertip is in a "touched" state, immediately sends a touch trigger signal, and feeds the trigger event back to the upper-level interaction system for operations such as interface response, control logic execution, or data recording.

[0034] The above describes the fingertip extraction touch method based on the convex hull algorithm in the embodiments of the present invention. The following describes the fingertip extraction touch system based on the convex hull algorithm in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the fingertip extraction touch system based on the convex hull algorithm in this invention includes: The layered sparse sampling module is used to acquire the original image of the hand and extract the layered sparse point set of the hand. The feature space mapping module is used to perform feature space mapping on the sparse point set of the hand layer to obtain the hand feature enhancement point set. The semi-periodic convex hull iteration module is used to generate the convex hull of the hand contour based on the enhanced point set of hand features; The point angle analysis module is used to perform point angle analysis on the convex hull of the hand contour to obtain a set of candidate points for the fingertips. The finger-interdigitation relationship constraint analysis module is used to perform finger-interdigitation relationship constraint analysis on the candidate point set of the hand's fingertips to obtain the target point set of the hand's fingertips; The touch detection module is used to perform critical angular velocity touch detection on the target point set of the hand's fingertips and generate touch trigger signals.

[0035] Through the collaborative efforts of the aforementioned components, a hierarchical sparse sampling strategy is employed to perform non-uniform downsampling of the hand contour. This preserves high-density feature points in the fingertip region while reducing redundant points in non-critical areas, significantly reducing the computational load of subsequent processing. By identifying changing and static regions, convex hull calculation is performed only on regions within the field of view where changes are predicted, while historical results are directly reused for static regions, greatly reducing the computational overhead in static regions and improving system response speed. By constructing a fingertip relationship network represented by vertices, edges, and weights, and using the graph Laplacian operator to quantify the physical correlation between fingertips, false detection points that violate hand anatomy are automatically identified and filtered, improving detection accuracy. Combining a fingertip deceleration function and adaptive thresholding, the dynamic features of user touch intentions are accurately captured, effectively distinguishing different operation intentions such as hovering, swiping, and clicking, significantly reducing the system's false touch rate. By mapping the contour point set to a sparse feature space through core equations, the feature representation of the fingertip region is strengthened, improving the recognition accuracy of fingertip candidate points and enhancing the system's ability to handle complex gestures. By periodically analyzing touch success rate and false touch rate, and dynamically updating angular velocity and deceleration threshold parameters, the system can adapt to different user operating habits and environmental conditions, improving overall system robustness. Seamless connection is achieved between real-time convex hulls of changing regions and historical convex hulls of static regions, ensuring the integrity and continuity of the final convex hull result, avoiding inconsistencies caused by region segmentation, and enhancing system stability during rapid gesture transitions.

[0036] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0037] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0038] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fingertip touch extraction method based on convex hull algorithm, characterized in that, include: Acquire raw images of the hand and extract layered sparse point sets of the hand; The hand segment sparse point set is mapped in the feature space to obtain the hand feature enhancement point set; A convex hull of the hand contour is generated based on the hand feature enhancement point set; A point angle analysis is performed on the convex hull of the hand contour to obtain a set of candidate points for the fingertips; The fingertip candidate point set is subjected to interfinite relationship constraint analysis to obtain the fingertip target point set; A critical angular velocity touch judgment is performed on the target point set of the hand fingertips to generate a touch trigger signal.

2. The fingertip extraction touch method based on convex hull algorithm according to claim 1, characterized in that, The process of acquiring the original hand image and extracting the layered sparse point set of the hand includes: Acquire an original hand image, perform depth image segmentation on the original hand image to obtain an initial hand contour point set, and evaluate the importance of the initial hand contour point set to obtain the hand contour hierarchical region. The sampling rate of the hand contour hierarchical region is allocated. Based on a preset density control function, the hand contour hierarchical region is divided into a first level to a m-th level, where the first level corresponds to the fingertip potential region and is allocated a sampling rate R1, and the second level to the (m-1)-th level corresponds to the finger region and is allocated a decreasing sampling rate R2 to R... m-1 The minimum sampling rate R is assigned to the inner region of the palm corresponding to the m-th level. m Satisfying R1>R2>...>R m The decreasing sampling rate relationship is used to obtain the hierarchical sampling parameter set; Calculate the local curvature value of each contour point in the initial hand contour point set to obtain a hand contour curvature distribution map; Based on the hierarchical sampling parameter set and the hand contour curvature distribution map, the initial hand contour point set is subjected to non-uniform downsampling processing to obtain a hierarchical sparse point set of the hand.

3. The fingertip extraction touch method based on convex hull algorithm according to claim 1, characterized in that, The step of mapping the sparse point set of the hand segment to its feature space to obtain the hand feature enhancement point set includes: Calculate the local density weight and regional importance kernel function of each point in the hand-layered sparse point set to obtain the point set density weight and importance kernel function values; The feature enhancement factor of each point in the sparse point set of the hand is calculated, and the feature weight is assigned by combining the curvature value of the point with the distance from the point to the hand boundary to obtain the feature enhancement factor value of the point set. By constructing a core equation for feature mapping and combining the point set density weights, the importance kernel function values, and the feature enhancement factor values, feature space coordinate mapping is performed on the points in the sparse point set of the hand layer to obtain the hand feature enhancement point set.

4. The fingertip extraction touch method based on convex hull algorithm according to claim 1, characterized in that, The step of generating the convex hull of the hand contour based on the hand feature enhancement point set includes: Construct a dynamic frame buffer and store a set of hand feature enhancement points for t consecutive frames, where t is a preset number of frames; A half-cycle iterative control is performed on the hand feature enhancement point set, and the convex hull calculation cycle is triggered only in odd frames or when the rate of change of the point set is detected to be greater than a preset threshold ε, so as to obtain the calculation trigger signal. Based on the calculation trigger signal, the set of hand feature enhancement points in the current frame is divided into a variable subset that needs to be calculated and a static subset that does not need to be calculated; Graham's convex hull algorithm is performed only on the changed subset, and the points are sorted by polar angle and the real-time convex hull of the changed region is constructed by stack operation and cross product judgment. The convex hull result of the previous calculation cycle is directly reused for the static subset without performing any new calculations to obtain the historical convex hull of the static region. A topological boundary stitching process is performed on the real-time convex hull of the changing region and the historical convex hull of the static region to obtain the convex hull of the hand contour.

5. The fingertip extraction touch method based on convex hull algorithm according to claim 4, characterized in that, The step of performing the Graham scan convex hull algorithm only on the changed subset, and constructing the real-time convex hull of the changed region by sorting the points according to their polar angles and using stack operations and cross product judgment, includes: Perform coordinate analysis on all points in the changed subset, find the point with the smallest y-coordinate as the reference point, and if there are multiple points with the smallest y-coordinate, select the point with the smallest x-coordinate to obtain the starting reference point of the convex hull. Based on the starting reference point of the convex hull, the polar angles of the remaining points in the changing subset are calculated to obtain the polar angle sorting sequence of the point set; Push the convex hull starting reference point and the two points with the smallest polar angles in the polar angle sorting sequence of the point set onto the convex hull point stack as the convex hull initial point stack. The remaining points in the polar angle sorting sequence of the point set are processed by stack operation in ascending order of polar angle. The turning direction is determined by calculating the cross product of the two vectors formed by the current point and the top two points of the stack. When the cross product is negative, it means that a right turn is formed. When the cross product is positive, it means that a left turn is formed. The current point is pushed onto the stack to obtain the convex hull candidate point stack. Connect all points in the candidate stack of the convex hull in clockwise order according to the order in which they were pushed onto the stack to obtain the real-time convex hull of the changing region.

6. The fingertip extraction touch method based on convex hull algorithm according to claim 1, characterized in that, The point angle analysis of the convex hull of the hand contour to obtain the candidate point set of the fingertips includes: Calculate the interior angle of each convex hull point in the convex hull of the hand contour, and mark the convex hull points with interior angles less than a preset angle threshold as initial fingertip feature points to obtain an initial set of fingertip candidate points; Calculate the local curvature features of each initial fingertip feature point in the initial fingertip candidate point set, and determine the radius of curvature by fitting the initial fingertip feature point and its neighboring convex hull points to obtain the candidate point curvature feature set; Calculate the corner response value of each candidate point in the initial fingertip candidate point set, and analyze the corner saliency based on the structure tensor centered on the candidate point to obtain the candidate point corner response value set; A fingertip probability scoring function is constructed, and a comprehensive score is calculated by combining the candidate point curvature feature set and the candidate point corner response value set to obtain the candidate point scoring results. At the same time, the candidate point scoring results are sorted, and the hand fingertip candidate point set with the highest score is selected.

7. The fingertip extraction touch method based on convex hull algorithm according to claim 1, characterized in that, The step of performing interfinite relationship constraint analysis on the candidate fingertip points to obtain the target fingertip point set includes: The points in the candidate fingertip points set are used as vertices of the graph. A first type of edge connection is established between point pairs that satisfy the first distance threshold and the second distance threshold, and a second type of edge connection is established between point pairs that satisfy the third distance threshold and the fourth distance threshold, to obtain an initial finger relationship topology graph. Based on the initial finger-relationship topology graph, the Euclidean distance contribution value between vertices, the angular constraint contribution value between vertices, and the neighborhood similarity contribution value between vertices are calculated to obtain a multi-dimensional feature-weighted finger-relationship graph. Perform orthogonal decomposition to calculate the degree matrix and graph Laplacian operator of the multidimensional feature-weighted inter-finite relationship graph, and extract stable feature vectors to obtain the main feature vector group and the secondary feature vector group; The first type of coherence score is calculated based on the main feature vector group, the second type of coherence score is calculated based on the secondary feature vector group, and the first type of coherence score and the second type of coherence score are fused to obtain a hierarchical fingertip coherence score set. By setting the interfinite distance ratio coefficient, effective range of included angle, relative position offset tolerance, and fingertip alignment consistency coefficient, the matching degree between candidate points and standard hand anatomical features is evaluated, resulting in a multidimensional anatomical constraint score set. Based on the candidate point set of the hand fingertips, the candidate points are graded and screened by combining the hierarchical fingertip coherence score set and the multidimensional anatomical constraint score set to obtain the target point set of the hand fingertips.

8. The fingertip extraction touch method based on convex hull algorithm according to claim 7, characterized in that, The orthogonal decomposition process calculates the degree matrix and graph Laplacian operator of the multidimensional feature-weighted inter-index graph and extracts stable feature vectors to obtain a principal feature vector set and a secondary feature vector set, including: The multidimensional feature-weighted inter-finite relationship graph is processed into a matrix representation to obtain the graph weight relationship matrix; The degree value of each vertex is calculated based on the graph weight relation matrix, and the degree values ​​of all vertices are organized into a graph degree diagonal matrix. Perform matrix subtraction based on the graph weight relation matrix and the graph degree diagonal matrix to obtain the graph Laplacian operator matrix; Perform eigenvalue decomposition and eigenvalue sorting on the graph Laplacian operator matrix to obtain a sorted eigenvalue sequence and a corresponding eigenvector sequence; Based on the sorted feature value sequence and the corresponding feature vector sequence, spectral clustering is performed. The feature vectors corresponding to the smallest k feature values ​​are divided into the main feature vector group, and the feature vectors corresponding to the second smallest k+1 to 2k feature values ​​are divided into the secondary feature vector group.

9. The fingertip extraction touch method based on convex hull algorithm according to claim 1, characterized in that, The step of performing a critical angular velocity touch judgment on the target point set of the fingertips and generating a touch trigger signal includes: By tracking the position of each fingertip point in the set of fingertip target points in consecutive frames, a set of fingertip motion trajectories is obtained; Based on the fingertip motion trajectory set, the instantaneous velocity vector and angular velocity value of each fingertip point are calculated to obtain the fingertip velocity feature dataset; The fingertip deceleration trend is calculated by comparing the speed magnitude difference between the current frame and the previous three frames. Based on the degree of deceleration, the fingertip action is divided into a rapid movement phase, a deceleration transition phase, and a touch preparation phase to obtain fingertip deceleration trend data. For each fingertip point, an initial speed-related dynamic threshold parameter is set. At the same time, based on the dynamic threshold parameter, the speed values ​​of the first three frames are multiplied by a preset coefficient and a basic offset value is added to obtain a fingertip touch judgment threshold table. Based on the fingertip touch judgment threshold table, multi-condition touch judgment analysis is performed on the fingertip speed feature dataset and the fingertip deceleration trend data. When the three conditions of fingertip deceleration trend being greater than a preset target value, angular velocity being greater than a first angular velocity threshold and instantaneous velocity being less than a second velocity threshold are met simultaneously, a touch trigger signal is obtained.

10. A fingertip extraction touch system based on convex hull algorithm, characterized in that, For performing the fingertip extraction touch method based on the convex hull algorithm as described in any one of claims 1-9, the fingertip extraction touch system based on the convex hull algorithm comprises: The layered sparse sampling module is used to acquire the original image of the hand and extract the layered sparse point set of the hand. The feature space mapping module is used to perform feature space mapping on the layered sparse point set of the hand to obtain the hand feature enhancement point set; A semi-periodic convex hull iteration module is used to generate a hand contour convex hull based on the hand feature enhancement point set; The point angle analysis module is used to perform point angle analysis on the convex hull of the hand contour to obtain a set of candidate points for the fingertips. The finger-interval relationship constraint analysis module is used to perform finger-interval relationship constraint analysis on the candidate fingertip point set of the hand to obtain the target fingertip point set of the hand; The touch detection module is used to perform critical angular velocity touch detection on the target point set of the hand fingertips and generate a touch trigger signal.

Images