A sign language handshape font generation method and a generation system thereof

Abstract

The application provides a sign language hand shape font generation method and system, and relates to the technical field of font generation, and comprises the following steps: constructing a Chinese sign language hand shape line drawing library; acquiring a hand shape picture and extracting hand key point coordinates, and calculating joint feature vectors; training a hand shape matching model based on the joint feature vectors, and outputting a hand shape label and a confidence degree for an uploaded hand shape picture; generating an uploaded hand shape picture line drawing; performing vectorization processing and path normalization processing on the generated line drawing to obtain a standard character shape contour; establishing a hand shape font mapping table; and generating a font file through a font construction engine. The application integrates hand shape recognition, line drawing generation, coding mapping and font construction, realizes uniform and expandable generation of line drawing character shape style, reduces the cost of manual drawing, improves the font generation efficiency, and improves the usability and consistency in corpus annotation, teaching and typesetting retrieval scenes.

Description

Technical Field

[0001] This invention relates to the field of font generation technology, and more specifically, to a method and system for generating sign language hand shape fonts. Background Technology

[0002] Sign language is a natural language used by the hearing impaired to communicate, and hand shapes are one of its core elements. In applications such as sign language teaching, dictionary compilation, and corpus annotation and recognition, it is often necessary to record and present hand shapes in a symbolic way. Compared to directly using images or video clips, using "fonts" to standardize the encoding and layout of hand shapes can significantly improve document editing efficiency and retrieval and reuse capabilities. Therefore, sign language hand shape fonts have significant application value. Furthermore, to facilitate clear display across font sizes, maintain a consistent style, and support vectorized editing and layout, in practical applications, hand shape fonts are often designed as line art fonts composed of outlines, using contour lines to express the hand shape structure.

[0003] The inventors of this application discovered in their research that existing textual representations of sign language hand shapes typically rely on sign language symbol systems or graphical annotation methods. These include manual selection and splicing based on pre-made graphic libraries, manually tracing hand shape line drawings in vector drawing software, or manually inputting and mapping hand shapes into font editing software to generate font files. These methods generally suffer from the following shortcomings: 1. The generation of hand shape fonts is highly dependent on manual drawing, requiring a high level of sign language knowledge and graphic design skills from the creator; 2. Hand shape materials from different sources are difficult to maintain consistency in outline style, stroke thickness, and proportions, resulting in inconsistent overall visual appeal of the line drawing font; 3. The number and types of hand shapes in the graphic library are insufficient to cover commonly used hand shape variations and fine-grained differences in different sign languages, leading to situations where different hand shapes are forced to be mapped to the same font in actual use, resulting in insufficient differentiation in hand shape expression. Summary of the Invention

[0004] To address the aforementioned problems, the present invention aims to provide a method and system for generating sign language hand shape fonts, which solves the problems of high generation costs, inconsistent styles, limited number and type coverage of hand shapes in the graphic library, and difficulty in expansion of existing hand shape fonts.

[0005] According to some embodiments, the present invention adopts the following technical solutions: A method for generating sign language hand gesture fonts, comprising: Step S1: Construct a Chinese Sign Language hand gesture line drawing library; Step S2: Extract the features of a single hand joint; Step S3: Train the hand shape matching model using joint features; Step S4: Obtain the uploaded hand image; Step S5: Extract joint feature data from the uploaded image; Step S6: Input the joint feature data extracted in step S5 into the matching model svm_model, and output the corresponding hand shape label and matching confidence. Step S7: Generate the corresponding hand shape line drawing based on the hand shape image and the output result of step S6, and determine the Unicode Private Area (PUA) encoding corresponding to the target line drawing; Step S8: Vectorize the line drawing obtained in step S7 to generate glyph outline paths, thereby obtaining standard glyph outlines that can be used in font files; Step S9: Based on the PUA encoding output in step S7 and the glyph outline path output in step S8, establish a font mapping table, which includes hand-shaped line drawing labels and PUA encoding values; Step S10: Import the glyph vector outlines obtained in step S8 and the character encoding mapping obtained in step S9 into the font building engine. Create glyph slots for each glyph according to the PUA encoding and write the vector outlines to generate a font file TTF. At the same time, export the CSV font mapping table.

[0006] Further, step S1 includes: Step S11: Based on the statistical analysis of Chinese sign language corpus, 64 sets of sign language hand shapes are obtained; Step S12: Draw a corresponding hand shape line drawing template for each hand shape and preset PUA encoding for each line drawing.

[0007] Further, step S2 includes: Step S21: Using MediaPipe, detect 21 key points of the hand in a single hand image, specifically including: wrist joint coordinates, thumb carpal joint coordinates, thumb metacarpophalangeal joint coordinates, thumb interphalangeal joint coordinates, thumb phalangeal joint coordinates, index finger metacarpophalangeal joint coordinates, index finger proximal interphalangeal joint coordinates, index finger distal interphalangeal joint coordinates, index finger phalangeal joint coordinates, middle finger metacarpophalangeal joint coordinates, middle finger proximal interphalangeal joint coordinates, middle finger distal interphalangeal joint coordinates, middle finger phalangeal joint coordinates, ring finger metacarpophalangeal joint coordinates, ring finger proximal interphalangeal joint coordinates, ring finger distal interphalangeal joint coordinates, ring finger phalangeal joint coordinates, little finger metacarpophalangeal joint coordinates, little finger proximal interphalangeal joint coordinates, little finger distal interphalangeal joint coordinates, and little finger phalangeal joint coordinates. This yields a set of hand joint coordinates. Where i represents the finger number (i=0 for thumb, i=1,2,3,4 for index finger, middle finger, ring finger, and little finger respectively), and j represents the joint number on the same finger. Let be the coordinates of the joint in three-dimensional space; Step S22: Scale the joint coordinates to obtain normalized joints. ; Step S23: Select three joint angles for each finger as joint angle features and record them as follows. ,in Let j represent the j-th joint angle of finger i, and let j' be the angle of the adjacent joints of this joint. i,j-i With j' i,j+i Two finger bone vectors a i,j =j' i,j -j' i,j-i b i,j =j' i,j+1 -j' i,j Then the joint angle ; Step S24: Extract the angle features between fingers. Specifically, the feature vector between fingers consists of the angle between the tips of two adjacent fingers, denoted as... ,in, This represents the fingertip of finger i and finger i+1. The included angle is calculated by taking the direction vectors of the proximal phalanges of two adjacent fingers. The angle between the fingertips is obtained, that is ; Step S25: Concatenate the joint angles and finger angles in sequence to obtain the feature vector F, where F is a 19-dimensional vector. .

[0008] Further, step S3 includes: Step S31: Using the 64 hand-shaped labels from step S1 as category labels, collect sample images for each category and perform step S2 to extract feature vectors, forming a training sample set. ,in Hand-shaped label; Step S32: In Python, call sklearn's SVM to train the classification model and obtain the hand shape matching model svm_model; Step S33: Divide the dataset into training and testing sets, obtain the test accuracy (precision) through prediction, and adjust the penalty parameter C and kernel function coefficients based on the precision. By combining the maximum number of iterations (max_iter), the final matching model (svm_model) is obtained. After training, the model is saved to the models folder.

[0009] Furthermore, step S7 is divided into two cases, including step S71 and step S72: Step S71: When the hand shape label output in step S6 is a known category label in the hand shape line drawing library, and the model matching confidence is greater than a preset threshold, obtain the corresponding hand shape line drawing template according to the index in the line drawing library constructed in step S1 based on the hand shape label, read the PUA code bound to the line drawing template, and output the target line drawing and its PUA code. Step S72: When the matching confidence score output in step S6 is less than a preset threshold, the DexiNed edge detection network is invoked to extract edges from the hand image in step S4 to generate a hand outline line drawing. An unused PUA is allocated from the preset encoding pool as the encoding for the line drawing. The target line drawing and its PUA encoding are then output, specifically including: (1) Input normalization: Let the input hand shape image be... The input is obtained after normalization. ,in and The channel mean and standard deviation; (2) DexiNed multi-sided output: The DexiNed network is represented as ,right Inference yielded K levels of edge response maps: ; (3) Edge fusion: The multi-sided outputs are weighted and fused to obtain the final edge probability map E: ,in, For the Sigmoid function, To integrate weights, For the bias term, E is resampled to the original resolution: ; (4) Thresholding to generate binary line art: ,in For edge threshold, For indicator functions; (5) Denoising and Closure Repair: Perform B-opening operation for denoising and closing operation for closure repair to obtain the repaired line drawing B*: ,in These represent corrosion and expansion, respectively. This serves as the structural element. The target outline line drawing B* is thus obtained; and temporary hand-shaped labels are generated for the DexiNed network line drawing, with the newly added labels and assigned PUA codes written into the font mapping table.

[0010] Further, the vectorization process in step S8 includes: Step S81: Threshold the line drawing to obtain a binary line drawing, and perform connected component denoising and break repair. Step S82: Obtain the path point sequence using contour tracking. ; Step S83: Perform curve fitting on the path point sequence to generate a character outline path composed of Bézier curves. ; Step S84 involves performing closure verification, smoothing, path simplification, elimination of self-intersection and overlap, and unification of character frame size and character center alignment on the character outline path to obtain a standard character outline that can be written to a font file.

[0011] The font building engine in step S10 is implemented using the Python interface of FontForge.

[0012] According to some embodiments, the present invention adopts the following technical solutions: A sign language hand gesture font generation system, comprising: The image acquisition module is used to acquire hand-shaped images to be processed. The sources of the hand-shaped images include camera frames and user-uploaded images. The joint feature extraction module is used to detect key points in hand images and extract joint feature vectors. The hand shape matching module is used to input the joint feature vector of the uploaded hand shape image into the matching model and output the corresponding hand shape label and matching confidence score; The line drawing generation module is used to generate a target hand shape line drawing based on the hand shape image and the output of the hand shape matching module. The line drawing generation module includes: (1) Line drawing library retrieval module, used to obtain and output the corresponding line drawing template of the hand line drawing library according to the hand tag index when the hand tag is a known category of the hand line drawing library and the matching confidence is not lower than the preset threshold; (2) Edge detection generation module, which is used to call the DexiNed edge detection network to extract the edges of the hand shape image when the matching confidence is lower than the preset threshold, generate an edge probability map, and obtain the hand shape outline line drawing after thresholding, denoising and closure repair processing and output it; The encoding mapping module is used to establish a font mapping table by combining the preset PUA encoding obtained by the line art library retrieval module and the PUA encoding assigned by the edge detection generation module. The font generation module imports the standardized glyph vector outline font mapping table into the font building engine, generates a font file TTF, and exports a CSV font mapping table.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: Achieve an integrated automated workflow for hand-shaped font generation: integrate "hand joint feature extraction - hand shape matching - line drawing generation - vectorization - encoding mapping - font export" into a complete chain, reduce manual drawing and manual import steps, and significantly improve the efficiency of hand-shaped font construction;

[0014] Improve the consistency of line art fonts: Through standardized line art generation and vectorization processing, achieve consistency in font outlines, stroke thickness, scale proportions, and centering position, improve the clarity of display across font sizes and the overall visual experience, and facilitate scheduling and long-term maintenance;

[0015] It balances template reuse and automatic generation capabilities: when the matching confidence is high, it can directly index the hand shape line drawing library templates for quick output; when the matching confidence is low, it uses the DexiNed edge detection network to generate outline line drawings, reducing the dependence on the size of the pre-made graphic library and enhancing the coverage of hand shape variants and newly added hand shapes. Attached Figure Description The accompanying drawings, which constitute a part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention. Figure 1 This is a schematic diagram of Chinese sign language hand shape line drawing and preset Unicode encoding provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of joint feature extraction provided in an embodiment of the present invention. Figure 3 This is a flowchart illustrating a method for generating sign language hand gesture fonts according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the structure of a sign language hand shape font generation system provided in an embodiment of the present invention.

Claims

1. A method for generating sign language hand gesture fonts, characterized in that, include: Constructing a Chinese Sign Language hand gesture line drawing library; Extract features from individual hand joints; A hand shape matching model is trained using joint features; Get the uploaded hand image; Extract joint feature data from uploaded images; Input the joint feature data into the matching model, and output the corresponding hand shape label and matching confidence score; Generate a corresponding hand shape line drawing based on the hand shape image and matching tags, and determine the Unicode Private Area (PUA) encoding corresponding to the target line drawing; Vectorize the line art to generate glyph outline paths, thereby obtaining standard glyph outlines that can be used in font files; A font mapping table is established by matching the PUA encoding of the line drawing with the glyph outline path. The mapping table includes the hand-shaped line drawing label and the PUA encoding value. Font files are generated using a font building engine.

2. A sign language hand gesture font generation system, characterized in that, include: The image acquisition module is used to acquire hand-shaped images to be processed. The sources of the hand-shaped images include camera frames and user-uploaded images. The joint feature extraction module is used to detect key points in hand images and extract joint feature vectors. The hand shape matching module is used to input the joint feature vector of the uploaded hand shape image into the matching model and output the corresponding hand shape label and matching confidence score; The line drawing generation module is used to generate a target hand shape line drawing based on the hand shape image and the output of the hand shape matching module. The line drawing generation module includes: (1) Line drawing library retrieval module, used to obtain and output the corresponding line drawing template of the hand line drawing library according to the hand tag index when the hand tag is a known category of the hand line drawing library and the matching confidence is not lower than the preset threshold; (2) Edge detection generation module, which is used to call the DexiNed edge detection network to extract the edges of the hand shape image when the matching confidence is lower than the preset threshold, generate an edge probability map, and obtain the hand shape outline line drawing after thresholding, denoising and closure repair processing and output it; The encoding mapping module is used to establish a font mapping table by combining the preset PUA encoding obtained by the line art library retrieval module and the PUA encoding assigned by the edge detection generation module. The font generation module imports the standardized glyph vector outline font mapping table into the font building engine, generates a font file TTF, and exports a CSV font mapping table.

Images