A mathematical and logical system for recognizing handwritten symbols based on differential geometry topological graph structure

Abstract

This invention discloses a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure, comprising: a trajectory recovery module, used to generate a continuous arc-length parameterized curve γ(s) from an input static image of a handwritten symbol or a standard printed symbol image through a trajectory recovery network based on manifold regularization; and a differential geometric parameterization module, used to calculate the Frenet–Serret active frame of the arc-length parameterized curve γ(s), extract the tangent vector T(s), normal vector N(s), curvature κ(s), and torsion τ(s), and combine it with the logarithmic velocity profile v(s) to generate an intrinsic feature curve F(s) with translation, rotation, and scale invariance. This invention's mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure only needs to match the character set of the standard answer to the current question, greatly eliminating inter-class confusion in traditional methods and significantly reducing the recognition difficulty; the differential geometric intrinsic features eliminate the influence of writing style, displacement, and rotation; and the elasticity measure tolerates stroke breaks and deformation.

Description

Technical Field

[0001] This invention belongs to the field of intelligent recognition technology, specifically relating to a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure. Background Technology

[0002] Handwritten character recognition is an important research direction in the field of pattern recognition, with wide application needs in scenarios such as educational informatization, intelligent grading, and human-computer interaction. Currently, handwritten character recognition mainly falls into two technical categories: The first category is static image-based recognition methods, such as using convolutional neural networks (CNNs) or Transformer models to directly classify handwritten character images. These methods require massive amounts of labeled training samples and are particularly sensitive to mathematical symbols (such as the integral sign ∫, partial differential symbols, etc.). The ability to distinguish between characters with similar shapes (such as the summation symbol ∑) and similar-looking characters (such as the Greek letter α and the Latin letter a, the number 2 and the letter z) is insufficient. Furthermore, this type of method only utilizes static image information of the characters and cannot take advantage of dynamic information during the writing process; therefore, the recognition accuracy is limited by changes in image quality and font style.

[0003] The second category is online recognition methods, which require collecting real-time coordinate sequences during writing (such as obtaining handwriting trajectories through a graphics tablet or touchscreen) and using temporal information for recognition. However, this type of method cannot process pre-written static handwritten images (such as scanned copies of paper exam papers or photographed answer sheets), severely limiting its application scenarios and failing to meet the actual needs of large-scale automatic grading in online education.

[0004] There are still some special difficulties in recognizing mathematical, physical, and chemical symbols: mathematical symbols (such as ∫, ... 、▽) are often composed of continuous curves, which are difficult to describe with simple stroke models; some symbols contain complex topological relationships such as intersection, closed loop, and nesting (such as θ, ∞, ∑); the handwriting styles of different writers vary greatly, with natural variations such as cursive, simplification, trembling, and broken strokes; mathematical and physical symbols contain hundreds of different symbols, and the symbol systems used in different subjects and different textbooks are different.

[0005] Existing technologies cannot achieve high-precision closed-set symbol recognition given only printed standard answers and a small sample of handwritten images.

[0006] Therefore, to address the aforementioned technical problems, it is necessary to provide a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure. Summary of the Invention

[0007] The purpose of this invention is to provide a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure. It regards characters as projections of writing actions onto manifolds. By extracting a unified morphological dynamic signature (MDS) from differential geometric space, Lie group action primitive space and graph topological space, and performing cross-space elastic measurement on printed standard symbols and student handwritten symbols, it achieves accurate recognition within a very small candidate set.

[0008] To achieve the above objectives, a specific embodiment of the present invention provides the following technical solution: A mathematical and physical symbol recognition system based on differential geometric topological graph structure includes: The trajectory recovery module is used to generate a continuous arc length parameterized curve γ(s) from the input static image of handwritten symbols or standard printed symbols through a trajectory recovery network based on manifold regularization. The differential geometry parameterization module is used to calculate the Frenet–Serret active frame of the arc length parameterized curve γ(s), extract the tangent vector T(s), normal vector N(s), curvature κ(s) and torsion τ(s), and combine it with the logarithmic velocity profile v(s) to generate the intrinsic characteristic curve F(s) with translation, rotation and scale invariance. The action primitive decomposition module is used to automatically divide the arc length parameterized curve γ(s) into several action primitive segments based on the Lie group parameterization method and using a switching linear dynamic system, and to estimate the corresponding Lie algebra parameters for each primitive segment, and output a structured representation Γ containing primitive labels, Lie algebra parameters and arc length information. The topology graph embedding module is used to construct temporal adjacency edges and spatial interaction edges using action primitive segments as nodes, and combines the continuous cohomology features of the trajectory to encode them into a fixed-length graph embedding vector E through a graph isomorphic network. G ; The matching and recognition module is used to calculate the shape space distance, primitive sequence distance and topological similarity between the handwritten symbol to be recognized and the standard symbol image in the multi-manifold space, and to fuse the matching probability through the Siamese metric network to determine the recognition result.

[0009] In one or more embodiments of the present invention, the manifold-regularized trajectory recovery network in the trajectory recovery module includes: The Vision Transformer encoder is used to map character images into latent vectors; A continuous-time RNN decoder is used to generate time-series trajectory points; Differentiable rasterization layers are used to ensure that the rendered trajectory is consistent with the original image; And a trajectory smoothing regularization term, used to recover the arc length parameterized curve γ(s).

[0010] In one or more embodiments of the present invention, the differential geometric parameterization module calculates the torsion τ(s) by supplementing the virtual pressure dimension, and the intrinsic characteristic curve F(s) is represented as F(s)=[T(s),κ(s),τ(s),v(s)], where T(s) is the unit tangent vector and v(s) is the logarithmic velocity profile.

[0011] In one or more embodiments of the present invention, the predefined action primitives in the action primitive decomposition module include: straight line segments, circular arcs, sharp corners, S-curves, elliptical arcs, hooks, spiral segments, sudden stop points, and wavy lines. Each type of primitive corresponds to a single-parameter subgroup on the SE(2) group or the Aff(2) group. The Lie algebra parameter ξ is used to describe the continuous deformation of the primitive segment.

[0012] In one or more embodiments of the present invention, the structured representation Γ includes the label, Lie algebra parameters and arc length information of each primitive segment in the primitive sequence, as well as the SE(2) relative transformation matrix between adjacent primitive segments.

[0013] In one or more embodiments of the present invention, in the topology graph embedding module, temporally adjacent edges carry relative pose information between adjacent primitive segments, and spatially interactive edges carry distance and intersection information between primitive segments; the persistent homology feature is the persistent landscape vector PH corresponding to the 1D persistent homology graph. vec The graph isomorphic network is a GIN network.

[0014] In one or more embodiments of the present invention, in the matching and recognition module, the shape space distance is calculated based on the square root velocity function to determine the geodesic distance d in the elastic shape space. shape ; The primitive sequence distance is calculated using structured dynamic time warping to determine the sequence alignment cost, and primitive merging and splitting are allowed.

[0015] In one or more embodiments of the present invention, the standard symbol profile is a morphological dynamic signature generated based on the printed standard answer, which includes a unified character profile containing differential geometric curves, action primitive sequences and topological graph embeddings, and the candidate set of the handwritten symbols to be identified is limited to the standard answer character set corresponding to the current question.

[0016] In one or more embodiments of the present invention, a synthetic data engine and a domain adaptation module are also included. The synthetic data engine transforms standard trajectories into handwritten samples of various styles through a handwriting style transfer network, and the domain adaptation module fine-tunes system parameters using real handwritten samples.

[0017] A method for recognizing handwritten mathematical and physical symbols based on differential geometric topological graph structures includes the following steps: S1: Obtain the input handwritten symbol image or printed standard symbol image, and generate a continuous arc length parameterized curve γ(s) through a trajectory recovery network based on manifold regularization; S2: Calculate the Frenet–Serret active frame of the arc length parameterized curve γ(s), extract the tangent vector T(s), normal vector N(s), curvature κ(s) and torsion τ(s), and combine it with the logarithmic velocity profile v(s) to generate the intrinsic characteristic curve F(s) with translation, rotation and scale invariance. S3: Based on the Lie group parameterization method, the arc length parameterized curve γ(s) is automatically divided into several action primitive segments by using a switching linear dynamic system. The corresponding Lie algebra parameters are estimated for each primitive segment, and a structured representation Γ containing primitive labels, Lie algebra parameters and arc length information is output. S4: Using action primitives as nodes, construct temporal adjacency edges and spatial interaction edges. Combined with the continuous cohomology features of the trajectory, encode them into a fixed-length graph embedding vector E through a graph isomorphic network. G This forms a dynamic signature. S5: Calculate the shape space distance, primitive sequence distance, and topological similarity between the handwritten symbol to be identified and the standard symbol image in the multi-manifold space, and fuse the matching probability through the Siamese metric network to determine the recognition result.

[0018] Compared with the prior art, the present invention has the following advantages: The handwritten symbol recognition system for mathematics, physics, and chemistry based on differential geometry topological graph structure of this invention only needs to be matched with the character set of the standard answer to the current question, which greatly eliminates inter-class confusion in traditional methods and significantly reduces the recognition difficulty. The intrinsic features of differential geometry eliminate the influence of writing style, displacement, and rotation; the elasticity measure tolerates stroke connection and deformation; topological verification ensures the differentiation of structural ambiguities; it has an inherent fault-tolerant mechanism for natural writing variations such as cursive, simplification, and trembling, and can adapt to the personalized writing habits of different writers; only one printed answer is needed to generate all standard images, without the need for a large amount of real handwritten training data, which greatly reduces the system deployment cost. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is a system overall framework diagram of a mathematical physics and chemistry handwritten symbol recognition system based on differential geometric topological graph structure in one embodiment of the present invention; Figure 2 This is a structural diagram of TrajNet++, a trajectory recovery network for a mathematical and physical handwritten symbol recognition system based on differential geometric topology graph structure, according to one embodiment of the present invention. Figure 3 This is a schematic diagram of the active frame extraction and intrinsic feature curve of a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure in one embodiment of the present invention (for the ∫ symbol). Figure 4 This is a topological relationship diagram and a flowchart of continuous homology calculation for a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure, according to one embodiment of the present invention. Figure 5 This is a schematic diagram of the SRVF-based elastic shape matching principle of a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure in one embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0022] like Figures 1-5 As shown, an embodiment of the present invention provides a mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure, comprising two parts: an offline stage and an online stage.

[0023] Offline stage: The printed standard answer image is input into the system and passes through the trajectory recovery module, differential geometry parameterization module, action primitive decomposition module and topology graph embedding module in sequence to generate the morphological dynamic signature (MDS) of the standard symbol and store it in the standard image library.

[0024] Online phase: The system inputs images of students' handwritten characters, generates MDS of students' handwritten characters through the same processing flow, and then matches them in the candidate set of the standard image library, i.e., the standard answer character set of the current question, through the matching and recognition module, and outputs the recognition result.

[0025] The entire system adopts a modular design, and the modules communicate with each other through standardized data interfaces, which facilitates independent optimization and replacement.

[0026] The trajectory recovery module is used to convert static character images into continuous arc-length parameterized curves γ(s), providing input for subsequent differential geometric analysis.

[0027] In some implementations, such as Figure 2 As shown, the manifold regularization-based trajectory recovery network (hereinafter referred to as TrajNet++) in the trajectory recovery module includes the following components: The Vision Transformer encoder is used to map an input character image into a latent vector.

[0028] Specifically: the input image is divided into several image patches, each preferably 16×16 pixels in size; a linear projection is performed on each image patch to obtain an image patch embedding vector; a class token is added before the image patch embedding sequence, and position encoding is added; a Transformer encoder consisting of a multi-head self-attention layer and a feedforward network layer outputs a latent vector Z containing global image information.

[0029] A continuous-time RNN decoder is used to receive the latent vector Z and generate a time-series trajectory point sequence.

[0030] Specifically: using the latent vector Z as the initial state, trajectory point coordinates are recursively generated through gated recurrent units or long short-term memory networks; the continuous-time RNN decoder employs a continuous-time neural differential equation framework, expanding the discrete time step into a continuous arc length parameter, thereby generating trajectory point coordinates at arbitrary arc length resolution; the trajectory point sequence output by the decoder is represented as {(x i ,y i )}, where i=1,2,...,N, and N is the number of trajectory points.

[0031] Differentiable rasterization layers are used to render the generated trajectory point sequence into a binary image and ensure that the rendered trajectory is consistent with the original input image.

[0032] Specifically: Differentiable B-spline interpolation is used to fit discrete trajectory points into a smooth curve; the curve is rendered into an image using a differentiable rasterization operator; the pixel-level difference between the rendered image and the original input image is calculated as the reconstruction loss; the gradient of the reconstruction loss is passed back to the decoder and encoder through backpropagation to achieve end-to-end training.

[0033] The trajectory smoothing regularization term is used to recover the smooth arc-length parameterized curve γ(s).

[0034] Specifically: a curvature square integral regularization term is added to the loss function to penalize the overly curved trajectory; a writing speed prior is introduced, assuming that the writing speed changes smoothly between adjacent points; the arc length parameterized curve γ(s) satisfies: γ(s)=(x(s),y(s)), where s is the arc length parameter, and ||dγ / ds||=1.

[0035] The training of TrajNet++ can employ a strategy that combines synthetic and real data. First, a synthetic data engine is used to generate a large-scale training sample, and then a small number of real handwritten samples are used for fine-tuning. The synthetic data engine, through a handwriting style transfer network, transforms standard trajectories into handwritten samples of various styles, covering different noise levels and deformation patterns.

[0036] The differential geometry parameterization module is used to extract intrinsic geometric features with translation, rotation and scale invariance from the arc-length parameterized curve γ(s).

[0037] In some implementations, the Frenet–Serret active frame for calculating the arc length parameterized curve γ(s) specifically includes: Tangent vector T(s): T(s)=γ'(s)=dγ / ds. Since arc length parameterization is used, ||T(s)||=1 always holds true.

[0038] Normal vector N(s): N(s) = T'(s) / ||T'(s)||, well defined when curvature κ(s) ≠ 0.

[0039] Curvature κ(s): κ(s) = ||T'(s)|| = ||γ''(s)||, which describes the degree of curvature of the curve at the current position.

[0040] Deflection τ(s): For planar curves, the original trajectory does not contain deflection information. This invention calculates the deflection τ(s) by supplementing the virtual pressure dimension.

[0041] Specifically, the two-dimensional trajectory γ(s)=(x(s),y(s)) is upgraded to a three-dimensional curve γ(s)=(x(s),y(s),p(s)), where p(s) is the virtual pressure dimension, which is determined in the following ways: the pressure dimension is modeled according to the pressure change pattern of the pen tip on the paper during the writing process; or the distribution of the pressure dimension is estimated from the training data through learning.

[0042] In some implementations, the intrinsic characteristic curve F(s) is represented as F(s)=[T(s),κ(s),τ(s),v(s)], where v(s) is the logarithmic velocity profile.

[0043] Specifically, the logarithmic velocity profile v(s) = log(||dγ / dt||) reflects the variation of writing speed at various positions on the trajectory. Incorporating velocity information into the intrinsic features makes the representation include not only shape information but also writing dynamic information, thus improving the ability to distinguish between similar-looking symbols.

[0044] The intrinsic characteristic curve F(s) has the following invariance: Translation invariance: The expression for F(s) does not depend on the absolute position of the trajectory in the plane; Rotation invariance: The expression for F(s) does not depend on the global rotation angle of the trajectory; Scale invariance: The expression for F(s) does not depend on the global scaling of the trajectory.

[0045] The aforementioned invariance is achieved by using arc length parameterization and the geometric nature of the Frenet–Serret frame, without requiring additional normalization steps.

[0046] The Action Primitive Decomposition module is used to decompose a continuous trajectory into a finite number of action primitive segments and estimate Lie algebra parameters for each segment.

[0047] In some implementations, the predefined motion primitives in the motion primitive decomposition module include the following nine categories: straight line segments, circular arcs, sharp corners, S-curves, elliptical arcs, hooks, spiral segments, emergency stop points, and wavy lines.

[0048] Each primitive corresponds to a single-parameter subgroup on the SE(2) group or the Aff(2) group: Line segment: corresponds to the translation subgroup on the SE(2) group, with Lie algebra parameters being the translation velocity and direction; Arc: Corresponds to the rotational subgroup on the SE(2) group, with Lie algebra parameters being the rotational velocity and radius of curvature; Sharp angle: corresponds to the splicing of two line segments, and the Lie algebra parameters are the included angle and the lengths of both sides; S-shaped curve: corresponds to the splicing of two reverse circular arcs, with Lie algebra parameters being the curvature of the two segments and the position of the transition point; Elliptical arc: corresponds to an anisotropic scaling + rotation subgroup on the Aff(2) group; Back hook: corresponds to the composite motion on the SE(2) group, which includes circular arc segments and straight line segments; Helical segment: Corresponds to helical motion on the SE(2) group, where curvature and torsion change simultaneously; Emergency stop point: corresponds to an isolated point where the velocity is zero, and the Lie algebra parameter is the dwell time; Wavy lines: correspond to periodic bending motion, with Lie algebra parameters being amplitude and frequency.

[0049] In some implementations, a Switching Linear Dynamical System (SLDS) is used to automatically divide the arc-length parameterized curve γ(s) into several action primitive segments.

[0050] Specifically, SLDS includes the following elements: Continuous state variables: current position, velocity, and acceleration; Discrete switching variable: the current primitive category; State transition matrix: Each primitive category corresponds to a linear dynamical system, describing the state evolution within that primitive; Switching logic: Based on Bayesian information criteria or variational inference, primitive switching is triggered when the state prediction error exceeds a threshold.

[0051] The segmentation process of SLDS does not require manual annotation. It automatically learns primitive categories and switching boundaries from trajectory data through the expectation-maximization algorithm.

[0052] In some implementations, the corresponding Lie algebra parameter ξ is estimated for each primitive segment. Specifically: For a basic element in the SE(2) group, the Lie algebra parameter ξ = (v x ,v y ,ω), where (v x ,v y ω is the translational velocity, and ω is the angular velocity. For a basic element in the group Aff(2), the Lie algebra parameter ξ = (v x ,v y ,ω,α,β), where α and β are anisotropic scaling parameters; Lie algebra parameters are estimated from trajectory data using least squares fitting or Kalman filtering.

[0053] In some implementations, the structured representation Γ output by the action primitive decomposition module includes: Γ=[(label1,ξ1,I1),(label2,ξ2,I2),...,(label M ,ξ M ,I M )] Where: label i ξ is the category label for the i-th primitive segment; i Let I be the Lie algebra parameter of the i-th primitive segment; i Let be the arc length of the i-th primitive segment; M is the total number of primitive segments.

[0054] Furthermore, the structured representation Γ also includes the SE(2) relative transformation matrix R between adjacent primitive segments. i, represents the relative pose transformation from the end point of the i-th primitive segment to the start point of the (i+1)-th primitive segment, i.e.: R i =[cosθ i ,-sinθ i ,t {x,i} ;sinθ i cosθ i ,t {y,i} ;0,0,1] Where θ i For the relative rotation angle, (t) {x,i} ,t {y,i} ) represents the relative translation amount.

[0055] The topology graph embedding module is used to convert action primitive sequences into graph structures and extract graph topology features.

[0056] In some implementations, an attribute graph G=(V,E) is constructed using action primitive segments as nodes, where: Each node v i ∈V corresponds to an action primitive segment, and the node features include: primitive class label (one-hot encoded); Lie algebra parameter ξ. i Arc length of the basic segment I i ; the average curvature and torsion of the primitive segment.

[0057] Construct two types of edges: Temporal adjacency edge: connects temporally adjacent primitive segments (v i ,v i+1 ), carrying relative pose information R between adjacent primitive segments i ; Spatial interaction edge: Connects primitive segments that interact in space. The criteria for judgment include: the minimum Euclidean distance between the two primitive segments is less than a preset threshold; the two primitive segments intersect or overlap in space; the two primitive segments enclose a closed area.

[0058] Spatial interaction edges carry distance and intersection information between primitive segments, specifically including minimum distance, intersection coordinates, and intersection angle.

[0059] In some implementations, calculating the persistent cohomology features of the trajectories specifically includes: treating the trajectory point set as a point cloud in metric space, constructing Vietoris–Rips complexes or α-complexes at different scales; calculating a 1D persistent cohomology map to record the birth and death times of closed-loop structures (i.e., 1D holes); and converting the persistent cohomology map into a persistent landscape vector PH. vec Specifically: for each persistent homology feature (loop closure), it is represented as a triangular function; the envelope of all triangular functions is taken to obtain the hierarchical function of the persistent landscape; the hierarchical function is discretized into a fixed-length vector PH.vec .

[0060] Continuous homology features can effectively characterize the topological structures of characters, such as closed loops and intersections, and play an important role in distinguishing similar symbols with different topological properties, such as the number 0 and the letter O.

[0061] In some implementations, the attribute graph G is encoded into a fixed-length graph embedding vector E using a Graph Isomorphism Network (GIN). G .

[0062] Specifically, the GIN network uses the following update rules:

[0063] in: The feature representation of node v at the k-th layer; For a multilayer perceptron of the k-th layer; is a learnable parameter; N(v) is the set of neighboring nodes of node v.

[0064] After K layers of iteration, global pooling (such as summation pooling or average pooling) aggregates the features of all nodes into a fixed-length graph embedding vector E. G .

[0065] The matching and recognition module is used to calculate the similarity between the handwritten symbol to be recognized and the standard symbol image in a multi-manifold space.

[0066] In some implementations, the shape space distance is calculated on the elastic shape space based on the square root velocity function (SRVF) to determine the geodesic distance d. shape .

[0067] Specifically: For an arc length parameterized curve γ(s), its SRVF is defined as q(s) = γ'(s) / √||γ'(s)||. SRVF means mapping a curve to a Hilbert space in which the reparameterization of the curve corresponds to the group action; The shape distance between two curves is defined as the shortest geodesic distance in SRVF space after rotation, translation and reparameterization. The optimal reparameterized matching is solved using a dynamic programming algorithm to obtain the geodesic distance d. shape .

[0068] SRVF shape distance is robust to writing style distortion and stroke breakage, and can tolerate a certain degree of deformation.

[0069] In some implementations, the primitive sequence distance is calculated using Structured Dynamic Time Warping (S-DTW) to determine the sequence alignment cost.

[0070] Specifically, for two primitive sequences A=[a1,a2,...,a...] m ] and B=[b1,b2,...,b n ]: Define the local distance function d(a) between primitives. i ,b j Taking into account both the differences in primitive categories and the differences in Lie algebra parameters; The optimal alignment path is calculated using a dynamic time warping algorithm, which allows for one-to-one, one-to-many, and many-to-one matching of primitives. The S-DTW algorithm allows primitive merging and splitting when calculating alignment costs, meaning that one primitive can be aligned to multiple primitives, or multiple primitives can be aligned to one primitive. The cost of primitive merging and splitting is measured by the interpolation distance in the Lie algebra parameter space.

[0071] S-DTW sequence distance can handle differences in the number of primitives under different writing styles, such as primitive merging caused by cursive writing or primitive splitting caused by discontinuous writing.

[0072] In some implementations, topological similarity is based on graph embedding vectors E. G Calculation of cosine similarity or Euclidean distance between them: sim top =cosine similarity (E G student E G standard ) Topological similarity is used to verify the consistency of the topological structure between the symbol to be identified and the standard symbol, effectively excluding symbols that are similar in shape but have different topological structures.

[0073] In some implementations, the final matching probability is output by fusing shape space distance, primitive sequence distance, and topological similarity through a Siamese Metric Network.

[0074] Specifically, the Siamese metric network receives the following input features: Shape space distance d shape ; Primitive sequence distance d seq ; Topological similarity sim top ; Other auxiliary features, such as aspect ratio and number of strokes; The Siamese metric network consists of multiple fully connected layers and outputs matching probabilities in the range [0,1]. The twin metric network is trained using either contrastive loss or cross-entropy loss.

[0075] In some implementations, the standard symbol profile is a morphological dynamic signature (MDS) generated based on printed standard answers, which includes a unified character profile containing differential geometric curves, action primitive sequences, and topological graph embeddings.

[0076] Specifically, the construction process of the standard portrait library includes: Parse the printed standard answer in LaTeX or MathML into a sequence of symbols; For each symbol, after rendering it into an image, it undergoes trajectory recovery, differential geometry parameterization, action primitive decomposition, and topological graph embedding in sequence to generate an MDS signature; The MDS signature is stored in the standard image library, indexed by the corresponding Unicode code point or LaTeX command ID.

[0077] The candidate set of handwritten symbols to be recognized is limited to the character set of the standard answer corresponding to the current question, usually 20-80 symbols, rather than the entire symbol set. This minimal candidate set strategy greatly eliminates inter-class confusion in traditional methods and significantly improves recognition accuracy.

[0078] Based on the above modules, this invention provides a method for recognizing handwritten mathematical and physical symbols based on differential geometric topological graph structures, comprising the following steps: S1: Track Recovery Steps The input handwritten symbol image or printed standard symbol image is obtained, and a continuous arc length parameterized curve γ(s) is generated by a trajectory recovery network based on manifold regularization.

[0079] Specifically, the input image, preferably 256×256 pixels, is fed into the TrajNet++ network. The latent vector is extracted by the VisionTransformer encoder, and the temporal trajectory points are generated by the continuous-time RNN decoder. The reconstruction quality is verified by the differentiable rasterization layer, and finally the arc length parameterized curve γ(s) is output.

[0080] S2: Differential geometric parameterization steps The Frenet–Serret active frame for calculating the arc length parameterized curve γ(s) is used to extract the tangent vector T(s), normal vector N(s), curvature κ(s), and torsion τ(s). Combined with the logarithmic velocity profile v(s), an intrinsic characteristic curve F(s)=[T(s),κ(s),τ(s),v(s)] with translation, rotation, and scale invariance is generated.

[0081] S3: Action Element Decomposition Steps Based on the Lie group parameterization method, the arc length parameterized curve γ(s) is automatically divided into several action primitive segments by using a switching linear dynamic system. The corresponding Lie algebra parameter ξ is estimated for each primitive segment, and the structured representation Γ containing primitive labels, Lie algebra parameters and arc length information is output.

[0082] Specifically, the structured representation Γ=[(label1,ξ1,I1),(label2,ξ2,I2),...,(label M ,ξ M ,I M )], and includes the SE(2) relative transformation matrix between adjacent primitive segments.

[0083] S4: Topology Graph Embedding Steps Using action primitives as nodes, temporal adjacency edges and spatial interaction edges are constructed. Combined with the continuous cohomology features of the trajectory, they are encoded into a fixed-length graph embedding vector E through a graph isomorphic network. G This forms a morphological dynamic signature (MDS).

[0084] S5: Matching and Recognition Steps In a multi-manifold space, the shape space distance (based on SRVF), primitive sequence distance (based on S-DTW), and topological similarity between the handwritten symbol to be identified and the standard symbol image are calculated. The matching probability is then fused through a Siamese metric network to determine the recognition result.

[0085] Example 1: Generation of Standard Printed Symbol Images This embodiment describes how to generate a standard symbol image from a printed standard answer.

[0086] Input: A LaTeX rendered image of the problem “Calculate the integral ∫x²dx”.

[0087] Step 1: Character splitting. Through LaTeX syntax tree parsing, the symbol set {∫, x, ...} is obtained. 2 Each symbol has a clear semantic label and Unicode code point.

[0088] Step 2: Trajectory Recovery. Each symbol is rendered as a 256×256 pixel grayscale image and fed into the TrajNet++ network. Taking the integral symbol ∫ as an example, TrajNet++ outputs a standard writing trajectory, represented as an arc extending from the upper left to the lower right, ending with a rightward hook. The trajectory is represented in arc-length parameterized form to ensure that the trajectory of each symbol has a uniform arc-length resolution.

[0089] Step 3: Differential Geometric Feature Extraction. The Frenet–Serret active frame for each symbol trajectory is calculated sequentially. For the integral symbol ∫, its trajectory features are: the curvature of the arc segment is large and its direction changes continuously, while the curvature direction of the hook segment reverses. The generated intrinsic feature curve F(s) records the tangent vector, curvature, torsion, and logarithmic velocity profile at each point along the trajectory.

[0090] Step 4: Action Element Decomposition. A switching linear dynamics system is used to automatically segment the trajectory. The trajectory of the integral sign ∫ is decomposed into two action element segments: the first segment is an "S-curve," corresponding to the main arc, and the second segment is a "left hook," corresponding to the final hook. The Lie algebra parameters of each segment are accurately estimated.

[0091] Step 5: Topology Graph Construction. An attribute graph is constructed using two primitive segments as nodes. Temporally adjacent edges connect the two segments, carrying relative pose transformations. Spatial interaction edges are empty in this example, and there is no closed-loop structure. Continuous cohomology analysis confirms the absence of 1D closed loops. The GIN network encodes the attribute graph into graph embedding vectors.

[0092] Step 6: Image Storage. Integrate all the above information—intrinsic feature curves, structured representations, and graph embedding vectors—into a morphological dynamic signature (MDS), store it in a standard image library, and index it as the corresponding Unicode code point or LaTeX command ID.

[0093] Example 2: Student Handwritten Symbol Recognition This embodiment describes how to recognize symbols in students' handwritten answers.

[0094] Input: Student's handwritten answer "∫x 3 The photo of "dx".

[0095] Step 1: Preprocessing and Segmentation. A formula detection model is used to locate the formula region in the image. Then, a character segmentation model is used to segment the formula region into single-character images, resulting in an image containing a hastily written integral sign ∫, an image containing the letter x, an image containing the superscript ³, and an image containing the letter d.

[0096] Step 2: Handwritten Image Generation. The segmented integral sign image is fed into the TrajNet++ network. Due to the jitter and ligatures in handwritten characters, the recovered trajectory may contain noise. The system first performs adaptive smoothing—adaptively adjusting the smoothing kernel size based on the rate of change of trajectory curvature to remove local jitter while maintaining the overall shape—then extracts the MDS signature of the handwritten symbol, including intrinsic feature curves, structured representation, and graph embedding vectors.

[0097] Step 3: Candidate Set Determination. Using the current problem "∫x 3 The standard answer character set for "dx" is {∫,x, 3,d} is used as the matching candidate set.

[0098] Step 4: Multimanifold matching: Calculate the shape distance d between the student integral sign and the standard integral sign. shape =0.12; Calculate the shape distance d between the student integral sign and the standard letter f. shape =0.67; Calculate the shape distance d between the student integral sign and the standard letter x. shape =0.89; The primitive sequence alignment cost analysis shows that the primitive sequence of the student integral sign is highly matched with the primitive sequence of the standard integral sign, and the alignment cost is significantly lower than the alignment cost with other symbols. Topological similarity analysis confirmed that neither of them has a closed-loop structure and their topologies are identical.

[0099] Step 5: Output Fusion. The Siamese metric network combines the similarity across the three dimensions mentioned above, outputting that the probability of the student's integral sign matching the standard integral sign is 0.98, the probability of matching the standard letter f is 0.12, and the probability of matching the standard letter x is 0.05. The system determines that the handwritten symbol is the integral sign ∫.

[0100] Step 6: Application of Results. Combine the recognition results into the formula "∫x 3 The "dx" option is compared with the original standard answer to determine the correctness of this step.

[0101] Example 3: Distinguishing between similar-looking symbols This embodiment illustrates how the present invention effectively distinguishes similar-looking symbols.

[0102] Scenario: Recognize the handwritten characters “α” (Greek letter alpha) and “a” (Latin letter a).

[0103] Challenge: The two shapes are highly similar, making them easy to confuse using traditional image classification methods.

[0104] The solution of this invention: Differential geometry characteristics: Although the overall shapes of the two are similar, there are subtle differences in the local curvature variation patterns. The curve of α generally has a more uniform curvature distribution, while the curve of a has an abrupt change in curvature at the right vertical stroke. The intrinsic characteristic curve F(s) is able to capture these subtle differences.

[0105] Action primitive sequences: α is usually formed by a single continuous curve (a single primitive or two smoothly connected primitives), while a is usually composed of a curve and a straight line segment (at least two primitives, with a distinct sharp transition). The structural differences in primitive sequences provide strong distinguishing clues.

[0106] Topological features: Neither has closed loops, so topological similarity is a neutral feature in this scenario and does not introduce confusion.

[0107] By combining information from three dimensions, this invention can distinguish between α and a with an accuracy rate of over 95%.

[0108] Example 4: Robustness to Writing Variations This embodiment illustrates the invention's tolerance to writing variations.

[0109] Scenario: Identify the integral sign ∫ in different handwriting styles, including: Standard writing: smooth curves and clear hooks; Cursive writing: The integral sign is written together with the following letter, and the hook is extended and connected to the next character; Simplified writing: The hook is omitted, and only the main body of the arc is retained; Trembling handwriting: The handwriting is shaky and the path is not smooth.

[0110] The robustness mechanism of this invention: Shape space elastic matching: SRVF shape distance automatically finds the optimal curve for reparameterization during calculation, which can tolerate a certain degree of local deformation and stretching. The elongation of the hook caused by cursive writing will not significantly increase the shape distance.

[0111] Tolerance for merging and splitting primitive sequences: The S-DTW algorithm allows for the merging and splitting of primitive sequences. The simplified notation treats two primitive sequences as "S-curve" single-primitive matching criteria, with merging costs remaining within acceptable limits.

[0112] Adaptive smoothing: During the trajectory recovery stage, jitter writing undergoes adaptive smoothing to eliminate the impact of local jitter on the overall shape.

[0113] Experiments show that, under the various writing variations mentioned above, the recognition accuracy of the present invention remains above 90%.

[0114] In some implementations, a synthetic data engine is used to generate large-scale training samples. The synthetic data engine transforms standard trajectories into millions of handwritten samples with different styles, covering various noise and deformations, through a handwriting style transfer network.

[0115] Specifically, the handwriting style transfer network employs a generative adversarial network or variational autoencoder framework: The encoder encodes standard trajectories into style-independent content representations; A style encoder that extracts style vectors from reference handwritten samples; The decoder fuses content representation with style vectors to generate handwritten trajectories with a specific style. The discriminator determines whether the generated trajectory is stylistically consistent with the real handwritten sample.

[0116] By adjusting the sampling distribution of style vectors, an infinite number of handwriting samples in various styles can be generated, covering different writing speeds, pressures, tilt angles, and deformation patterns.

[0117] In some implementations, a phased training strategy is employed: In the first stage, TrajNet++ is trained separately. Synthetic data is used to train the network using a joint loss function consisting of trajectory point coordinate reconstruction loss, differentiable rasterization reconstruction loss, and trajectory smoothing regularization term until the network converges.

[0118] The second stage involves training the Segmentation Module (SLDS). Standard trajectories are extracted using pre-trained TrajNet++, and a small number of primitive segmentation boundaries of these trajectories are manually labeled as weak supervision signals. The parameters of the SLDS are then learned using the expectation-maximization algorithm.

[0119] The third stage involves a joint micro-scheduling network. With the parameters of TrajNet++ and the primitive segmenter fixed, a Siamese metric network is trained using triplet loss or contrastive loss, ensuring that MDS signatures of similar symbols are close together in the metric space, while MDS signatures of dissimilar symbols are far apart.

[0120] In some implementations, parameters are fine-tuned using a small number of real handwritten samples to adapt to specific paper and handwriting.

[0121] Specifically: Collect approximately 100-500 images of handwritten characters in real-world scenarios and their labels; Based on the pre-trained model using synthetic data, TrajNet++ and the metric network were fine-tuned using real data with a small number of iterations. A learning rate decay strategy is adopted to prevent overfitting on a small amount of data.

[0122] On a self-built dataset containing 200 classes of mathematical and physical symbols and 10 writing styles, the system based on this invention compares with the baseline CNN classifier: The recognition accuracy improved from 89.7% to 98.2%, while the relative error rate decreased by 81%. The accuracy rate for distinguishing similar-looking symbols (such as α / a, 2 / z, ∫ / f) is improved by more than 15 percentage points; A standard profile can be constructed with just one printed sample, enabling effective recognition, while baseline methods require at least several hundred training samples per class. In scenarios involving cursive, simplified, or shaky handwriting, the accuracy of this invention decreases by less than 5%, while the accuracy of the baseline method decreases by more than 20%.

[0123] It will be apparent to those skilled in the art that this disclosure is not limited to the details of the exemplary embodiments described above, and that this disclosure can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of this disclosure is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this disclosure. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0124] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure, characterized in that, include: The trajectory recovery module is used to generate a continuous arc length parameterized curve γ(s) from the input static image of handwritten symbols or standard printed symbols through a trajectory recovery network based on manifold regularization. The differential geometry parameterization module is used to calculate the Frenet–Serret active frame of the arc length parameterized curve γ(s), extract the tangent vector T(s), normal vector N(s), curvature κ(s) and torsion τ(s), and combine it with the logarithmic velocity profile v(s) to generate the intrinsic characteristic curve F(s) with translation, rotation and scale invariance. The action primitive decomposition module is used to automatically divide the arc length parameterized curve γ(s) into several action primitive segments based on the Lie group parameterization method and using a switching linear dynamic system, and to estimate the corresponding Lie algebra parameters for each primitive segment, and output a structured representation Γ containing primitive labels, Lie algebra parameters and arc length information. The topology graph embedding module is used to construct temporal adjacency edges and spatial interaction edges using action primitive segments as nodes, and combines the continuous cohomology features of the trajectory to encode them into a fixed-length graph embedding vector E through a graph isomorphic network. G ; The matching and recognition module is used to calculate the shape space distance, primitive sequence distance and topological similarity between the handwritten symbol to be recognized and the standard symbol image in the multi-manifold space, and to fuse the matching probability through the Siamese metric network to determine the recognition result.

2. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, The trajectory recovery module includes a manifold regularization-based trajectory recovery network comprising: The Vision Transformer encoder is used to map character images into latent vectors; A continuous-time RNN decoder is used to generate time-series trajectory points; Differentiable rasterization layers are used to ensure that the rendered trajectory is consistent with the original image; And a trajectory smoothing regularization term, used to recover the arc length parameterized curve γ(s).

3. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, The differential geometric parameterization module calculates the torsion τ(s) by supplementing the virtual pressure dimension. The intrinsic characteristic curve F(s) is represented as F(s)=[T(s),κ(s),τ(s),v(s)], where T(s) is the unit tangent vector and v(s) is the logarithmic velocity profile.

4. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, The predefined action primitives in the action primitive decomposition module include: straight line segments, circular arcs, sharp corners, S-curves, elliptical arcs, hooks, spiral segments, sudden stop points, and wavy lines. Each type of primitive corresponds to a single-parameter subgroup on the SE(2) group or the Aff(2) group. The Lie algebra parameter ξ is used to describe the continuous deformation of the primitive segment.

5. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, The structured representation Γ includes the labels, Lie algebra parameters, and arc length information of each primitive segment in the primitive sequence, as well as the SE(2) relative transformation matrix between adjacent primitive segments.

6. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, In the topology graph embedding module, temporal adjacency edges carry relative pose information between adjacent primitive segments, and spatial interaction edges carry distance and intersection information between primitive segments; the persistent homology feature is the persistent landscape vector PH corresponding to the 1D persistent homology graph. vec The graph isomorphic network is a GIN network.

7. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, In the matching and recognition module, the shape space distance is calculated based on the square root velocity function to determine the geodesic distance d in the elastic shape space. shape ; The primitive sequence distance is calculated using structured dynamic time warping to determine the sequence alignment cost, and primitive merging and splitting are allowed.

8. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, The standard symbol profile is a dynamic signature generated based on the printed standard answer, which includes a unified character profile containing differential geometric curves, action primitive sequences, and topological graph embeddings. The candidate set of handwritten symbols to be identified is limited to the character set of the standard answer corresponding to the current question.

9. The mathematical and physical handwritten symbol recognition system based on differential geometric topological graph structure according to claim 1, characterized in that, It also includes a synthetic data engine and a domain adaptation module. The synthetic data engine transforms standard trajectories into handwritten samples of various styles through a writing style transfer network, and the domain adaptation module uses real handwritten samples to fine-tune the system parameters.

10. A method for recognizing handwritten mathematical symbols based on differential geometric topological graph structures, characterized in that, Includes the following steps: S1: Obtain the input handwritten symbol image or printed standard symbol image, and generate a continuous arc length parameterized curve γ(s) through a trajectory recovery network based on manifold regularization; S2: Calculate the Frenet–Serret active frame of the arc length parameterized curve γ(s), extract the tangent vector T(s), normal vector N(s), curvature κ(s) and torsion τ(s), and combine it with the logarithmic velocity profile v(s) to generate the intrinsic characteristic curve F(s) with translation, rotation and scale invariance. S3: Based on the Lie group parameterization method, the arc length parameterized curve γ(s) is automatically divided into several action primitive segments by using a switching linear dynamic system. The corresponding Lie algebra parameters are estimated for each primitive segment, and a structured representation Γ containing primitive labels, Lie algebra parameters and arc length information is output. S4: Using action primitives as nodes, construct temporal adjacency edges and spatial interaction edges. Combined with the continuous cohomology features of the trajectory, encode them into a fixed-length graph embedding vector E through a graph isomorphic network. G This forms a dynamic signature. S5: Calculate the shape space distance, primitive sequence distance, and topological similarity between the handwritten symbol to be identified and the standard symbol image in the multi-manifold space, and fuse the matching probability through the Siamese metric network to determine the recognition result.

Images