Writing evaluation method, system and device based on multi-modal fusion and brain-like computing

Abstract

The present application belongs to the field of machine learning and educational technology, and specifically relates to a writing evaluation method, system and device based on multi-modal fusion and brain-like computing. The method comprises: S1, collecting stroke trajectory data and calculating frame-level evaluation scores; performing dynamic time warping calculation on complete stroke trajectories and template trajectories to obtain stroke scores; and calculating the spatial overlap of writing result images and template images to obtain word-level result similarity; S2, obtaining comprehensive writing quality scores based on frame-level evaluation scores, stroke scores and word-level result similarity; S3, arranging the comprehensive writing quality scores in time sequence to form a scalar time sequence, mapping to a spatial state vector, updating the score prediction value at the next time through a reservoir computing architecture, and generating an evolution trajectory. Through the combination of the three-level scoring mechanism and the brain-like time sequence modeling, the present application realizes the transformation of writing quality from static result evaluation to dynamic process tracking, and improves the accuracy and interpretability of writing quality evaluation.

Description

Technical Field

[0001] This invention belongs to the field of machine learning and education technology, specifically relating to a writing assessment method, system, and device based on multimodal fusion and brain-like computing. Background Technology

[0002] In recent years, neuromorphic computing and multimodal data fusion technology, as emerging directions in artificial intelligence, have gained widespread attention in fields such as behavior monitoring, state assessment, and human-computer interaction. By simulating the information processing mechanisms of biological nervous systems, neuromorphic computing has demonstrated superior performance to traditional models in time-series data modeling.

[0003] Some studies have attempted to apply the above technologies to physiological signal analysis and behavior recognition in the field of education, but their in-depth application for accurate assessment of writing quality and dynamic tracking of writing ability is still insufficient. Specifically, existing methods still have significant shortcomings in the following aspects: (1) lack of a multi-level quality evaluation mechanism for the writing process; (2) failure to incorporate the temporal evolution characteristics of writing quality into the evaluation system; and (3) lack of a systematic writing assessment framework that integrates multi-level scoring results with brain-like temporal modeling capabilities. Summary of the Invention

[0004] The purpose of this invention is to provide a writing evaluation method and system based on multimodal fusion and brain-like computing.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: A writing assessment method based on multimodal fusion and brain-like computing includes the following steps: S1. Collect stroke trajectory data during the writing process, obtain the pen tip position frame by frame, calculate the frame-level distance based on the pen tip position and the template trajectory point position, and calculate the frame-level evaluation score based on the frame-level distance and the trajectory point scale parameter. The complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance. The optimal cumulative distance is then normalized, and the stroke score is obtained based on the normalized optimal cumulative distance and the stroke scale parameter. At the character level, the similarity of the character-level results is calculated based on the intersection area and the union area between the written result image and the template image. S2. Based on frame-level evaluation scores, stroke scores, and character-level result similarity, construct an overall evaluation function to obtain a comprehensive handwriting quality score; S3. Arrange the comprehensive writing quality scores in the order of writing time to form a time-series observation sequence. Extract a certain length of the segment to obtain a time delay vector. Based on the delay embedding theory, map the time delay vector to obtain a spatial state vector. Perform recursive state updates on the spatial state vector to obtain the spatial state vector at the next moment. Map it to obtain the time delay vector at the next moment and the comprehensive writing quality prediction score at the next moment. Generate the score change trajectory of writing ability with writing time.

[0006] In S3, based on delay embedding theory, a spatial state vector is obtained by mapping the time delay vector. The spatial state vector is then recursively updated to obtain the spatial state vector for the next frame. Specifically: Let the target variable be a scalar time series composed of comprehensive writing quality scores. Extracting a segment from it yields the current time T. t Time delay vector Y Tt : Y Tt =[S(T t ),S(T t -1),…,S(T t -L+1)], Where L is the preset embedding dimension; Given that the embedding dimension L > 2d, where d is the attractor dimension of the latent dynamical system, the following holds: ; Where Ψ is the encoding matrix, used to map the spatial state vector to a time delay vector, and Φ is the decoding matrix, used to map the time delay vector to the spatial state vector. This is the spatial state vector at the current moment. Represents an identity mapping; The state r of the reserve pool is obtained through high-dimensional nonlinear mapping. ): , Where, r( () represents the reserve pool at the current time T. t The state vector, Let W be the input weight matrix, and W be the internal connection weight matrix of the reserve pool. The dimension n of the reserve pool satisfies n >> L. Recursively update the state of the spatial state vector: in, This is the spatial state vector for the next time step. For leakage rate parameters, To output the feedback weight matrix, Let T be the current time.t The predicted value of the overall writing quality score.

[0007] In S1, the frame-level evaluation score is calculated based on the frame-level distance and trajectory point scale parameters, specifically as follows: , Where, d t For frame-level distance, For trajectory point scale parameters, This represents the frame-level evaluation score for the current frame. This represents the pen tip in frame t. Indicates the first A template trajectory point.

[0008] In S1, the complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance: Complete stroke trajectory With template trajectory After dynamic time warping calculation, the optimal cumulative distance is: , in, Indicates the first The trajectory of the first stroke and the first The optimal cumulative distance of a template trajectory This represents the i-th stroke. Indicates the first A template trajectory; For the trajectory of the (i-1)th stroke and the... The optimal cumulative distance of a template trajectory For the first The optimal cumulative distance between the trajectory of the first stroke and the trajectory of the (j-1)th template. This represents the optimal cumulative distance between the (i-1)th stroke trajectory and the (j-1)th template trajectory.

[0009] The stroke score in S1 is calculated as follows: , in, y t Indicates the score based on the number of strokes; This represents the normalized optimal cumulative distance; This indicates the stroke scale parameter.

[0010] The similarity of character-level results in S1 is calculated as follows: , Where IoU represents the word-level similarity. Represents the area of ​​intersection. Represents the area of ​​the union. The image represents the writing result. This represents a template image.

[0011] The overall evaluation function in S2 is constructed using the following formula: , Where S represents the overall writing quality score, , , These are the weighting coefficients. Let t be the total number of frames, and t be the t-th frame. This represents the frame-level evaluation score for the current frame. The stroke score represents the similarity score at the character level, while IoU represents the similarity score at the character level.

[0012] The trajectory of the score change in writing ability over writing time is generated as follows: The overall handwriting quality score and the predicted overall handwriting quality score are arranged in chronological order to form a scoring sequence; By plotting the writing time on the horizontal axis and the scoring sequence on the vertical axis, the scoring sequence corresponding to each moment is plotted on a two-dimensional coordinate plane, thus obtaining the trajectory curve of the score change of writing ability with writing time.

[0013] A handwriting assessment system based on multimodal fusion and brain-inspired computing is used to implement a handwriting assessment method based on multimodal fusion and brain-inspired computing, including: The evaluation score calculation module collects stroke trajectory data during the writing process, obtains the pen tip position frame by frame, calculates the frame-level distance based on the pen tip position and the template trajectory point position, and calculates the frame-level evaluation score based on the frame-level distance and the trajectory point scale parameter. The complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance. The optimal cumulative distance is then normalized, and the stroke score is obtained based on the normalized optimal cumulative distance and the stroke scale parameter. At the character level, the similarity of the character-level results is calculated based on the intersection area and the union area between the written result image and the template image. The comprehensive handwriting quality score calculation module constructs an overall evaluation function based on frame-level evaluation scores, stroke scores, and character-level result similarity to obtain a comprehensive handwriting quality score. The prediction module arranges the comprehensive handwriting quality scores in chronological order to form a time-series observation sequence. It extracts a segment of a certain length to obtain a time delay vector. Based on the delay embedding theory, it maps the time delay vector to obtain a spatial state vector. It then recursively updates the spatial state vector to obtain the spatial state vector at the next moment. This is then mapped to obtain the time delay vector at the next moment, as well as the predicted comprehensive handwriting quality score at the next moment, generating a trajectory of the handwriting ability score change over time.

[0014] A writing evaluation device based on multimodal fusion and neuromorphic computing includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a writing evaluation method based on multimodal fusion and neuromorphic computing.

[0015] Compared with the prior art, the technical solution provided by this invention has the following advantages: (1) A three-level writing quality evaluation system was constructed, namely frame-level, stroke-level, and character-level. Most existing methods only evaluate writing results at the character level, which cannot reflect fine-grained issues in the writing process. This invention quantifies and scores writing quality at three levels: frame-level matching similarity, stroke dynamic time regularization distance, and character-level spatial overlap, providing fine-grained diagnostic basis for subsequent personalized feedback and intervention.

[0016] (2) Addressing the shortcoming of the background technology that "the temporal evolution characteristics of handwriting quality are not incorporated into the evaluation system," this invention arranges the comprehensive handwriting quality scores in chronological order to form a scalar time series. It maps the temporal data into a spatial state vector using delayed embedding theory and employs a reservoir computing architecture to perform high-dimensional nonlinear modeling and recursive state updates on the handwriting quality score sequence. This not only outputs the comprehensive score at the current moment but also the predicted score for the next moment, generating an evolutionary trajectory of handwriting ability as practice progresses. Compared to existing methods that only provide static evaluation results such as "how many points were written this time," this invention reveals dynamic information.

[0017] (3) This invention integrates frame-level, stroke-level, and character-level scoring results into a comprehensive handwriting quality score through an overall evaluation function. The comprehensive score is then input sequentially into a brain-like multimodal knowledge tracking model for dynamic modeling, forming a complete closed loop of multi-level scoring, comprehensive quality scoring, and sequential dynamic tracking. The data flow between each stage within the framework is clear, and the input-output relationship is explicit: the three-level scoring provides multi-dimensional input to the comprehensive score, the comprehensive score provides temporal observation data to the brain-like model, and the prediction results of the brain-like model can be fed back to the scoring stage as a reference. Compared to the loosely structured, independent approach of existing methods, this invention achieves deep integration and collaborative work between assessment and tracking, enabling handwriting quality assessment to not only have diagnostic functions but also predictive capabilities. Detailed Implementation

[0018] To further understand the content of this invention, the invention will be described in detail with reference to the embodiments. A writing assessment method based on multimodal fusion and brain-like computing includes the following steps: S1. Collect stroke trajectory data during the writing process, obtain the pen tip position frame by frame, calculate the frame-level distance based on the pen tip position and the template trajectory point position, and calculate the frame-level evaluation score based on the frame-level distance and the trajectory point scale parameter. The complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance. The optimal cumulative distance is then normalized, and the stroke score is obtained based on the normalized optimal cumulative distance and the stroke scale parameter. At the character level, the similarity of the character-level results is calculated based on the intersection area and the union area between the written result image and the template image.

[0019] Before the experiment began, all data acquisition devices were uniformly calibrated to ensure that the acquisition range of each device matched the needs of writing behavior monitoring. Then, all devices were started by a synchronous trigger signal to enable multiple devices to start acquiring data simultaneously. All acquired data were bound with high-precision timestamps to ensure consistency in the time dimension. After the acquisition was completed, the raw data was initially screened to remove redundant data, blank data, and distorted data caused by no writing behavior, device failure, etc., and retained the multimodal raw data within the effective writing period to form the original valid dataset.

[0020] In terms of writing quality evaluation, a multi-level scoring mechanism is constructed at the process frame level, stroke level, and character level.

[0021] Frame-level evaluation: The frame-level distance between the pen tip position and the template trajectory points is calculated, and the matching similarity of the current frame is calculated based on the frame-level distance and scale parameters to obtain the frame-level evaluation score. Let the i-th... The position of the pen tip is The template trajectory point set is The frame-level distance is then defined as: , Where, d t For frame-level distance, For trajectory point scale parameters, This represents the frame-level evaluation score for the current frame. This represents the pen tip in frame t. Indicates the first Template trajectory points.

[0022] Stroke-level evaluation: Dynamic time warping is performed on the complete stroke trajectory and the template trajectory to obtain the optimal cumulative distance between two different points, thus evaluating the complete stroke trajectory. With template trajectory After dynamic time warping calculation, the optimal cumulative distance is: , in, Indicates the first The trajectory of the first stroke and the first The optimal cumulative distance of a template trajectory This represents the i-th stroke. Indicates the first A template trajectory; For the trajectory of the (i-1)th stroke and the... The optimal cumulative distance of a template trajectory For the first The optimal cumulative distance between the trajectory of the first stroke and the trajectory of the (j-1)th template. It represents the optimal cumulative distance between the (i-1)th stroke trajectory and the (j-1)th template trajectory.

[0023] It should be noted that since the scenario of this invention is for strokes collected after writing has begun, i and j are both greater than 1, and there is no case where i-1 and j-1 are less than or equal to 0.

[0024] The optimal cumulative distance is normalized, and a stroke score is obtained based on the normalized distance and stroke scale parameters, thus achieving a quantitative assessment of the writing quality of a single stroke; the specific calculation method is as follows: , in, y t Indicates the score based on the number of strokes; This represents the normalized optimal cumulative distance; This indicates the stroke scale parameter.

[0025] Character-level evaluation: Spatial overlap is calculated based on the intersection and union areas of the written result image and the template image to obtain the character-level similarity. The specific calculation method is as follows: , Where IoU represents the word-level similarity. Represents the area of ​​intersection. Represents the area of ​​the union. The image represents the writing result. This represents a template image.

[0026] A three-tiered writing quality evaluation system—frame-level, stroke-level, and character-level—was constructed. Most existing methods only evaluate writing results at the character level, failing to reflect fine-grained issues in the writing process. This invention quantifies writing quality at three levels: frame-level matching similarity, stroke dynamic temporal regularization distance, and character-level spatial overlap, providing fine-grained diagnostic basis for subsequent personalized feedback and intervention.

[0027] S2. Based on frame-level evaluation scores, stroke scores, and character-level result similarity, construct an overall evaluation function to obtain a comprehensive writing quality score.

[0028] The overall evaluation function is constructed using the following formula: , Where S represents the overall writing quality score, , , These are the weighting coefficients. Let t be the total number of frames, and t be the t-th frame. This represents the frame-level evaluation score for the current frame. The stroke score represents the similarity score at the character level, while IoU represents the similarity score at the character level.

[0029] This multimodal, multi-level model achieves a joint characterization of the writing process and results within a unified feature space, providing a theoretical basis and technical path for intelligent and interpretable writing quality assessment.

[0030] This invention achieves an organic unity between writing process scoring and result scoring. Most existing methods only focus on the image similarity of the writing result, belonging to the "result evaluation" paradigm. This invention incorporates frame-level and stroke-level writing process scoring and character-level result similarity into the overall evaluation function through a weighted fusion method, achieving a comprehensive evaluation transformation and making the scoring results more comprehensive and interpretable.

[0031] S3. Arrange the comprehensive writing quality scores in the order of writing time to form a time-series observation sequence. Extract a certain length of the segment to obtain a time delay vector. Based on the delay embedding theory, map the time delay vector to obtain a spatial state vector. Perform recursive state updates on the spatial state vector to obtain the spatial state vector at the next moment. Map it to obtain the time delay vector at the next moment and the comprehensive writing quality prediction score at the next moment. Generate the score change trajectory of writing ability with writing time.

[0032] This invention constructs a structured mapping framework from spatial to temporal information based on a brain-like reservoir multimodal cognitive tracking model. It proposes a reservoir computing architecture based on delayed embedding and spatiotemporal information conversion theory to achieve prediction and cognitive state characterization of stroke-level dynamic behavioral sequences. The model is grounded in Takens' delayed embedding theorem, mapping high-dimensional spatial states to future time-series vectors. Under the condition of sufficiently large embedding dimension, it establishes an invertible correspondence between spatial and temporal representations, thereby achieving prediction and cognitive state characterization of stroke-level dynamic behavioral sequences. The specific details are as follows: Based on delay embedding theory, a spatial state vector is obtained by mapping the time delay vector. The spatial state vector is then recursively updated to obtain the spatial state vector for the next frame. Specifically: Let the target variable be a scalar time series composed of comprehensive writing quality scores. Extracting a segment from it yields the current time T. tTime delay vector Y Tt : Y Tt =[S(T t ),S(T t -1),…,S(T t -L+1)] Where L is the preset embedding dimension, which represents the number of rating data points contained in the time delay vector, and also represents the number of historical time steps back to predict the rating at the next moment. Given that the embedding dimension L > 2d, where d is the attractor dimension of the latent dynamical system, the following holds: ; Where Ψ is the encoding matrix, used to map the spatial state vector to a time delay vector, and Φ is the decoding matrix, used to map the time delay vector to the spatial state vector. This is the spatial state vector at the current moment. This represents an identity mapping.

[0033] The decoding matrix Φ is obtained by least-squares fitting of historical time-series observation data, specifically by arranging the time delay vectors of each historical moment in columns to form the input matrix Y. in The output matrix X is formed by arranging the spatial state vectors at the corresponding time points in columns. out Through Φ=X out ·Y in ᵀ ·(Y in ·Y in ᵀ ) -1 The decoding matrix Φ is calculated; the encoding matrix Ψ is the Mulpenrose generalized inverse of the decoding matrix Φ, Ψ=Φ. ᵀ ·(Φ·Φ ᵀ ) -1 , such that Φ·Ψ= It holds true, where id is the identity matrix.

[0034] The state r of the reserve pool is obtained through high-dimensional nonlinear mapping. ): , Where, r( () represents the reserve pool at the current time T. t The state vector, The input weight matrix maps the input spatial state vector to a high-dimensional reservoir space. It is independently sampled from a uniform or normal distribution and needs to be scaled after generation, usually by multiplying by an input scaling factor (input gain) to control the excitation intensity of the input signal on the reservoir state. The input scaling factor usually takes a value between 0.1 and 1.0. W is the internal connection weight matrix of the reservoir, which defines the connection strength between neurons within the reservoir and is the source of the reservoir's "memory" ability. W is randomly generated during model initialization and scales with the spectral radius, which is defined as the maximum absolute value among all eigenvalues ​​of matrix W. W does not change after generation during the entire calculation process. The reservoir dimension satisfies >> L.

[0035] Furthermore, since the present invention is designed for strokes captured after writing has already begun, therefore It is greater than 1, and does not exist. -1 is less than or equal to 0.

[0036] The spatial state vector is recursively updated based on the state of the reservoir: in, This is the spatial state vector for the next time step. For leakage rate parameters, To output the feedback weight matrix, Let T be the current time. t The predicted value of the overall writing quality score.

[0037] Furthermore, α is a scalar (single numerical value) with a value range of (0,1], which controls the speed of the state update of the reservoir, i.e. the memory update rate; the leakage rate determines how much of the reservoir state from the previous moment is retained and how much of the new input is absorbed. The closer α is to 1, the faster the state update and the more sensitive the response to the most recent input, but the faster the historical information decays (the shorter the memory). The smaller α is, the closer it is to 0: the slower the state update, the more historical information is retained (the longer the memory), and the smoother the response to the input.

[0038] Output feedback weight matrix The predicted output from the previous moment is fed back to the reserve pool, forming a closed-loop feedback. The value is generated randomly and remains constant; it is related to W. in similar, It also adopts a random generation followed by a fixed method, sampling from a uniform distribution on [-1,1] or from a standard normal distribution, and then multiplying by a feedback scaling factor. The feedback scaling factor is generally much smaller than the input scaling factor to maintain the stability of the reservoir.

[0039] The spatial state vector is recursively updated using the state update equation of the reservoir to obtain the spatial state vector at the next time step. Then, through the decoding matrix Φ, Mapped to the time delay vector Y of the next moment Tt+1 and from Y Tt+1 Extract the first element as the predicted value S(T) of the comprehensive writing quality score for the next moment. t +1), thus generating a trajectory of the score change in writing ability over writing time.

[0040] The trajectory of the score change in writing ability over writing time is generated as follows: The overall handwriting quality score and the predicted overall handwriting quality score are arranged in chronological order to form a scoring sequence; By plotting the writing time on the horizontal axis and the scoring sequence on the vertical axis, the scoring sequence corresponding to each moment is plotted on a two-dimensional coordinate plane, thus obtaining the trajectory curve of the score change of writing ability with writing time.

[0041] This invention introduces delayed embedding theory and a reservoir computing architecture into the field of dynamic tracking of handwriting ability. Existing machine learning methods struggle to effectively capture the temporal evolution of handwriting quality. Based on Takens' delayed embedding theorem, this invention constructs a bidirectional invertible mapping between a temporal delay vector and a spatial state vector. Through a reservoir computing architecture, it performs high-dimensional nonlinear modeling and recursive state updates on the handwriting quality scoring sequence, achieving dynamic estimation of handwriting ability states and prediction of future scoring trends. This overcomes the limitation of traditional methods, which can only score statically and cannot dynamically track handwriting ability.

[0042] A handwriting assessment system based on multimodal fusion and brain-inspired computing is used to implement a handwriting assessment method based on multimodal fusion and brain-inspired computing, including: The evaluation score calculation module collects stroke trajectory data during the writing process, obtains the pen tip position frame by frame, calculates the frame-level distance based on the pen tip position and the template trajectory point position, and calculates the frame-level evaluation score based on the frame-level distance and the trajectory point scale parameter. The complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance. The optimal cumulative distance is then normalized, and the stroke score is obtained based on the normalized optimal cumulative distance and the stroke scale parameter. At the character level, the similarity of the character-level results is calculated based on the intersection area and the union area between the written result image and the template image. The comprehensive handwriting quality score calculation module constructs an overall evaluation function based on frame-level evaluation scores, stroke scores, and character-level result similarity to obtain a comprehensive handwriting quality score. The prediction module arranges the comprehensive handwriting quality scores in chronological order to form a time-series observation sequence. It extracts a segment of a certain length to obtain a time delay vector. Based on the delay embedding theory, it maps the time delay vector to obtain a spatial state vector. It then recursively updates the spatial state vector to obtain the spatial state vector at the next moment. This is then mapped to obtain the time delay vector at the next moment, as well as the predicted comprehensive handwriting quality score at the next moment, generating a trajectory of the handwriting ability score change over time.

[0043] A writing evaluation device based on multimodal fusion and neuromorphic computing includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement a writing evaluation method based on multimodal fusion and neuromorphic computing.

Claims

1. A writing assessment method based on multimodal fusion and brain-like computing, characterized in that, Includes the following steps: S1. Collect stroke trajectory data during the writing process, obtain the pen tip position frame by frame, calculate the frame-level distance based on the pen tip position and the template trajectory point position, and calculate the frame-level evaluation score based on the frame-level distance and the trajectory point scale parameter. The complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance. The optimal cumulative distance is then normalized, and the stroke score is obtained based on the normalized optimal cumulative distance and the stroke scale parameter. At the character level, the similarity of the character-level results is calculated based on the intersection area and the union area between the written result image and the template image. S2. Based on frame-level evaluation scores, stroke scores, and character-level result similarity, construct an overall evaluation function to obtain a comprehensive handwriting quality score; S3. Arrange the comprehensive writing quality scores in the order of writing time to form a time-series observation sequence. Extract a certain length of the segment to obtain a time delay vector. Based on the delay embedding theory, map the time delay vector to obtain a spatial state vector. Perform recursive state updates on the spatial state vector to obtain the spatial state vector at the next moment. Map it to obtain the time delay vector at the next moment and the comprehensive writing quality prediction score at the next moment. Generate the score change trajectory of writing ability with writing time.

2. The writing assessment method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, In S3, based on delay embedding theory, a spatial state vector is obtained by mapping the time delay vector. The spatial state vector is then recursively updated to obtain the spatial state vector for the next frame. Specifically: Let the target variable be a scalar time series composed of comprehensive writing quality scores. Extracting a segment from it yields the current time T. t Time delay vector Y Tt : Y Tt =[S(T t ),S(T t -1),…,S(T t -L+1)], Where L is the preset embedding dimension; Given that the embedding dimension L > 2d, where d is the attractor dimension of the latent dynamical system, the following holds: ； Where Ψ is the encoding matrix, used to map the spatial state vector to a time delay vector, and Φ is the decoding matrix, used to map the time delay vector to the spatial state vector. This is the spatial state vector at the current moment. Represents an identity mapping; The state r of the reserve pool is obtained through high-dimensional nonlinear mapping. ): ， Where, r( () represents the reserve pool at the current time T. t The state vector, Let W be the input weight matrix, and W be the internal connection weight matrix of the reserve pool. Recursively update the state of the spatial state vector: in, This is the spatial state vector for the next time step. For leakage rate parameters, To output the feedback weight matrix, Let T be the current time. t The predicted value of the overall writing quality score.

3. The writing assessment method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, In S1, the frame-level evaluation score is calculated based on the frame-level distance and trajectory point scale parameters, specifically as follows: ， Where, d t For frame-level distance, For trajectory point scale parameters, This represents the frame-level evaluation score for the current frame. This represents the pen tip in frame t. Indicates the first Template trajectory points.

4. The writing assessment method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, In S1, the complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance: Complete stroke trajectory With template trajectory After dynamic time warping calculation, the optimal cumulative distance is: ， in, Indicates the first The trajectory of the first stroke and the first The optimal cumulative distance of a template trajectory This represents the i-th stroke. Indicates the first A template trajectory; For the trajectory of the (i-1)th stroke and the... The optimal cumulative distance of a template trajectory For the first The optimal cumulative distance between the trajectory of the first stroke and the trajectory of the (j-1)th template. It represents the optimal cumulative distance between the (i-1)th stroke trajectory and the (j-1)th template trajectory.

5. The writing assessment method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, The stroke score in S1 is calculated as follows: ， in, y t Indicates the score based on the number of strokes; This represents the normalized optimal cumulative distance; This indicates the stroke scale parameter.

6. The writing assessment method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, The word-level similarity in S1 is calculated as follows: ， Where IoU represents the word-level similarity. Represents the area of ​​intersection. Represents the area of ​​the union. The image represents the writing result. This represents a template image.

7. The writing assessment method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, The overall evaluation function in S2 is constructed using the following formula: ， Where S represents the overall writing quality score, , , These are the weighting coefficients. Let t be the total number of frames, and t be the t-th frame. This represents the frame-level evaluation score for the current frame. The stroke score represents the similarity score at the character level, while IoU represents the similarity score at the character level.

8. The writing evaluation method based on multimodal fusion and brain-like computing according to claim 1, characterized in that, The trajectory of the score change in writing ability over writing time is generated as follows: The overall handwriting quality score and the predicted overall handwriting quality score are arranged in chronological order to form a scoring sequence; By plotting the writing time on the horizontal axis and the scoring sequence on the vertical axis, the scoring sequence corresponding to each moment is plotted on a two-dimensional coordinate plane, thus obtaining the trajectory curve of the score change of writing ability with writing time.

9. A writing evaluation system based on multimodal fusion and brain-like computing, used to implement the writing evaluation method based on multimodal fusion and brain-like computing as described in any one of claims 1-8, characterized in that, include: The evaluation score calculation module collects stroke trajectory data during the writing process, obtains the pen tip position frame by frame, calculates the frame-level distance based on the pen tip position and the template trajectory point position, and calculates the frame-level evaluation score based on the frame-level distance and the trajectory point scale parameter. The complete stroke trajectory and the template trajectory are dynamically time-warped to obtain the optimal cumulative distance. The optimal cumulative distance is then normalized, and the stroke score is obtained based on the normalized optimal cumulative distance and the stroke scale parameter. At the character level, the similarity of the character-level results is calculated based on the intersection area and the union area between the written result image and the template image. The comprehensive handwriting quality score calculation module constructs an overall evaluation function based on frame-level evaluation scores, stroke scores, and character-level result similarity to obtain a comprehensive handwriting quality score. The prediction module arranges the comprehensive handwriting quality scores in chronological order to form a time-series observation sequence. It extracts a segment of a certain length to obtain a time delay vector. Based on the delay embedding theory, it maps the time delay vector to obtain a spatial state vector. It then recursively updates the spatial state vector to obtain the spatial state vector at the next moment. This is then mapped to obtain the time delay vector at the next moment, as well as the predicted comprehensive handwriting quality score at the next moment, generating a trajectory of the handwriting ability score change over time.

10. A writing evaluation device based on multimodal fusion and neuromorphic computing, characterized in that, It includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the writing evaluation method based on multimodal fusion and neuromorphic computing as described in any one of claims 1-8.

Images