Method and system for evaluating scientific creativity based on machine vision and natural language processing

Abstract

This invention discloses a method and system for evaluating scientific creativity based on machine vision and natural language processing. The method includes: acquiring handwritten text images from user writing, and recording behavioral characteristics within a preset time period; performing text recognition on the handwritten text images using a text recognition module, and recognizing mind map-related elements in the handwritten text images using a mind map recognition module; forming a tensor table of the user's handwriting process over time and based on the behavioral characteristics and recognition results; encoding the tensor table from both content and behavioral characteristic dimensions; and predicting the encoding results using a scientific creativity prediction model to obtain a scientific creativity score. This invention enables multi-dimensional and objective evaluation of creativity from the perspective of combining process and content.

Description

Technical Field

[0001] This invention belongs to the field of creativity evaluation, and specifically relates to a scientific creativity evaluation method and system based on machine vision and natural language processing. Background Technology

[0002] Currently, existing technologies related to creativity assessment mainly cover handwritten text recognition, handwritten mind map recognition, and automated assessment of scientific creativity.

[0003] In the field of handwritten text recognition, technologies such as PaddleOCR, based on CRNN models, perform well in recognizing printed or neat adult handwriting. However, when applied to irregular handwriting such as that of primary and secondary school students, where there are overlapping strokes, illegible characters, or scattered structures, the recognition accuracy of existing models drops significantly. This is mainly because most models are trained on standard adult handwriting or printed fonts, which have significant limitations in adapting to the handwriting of primary and secondary school students, whose stroke control is weak and whose font structure is unstable.

[0004] The technical challenges are even greater in the field of handwritten mind map recognition. Existing methods can typically only extract the text content of each node in a mind map, but they struggle to effectively identify and reconstruct the non-hierarchical and divergent logical relationships between nodes. Due to the graphical and unstructured nature of mind maps, accurately understanding the underlying thought processes is currently the main bottleneck facing the technology.

[0005] In the field of automated evaluation of scientific creativity, there are two main existing technical approaches: one is based on semantic network analysis to deconstruct the structure of the response text; the other is to directly use large language models to assess the originality, fluency, and other characteristics of ideas. However, the former often focuses too much on the structural features of the text and may fail to deeply understand the substance and innovation of the response content; the latter is limited by the inherent "illusion" problem of the model and the "black box" nature of the decision-making process, resulting in deficiencies in objectivity, fairness, and interpretability. In addition, existing evaluation schemes generally have two limitations: first, the evaluation object is singular, mainly relying on textual responses and lacking the ability to evaluate diverse outputs such as mind maps; second, the evaluation perspective is singular, mostly result-oriented, lacking dynamic tracking and analysis of the creative process and failing to consider the correlation between writing behavior characteristics and content innovation.

[0006] In summary, existing technologies have not yet provided a comprehensive solution that can effectively adapt to irregular handwriting, accurately understand the logic of graphical thinking, and conduct multi-dimensional and objective evaluation of creativity from the perspective of combining process and content. Summary of the Invention

[0007] In order to at least address one of the problems existing in the prior art, this invention discloses an automatic evaluation method and system for scientific creativity based on machine vision and natural language processing.

[0008] The scientific creativity evaluation method based on machine vision and natural language processing provided by this invention includes the following steps:

[0009] Collect the content written by the user to obtain a handwritten text image, and record the behavioral characteristics within a preset time period, including the percentage of writing speed and the percentage of writing time. The text recognition module performs text recognition on the handwritten text image, and the mind map recognition module identifies mind map-related elements in the handwritten text image; based on the behavioral characteristics and recognition results, a tensor table of the user's handwriting process regarding time and content is formed. The tensor is encoded from the content dimension and the behavioral feature dimension. The encoding results are then predicted by the scientific creativity prediction model to obtain the scientific creativity score. The scientific creativity prediction model includes a meta-learner and multiple base learners. The multiple base learners represent different regression models. The prediction results of each base learner are used as meta-features input into the meta-learner to obtain the final prediction result.

[0010] The scientific creativity evaluation system based on machine vision and natural language processing provided by this invention includes the following units: The data extraction and processing unit is used to collect the content written by the user, obtain the handwritten text image, and record the behavioral characteristics within a preset time period, including the percentage of writing speed and the percentage of writing time. The recognition unit is used to perform text recognition on handwritten text images through the text recognition module, and to recognize mind map-related elements in handwritten text images through the mind map recognition module; based on the behavioral characteristics and recognition results, a tensor table of handwriting and content related to the time of the user's writing process is formed. The evaluation unit is used to predict the code of the tensor scale from the content dimension and the behavioral feature dimension through the scientific creativity prediction model to obtain the scientific creativity score. The scientific creativity prediction model includes a meta-learner and multiple base learners. The multiple base learners represent different regression models. The prediction results of each base learner are used as meta-features input into the meta-learner to obtain the final prediction result.

[0011] The present invention also provides a device.

[0012] The present invention also provides a computer-readable storage medium.

[0013] This invention can collect students' answers to open-ended questions using a dot matrix pen, then identify the handwritten text and handwritten mind maps of the writers, and combine this with the writing behavior feature data collected by the dot matrix pen to form an answer tensor based on a preset time interval (e.g., 20 seconds). Subsequently, based on six dimensions (fluency, effectiveness, flexibility, originality, logic, and elaboration) and behavioral feature dimensions, it uses large-scale model-assisted natural language processing technology to perform encoding operations, integrates the encoded input into a regression model, and finally outputs an evaluation score for the student's scientific creativity.

[0014] Various regression models include linear models, ridge regression models, Lasso regression models, elastic network regression models, support vector machines, support vector regression machines, kernel ridge regression, tree model ensemble algorithms, extreme gradient boosting trees, lightweight gradient boosting machines, gradient boosting regression trees, and neural network models.

[0015] Compared with the prior art, the present invention can achieve at least the following beneficial effects: (1) Use tensor scales for process analysis and recording.

[0016] (2) A novel handwritten font recognition scheme was proposed.

[0017] (3) A novel handwritten mind map recognition scheme was proposed.

[0018] (4) Use coding to interpret the tensor table.

[0019] (5) Use an integrated regression model to evaluate tensor table coding.

[0020] (6) The present invention can identify the writing text image to know the writing situation of students in that time period. It can not only know the final writing result of students, but also pay attention to the answering process, which can solve the problem of lack of process in the existing technology that is result-oriented.

[0021] (7) When collecting data, the present invention can obtain behavioral characteristics based on the collected data, namely, the handwriting speed (pixels / s) and time percentage (the percentage of writing time in the entire preset time period) of students within a preset time period (e.g., 20s) are evaluated together, which can solve the problem of lack of combination between writing content and writing behavioral characteristics in the evaluation process of the existing technology. Attached Figure Description

[0022] Figure 1 This is a flowchart of an embodiment of the present invention.

[0023] Figure 2This is a schematic diagram illustrating an example of a dot matrix pen collecting JSON data in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram illustrating an example of converting JSON data into images at 20-second time intervals in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the framework of the single-character cutting model in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram illustrating the training process of the single-character segmentation model in an embodiment of the present invention.

[0027] Figure 6a This is a schematic diagram of the training process of the binary classification model in an embodiment of the present invention.

[0028] Figure 6b This is a schematic diagram illustrating the verification of the binary classification model in an embodiment of the present invention.

[0029] Figure 7 This is a schematic diagram illustrating the training process of the Chinese character branch single-character recognition model in an embodiment of the present invention.

[0030] Figure 8 This is a schematic diagram illustrating the training process of the single-character recognition model for alphanumeric branches in an embodiment of the present invention.

[0031] Figure 9a and Figure 9b This is a schematic diagram illustrating the practical application of the spatial logic-based sorting algorithm in this embodiment of the invention.

[0032] Figure 10 This is a schematic diagram illustrating the training process of the mind map-related element recognition model in an embodiment of the present invention.

[0033] Figure 11 This is a schematic diagram of Example 2 (curly braces) of handwritten mind map recognition in an embodiment of the present invention.

[0034] Figure 12 This is a schematic diagram of Example 3 (arrow) of handwritten mind map recognition in an embodiment of the present invention.

[0035] Figure 13 This is a schematic diagram of Example 4 (claw) of handwritten mind map recognition in an embodiment of the present invention.

[0036] Figure 14 This is a schematic diagram of Example 1 (lines) of handwritten mind map recognition in an embodiment of the present invention.

[0037] Figure 15 This is a schematic diagram illustrating an example of the automated coding results for all students in a class, as described in an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] Please see Figure 1 The scientific creativity evaluation method based on machine vision and natural language processing provided in this embodiment of the invention includes the following steps: Step 1: Collect the content written by the user to obtain a handwritten text image, and record the behavioral characteristics within a preset time period. The behavioral characteristics include the percentage of writing speed and the percentage of writing time.

[0040] The dot-matrix pen (smart paper pen) is used to extract data from individuals with non-standard handwriting, such as students, on open-ended scientific creativity questions, and saves the data in JSON format. An example data structure is: {"type":0,"p":0.63,"x":99.83,"y":44.5,"ts":1733796697000}, where `type` represents the data state category. In one embodiment, `type` has three cases: 0, 1, and 2, representing the pen-down state, writing state, and pen-up state, respectively. `p` represents the pen pressure measurement value, `x` and `y` represent coordinate points, and `ts` is a standard timestamp (starting from 1970, with 1ms precision). Each JSON data point represents a parameter of a coordinate point, and a single JSON file contains the data for each coordinate point written by the user on that page.

[0041] Data extraction and processing are performed, including: sequentially plotting coordinate points on the canvas, defining a "stroke" as the process from "0" to "2"; connecting the coordinate points within each stroke using a smooth connection method; and finally, reconstructing the user's handwriting. This process is repeated every 20 seconds, recording the percentage of writing speed (the ratio of an individual's instantaneous handwriting speed to their average handwriting speed; handwriting speed is the ratio of the path (or number of pixels) traversed by the pen tip to time) and the percentage of writing time in a CSV file. For each page of content, the converted handwritten text image and the CSV file are saved in the same folder. This completes the data extraction and processing.

[0042] This invention uses a dot matrix pen for data acquisition, which allows the acquired data to have a temporal sequence and can generate students' writing images at preset time intervals.

[0043] Step 2: Recognize the text and mind map related elements in the extracted handwritten text image, and add the content recognized every 20 seconds to a CSV file to form a "handwriting + content" tensor table of the user's writing process over time.

[0044] The recognition process comprises two parts: text recognition and mind map-related element recognition. Specifically, the text recognition module identifies the text content within the handwritten text image. This text recognition module includes a single-character segmentation model and a binary classification model. The system includes a single-character recognition model for alphanumeric branches, a single-character recognition model for Chinese character branches, and a graph logic sorting model; it also uses a mind map recognition module to identify relevant elements in mind maps.

[0045] Text recognition employs a character-by-character recognition approach. First, a single-character segmentation model is used to identify and retrieve individual characters from the handwritten text image. Then, each retrieved character region is cropped and input into a binary classification model to determine whether it is a Chinese character or a number / letter. Based on the classification results, it is determined whether to use the number / letter branch single-character recognition model or the Chinese character branch single-character recognition model. Finally, a visual algorithm is run, considering the positional relationship of each character frame, to obtain the arrangement order of the recognized characters.

[0046] The single-character segmentation model is used to detect the region of each character. In one embodiment, a YOLOv8+CBAM architecture is used, such as... Figure 4 As shown, the map50 value reached 95.7% by fine-tuning the hyperparameters. The dataset was obtained as follows: students answered open-ended questions for scientific creativity assessment using dot-matrix paper pens (smart paper pens, a single device), resulting in JSON data of each student's answers to each question; the data was then converted into images at 20-second intervals; and the labelImg software was used for data labeling. Each character was labeled with the same category, meaning the entire model has only one category.

[0047] The single-character segmentation model in this embodiment of the invention is built based on the YOLOv8 object detection framework and is used for single-character segmentation tasks of handwritten characters. This task can be viewed as a single-class object detection problem, meaning the model only needs to identify and locate each individual character in the image. Figure 4As shown, the overall structure of the single-character segmentation model consists of two main parts: the backbone network and the head. A CBAM (Convolutional Block Attention Module) is integrated into the key feature layer of the head. In one embodiment, CBAM modules are added to layers P3, P4, and P5 of the head. The "concatenation" operation in the diagram connects the upsampled high-level features with the features output by the C2f module at the same level along the channel dimension, thereby achieving multi-scale feature fusion. This allows the model to possess both deep semantic information and shallow detail information, improving detection performance.

[0048] The workflow of the single-character segmentation model is as follows: The input image first undergoes multi-level feature extraction through the backbone network, generating feature maps at different scales (P3, P4, P5). These feature maps are then fed into the detection head for feature fusion and enhancement. In the detection head, the single-character segmentation model fuses feature maps from different scales through upsampling and concatenation operations, and further processes them using the C2f module. After the C2f module processes the feature maps at the P3, P4, and P5 scales, a CBAM module is inserted to enable the model to adaptively optimize feature representation. Finally, the bounding box prediction results of the characters are output through the Detect detection head at the end of the Head structure. The prediction output dimension of the entire single-character segmentation model is (N, 4+1), where 4 represents the bounding box coordinates (represented as the center point coordinates x, y, width w, and height h), 1 represents the confidence that the target exists within the bounding box (since it is a single class, a complex class probability vector is not required), and N represents the total number of bounding boxes predicted by the model for a single input image.

[0049] In one embodiment, the binary classification model is a Chinese character & alphanumeric binary classification model trained based on a modified ResNet18 model, i.e., an improved ResNet18 model. This improved ResNet18 model adds a CBAM module to the two BasicBlock layers of the first layer of the existing ResNet18 model. The most fundamental difference between alphanumeric characters and Chinese characters lies in the underlying visual features such as stroke complexity, structural density, and spatial layout. The first layer (Layer 1) of the ResNet18 model, as the first residual layer encountered by the network, outputs feature maps containing the richest details such as edges, corners, and textures. Adding a CBAM module here is equivalent to installing a "guide" at the source of information processing, immediately guiding the model to focus on those subtle traces crucial for distinguishing the two types of characters. The CBAM module automatically enhances detailed features beneficial to classification (such as the complex stroke connections of Chinese characters) while suppressing irrelevant information, providing cleaner and more discriminative feature representations for subsequent layers. This improves feature quality. Furthermore, by strengthening the differences in low-level features between Chinese characters and alphanumeric characters, the CBAM module helps the model learn stable decision boundaries more easily in subsequent layers, enhancing the model's discriminative ability and thus improving classification accuracy and robustness. Since it is only integrated in Layer 1, the computational overhead and parameter increment introduced by the CBAM module are very limited, ensuring that the model achieves performance improvement while maintaining the efficient inference speed of ResNet18.

[0050] In one embodiment, digits and letters are trained as one class, and Chinese characters as another. The training sets are the Handwritten Chinese Character Dataset (HWDB) from the Institute of Automation, Chinese Academy of Sciences, the EMINST dataset (both open-source datasets), and a dataset of handwritten characters collected from primary and secondary school students. If the binary classification model classifies a single character as a digit or letter, the single-character recognition model of the digit / letter branch is subsequently called for recognition; if the result is a Chinese character, the single-character recognition model of the Chinese character branch is called for recognition.

[0051] Figure 6a The left graph shows the changes in training loss and validation loss (ValLoss) as the number of training epochs increases. It can be seen that the final loss values ​​both decrease to below 0.02, indicating a good fit. Figure 6aThe right-hand graph shows the growth of the model's F1 score (F1 = 2 * (Precision * Recall) / (Precision + Recall), where Precision is the accuracy, representing the percentage of true positives identified by the model (TP = TP / (TP + FP), and Recall is the recall, representing the number of positive examples identified by the model (TP = TP / (TP + FN) where TP represents true positives, FP represents false positives, and FN represents false negatives)) as the number of training epochs increases. It can be seen that the increase in F1 score is steep before leveling off, indicating a good model fit.

[0052] Figure 6b This represents the validation results of the binary classification model. The horizontal axis represents the predicted labels, and the vertical axis represents the true labels. Specifically, 128,100 Chinese character samples were identified as Chinese characters by the model, 31 Chinese character samples were identified as numbers or letters, 29 number / letter samples were identified as Chinese characters, and 91,198 number / letter samples were identified as numbers or letters.

[0053] Among them, the Chinese character branch single character recognition model adopts a model modified based on ResNet50, and the number and letter branch single character recognition model adopts the ResNet18 model.

[0054] The Chinese character recognition model uses the ResNet50 framework, with channel attention mechanisms added to layers 2, 3, and 4. The core idea of ​​this channel attention mechanism is to allow the model to automatically learn the importance weights of each feature channel. Through Global Average Pooling (GAP), the mechanism compresses the two-dimensional feature map of each channel into a scalar, representing the global activation intensity of the feature in that channel. Then, using a small neural network (in one embodiment, including two fully connected layers with dimensionality reduction and ReLU activation in between, and finally using a sigmoid output with weight values ​​between 0 and 1), the model calculates a weight for each channel. Finally, each channel of the original feature map is multiplied by its corresponding weight, achieving the goal of "enhancing useful features and suppressing useless features," which not only accelerates the model's convergence speed but also increases the model's recognition accuracy.

[0055] The following is a description of ResNet50 layers 2, 3, and 4, as well as the channel attention mechanism: ResNet50's second layer (Layer 2): Guides the model to focus on key stroke features. It deals with features at a medium level of abstraction. Adding a channel attention mechanism after Layer2 can guide the model to filter out local patterns that are more crucial for differentiating different Chinese characters among numerous basic stroke features, such as a unique "folding point" or "crossing" structure.

[0056] The third layer (Layer3) of ResNet50: Enhancing the discriminative power of component-level features The features of Layer3 have already started to represent the components of Chinese characters (such as radicals). The channel attention mechanism after it can further amplify the component features that are decisive for differentiating similar-looking Chinese characters. For example, for "晴" (sunny) and "睛" (eyeball), the attention mechanism will enhance the responses of the channels where "日" (sun) and "目" (eye) are located.

[0057] The fourth layer (Layer4) of ResNet50: Making the final calibration at the high-level semantic level Before entering the classification head, the features output by Layer4 are highly abstract. The channel attention module after it plays the role of the final "quality inspector" to ensure that the feature vector fed into the fully connected layer is the cream of the crop, directly improving the classification confidence.

[0058] This hierarchical attention design forms a progressive feature optimization process from local to global, enabling the model to dynamically adjust its focus at different stages of feature extraction.

[0059] In one embodiment, when training the Chinese character branch single-character recognition model, the training set mainly used is the Handwritten Chinese Character Dataset (HWDB) of the Institute of Automation, Chinese Academy of Sciences. There are a total of 3,755 categories. As Figure 7 shown, the training speed of the Chinese character branch single-character recognition model has increased significantly compared to the original framework. It has reached near perfect fitting after 20 rounds, while the original framework takes 60 rounds). The training accuracy of the model is as high as 99.3%, and the validation accuracy is as high as 96.4%.

[0060] The digital and alphabetic branch single-character recognition model uses the traditional ResNet18 model for recognition. In one embodiment, the training of the digital and alphabetic branch single-character recognition model uses the EMNIST dataset. Both the model framework and the data are open-source frameworks and open-source datasets.

[0061] The graph logic sorting model uses a sorting algorithm based on spatial logic to mark the single-character regions retrieved by the single-character cutting model on the graph.

[0062] The spatial logic-based sorting algorithm works as follows: For each character frame, take its center point; select the top-left corner as the starting point, and connect the center point of each character frame to this starting point; select points with tilt angles less than a preset threshold, and determine the point with the shortest distance from the starting point as the next character; then, using the newly marked point as the starting point, iterate sequentially to the right, finally obtaining the connection relationship of the first row of characters. Repeat this process row by row.

[0063] The detailed steps of the spatial logic-based sorting algorithm include: 1. Input preprocessing and feature extraction First, obtain the set of bounding box coordinates for all handwritten characters detected by the single-character segmentation model in the handwritten text image. The bounding box of each character is defined by its top-left and bottom-right corner coordinates in the image coordinate system.

[0064] 1) Character Sorting: To avoid the influence of the initial order, characters are first sorted from left to right based on the center horizontal coordinates of their bounding boxes. The formula for calculating the center coordinates is: c_x =( x_ 1+ x_ 2) / 2, c_y =( y_ 1+ y_ 2) / 2, (x_1, y_1) are the coordinates of the top left corner, and (x_2, y_2) are the coordinates of the bottom right corner; 2) Feature Calculation: Calculate the width w and height h for each character. Finally, the bounding box of each character is represented as a tuple (c_x, c_y, w, h) containing its geometric properties.

[0065] 2. Dynamic grouping threshold calculation This invention abandons the fixed threshold and proposes a dynamic threshold calculation method based on image content adaptation to improve the robustness of the algorithm to different writing styles.

[0066] 1) Horizontal Spacing Threshold: Calculate the horizontal distance d_x = c_x^(i) – c_x^(i-1) between the center points of all adjacent characters (in sorted order). Take the median(d_x) of these horizontal distances as a robust estimate of the average character spacing. The maximum allowable horizontal spacing threshold is defined as: T_h = median(d_x)*k_h, where i represents the character index; k_h represents the horizontal spacing multiplier (or empirical coefficient). In one embodiment, k_h is set to 2.5 to tolerate normal variations in character spacing during writing.

[0067] 2) Vertical offset threshold: Calculate the median of the heights of all characters. The maximum vertical offset threshold allowed for characters within the same line is defined as: T_v = median(h) * k_v, where k_v is an empirical coefficient, which is set to 0.8 in one embodiment to ensure that characters in the same line are roughly aligned vertically.

[0068] 3. Line grouping based on multi-constraint breadth-first search During the search process, to determine whether a candidate character can be grouped into the current line, the following multiple spatial geometric constraint conditions need to be satisfied simultaneously: 1) Horizontal spacing constraint: The horizontal distance dx between the center of the candidate character and the current character must satisfy 0 < d_x ≤ T_h. This ensures the relative order and reasonable spacing of the characters.

[0069] 2) Vertical alignment constraint: The vertical distance |d_y| = |(c_y)^(cand)-(c_y)^(curr)| between the center of the candidate character and the current character must satisfy |d_y| <= T_v. This guarantees the vertical alignment of characters in the same line. cand and curr represent the candidate character and the current character respectively.

[0070] 3) Line direction angle constraint: The absolute value of the angle Θ = arctan(d_y / d_x) between the vector connecting the centers of two characters and the horizontal axis must be less than the set angle threshold to prevent mis-merging characters that are clearly in the upper and lower lines. In one embodiment, the set angle threshold is 30°, that is, |Θ| < 30° needs to be satisfied.

[0071] 4) Non-overlap constraint: The bounding box of the candidate character must not overlap with the bounding boxes of the existing characters in the current line on the horizontal axis. This is achieved by checking whether the center X coordinate is outside the X-axis range of other character boxes.

[0072] 4. Output collation 1) In-line sorting: Each character group obtained through the search is sorted in ascending order of its center X coordinate to form a character sequence from left to right.

[0073] 2) Inter-line sorting: Calculate the average value y^ of the center Y coordinates of all characters in each line, and sort all line groups from top to bottom based on this average value y^.

[0074] 3) Final output: Return an ordered list, where each element represents a line of characters, and the characters within the line are arranged in the reading order.

[0075] In the mind map recognition module, the model based on YOLOv11 is first called to recognize the mind map-related elements in the image. Then, the text recognition module is called to recognize the content of the text area. Finally, the spatial logic algorithm is used to determine the parent-child relationship between the text areas to form markdown text.

[0076] In one embodiment, the mind map elements include: text, arrows, lines, left brackets (l-brace), right brackets (r-brace), and paw symbols.

[0077] In this embodiment of the invention, the model modified based on YOLOv11 is constructed by adding a CBAM module at the end of the backbone layer and a C2PSA module at the end of the head layer, building upon the original YOLOv11 model framework. Adding the CBAM module at the end of the backbone layer enables the filtering and extraction of advanced features, enhancing the feature response to irregular and ambiguous hand-drawn symbols (such as braces and arrows) and suppressing noise interference from murals. Adding the C2PSA module at the end of the head layer facilitates multi-scale feature fusion and spatial relationship modeling, improving the understanding of the relative positions and connections between elements, and accurately determining line connections and text region affiliation. The CBAM and C2PSA modules work together to improve the performance of the YOLOv11-based model in scenarios with complex spatial structures and diverse visual elements, such as handwritten mind maps, enabling it to perform recognition tasks more accurately and robustly. In one embodiment, the weight parameter is adjusted to 1.2, with some fewer sample elements oversampled to 2.7. The recognition accuracy (mAp50) is improved by 5% compared to the original framework, reaching 90.9%. Only two categories with fewer samples (l-brace, paw) had an accuracy rate below 90%, but the accuracy rate of the lowest-performing element improved by 10% compared to the original framework, reaching 80%. The training set was obtained by having students answer open-ended questions in a science creativity assessment using dot-matrix paper (a smart paper pen, a single device), generating JSON data of each student's answers to each question. The data was then converted into images at 20-second intervals. The labelImg software was used for data annotation. The categories included right bracket (r-brace), left bracket (l-brace), arrow, paw, and line.

[0078] The spatial logic algorithm for determining parent-child relationships includes: 1) Left and right curly braces (l-brace / r-brace) Objective: Connect all nodes on the "open" side of the brackets to a node on the "closed" side.

[0079] This logic is divided into two steps: Step 1: Find the parent node (Parent) Strategy: The parent node is assumed to be on the "convex" side of the parentheses (i.e., the closed side).

[0080] Implementation: First, determine the "midpoint of the convex side" of the parentheses.

[0081] For the l-brace ({), the convex side is on the right, so take (brace.x2,...) as the midpoint. The l-brace indicates that the recognized target category is the left curly brace (i.e., the symbol with the shape of "{"), and brace.x2 represents the abscissa of the lower right corner of the detection box (Bounding Box) of this curly brace in the image coordinate system (i.e., the rightmost boundary value of the detection box).

[0082] For the r-brace (}), the convex side is on the left, so take (brace.x1,...) as the midpoint. The r-brace indicates that the recognized target category is the right curly brace (i.e., the symbol with the shape of "}"), and brace.x1 represents the abscissa of the upper left corner of the detection box of this curly brace in the image coordinate system (i.e., the leftmost boundary value of the detection box).

[0083] A "search area" will be created: The l-brace searches on its right (node.obj.center[0]>brace.center[0]).

[0084] The r-brace searches on its left (node.obj.center[0]<brace.center[0]).

[0085] Final judgment: In this search area, the text node with the closest Euclidean distance to the "midpoint of the convex side" is designated as the parent node (Parent).

[0086] Step 2: Find the child nodes (Children) Strategy: The child nodes are assumed to be on the "open" side of the parentheses and aligned with the parentheses in the vertical direction.

[0087] Implementation: Vertical alignment (Y-axis): To capture all relevant child nodes, a relatively loose vertical range is created. The height of the bracket is obtained, and a margin_y is calculated. Then, the y1 of the bracket is extended upward and the y2 is extended downward to form an "extended Y-axis range". y1 represents the top left y-coordinate of the bracket detection box (i.e., the top boundary value of the detection box), and y2 represents the bottom right y-coordinate of the bracket detection box (i.e., the bottom boundary value of the detection box).

[0088] Horizontal alignment (X-axis): Child nodes must be on the "open" side of the brackets.

[0089] l-brace({): The center point of the node must be to the left of brace.x2 (node.obj.center[0]). <brace.x2)。

[0090] r-brace (}): The center point of the node must be to the right of brace.x1 (node.obj.center[0]>brace.x1).

[0091] Final decision: All text nodes that simultaneously meet the conditions of "within the expanded Y-axis range" and "on the open side of the X-axis" are designated as child nodes.

[0092] 2) Arrow Objective: The arrow is the most explicit connector, pointing from a parent node to a child node.

[0093] This logic also involves two steps: Step 1: Determine the direction of the arrow Strategy: Use image processing techniques to find the "tail" and "head" of the arrow.

[0094] accomplish: Skeletonization: Thins the binarized arrow image into a skeleton line that is 1 pixel wide.

[0095] Find endpoints (cv2.filter2D): Use convolution kernel ([[1,1,1],[1,10,1],[1,1,1]]) to detect endpoints on the skeleton (i.e., those pixels with only one neighbor, whose filtered value is 11).

[0096] Determine the beginning and end: Assume that the "tail" of the arrow is the endpoint closest to the center of the arrow detection box, while the "head" is the endpoint farthest from the center.

[0097] The function returns two global coordinates: (tail_global, head_global).

[0098] Step 2: Assign parent and child nodes Strategy: Map the coordinates of the arrow's beginning and end to the nearest text node.

[0099] accomplish: In build_mind_map_graph, build_mind_map_graph() is the function that builds the mind map graph structure. The function's role is to integrate all the detection results, analyze geometric relationships, and establish connections between nodes.

[0100] Parent node: Find the node whose center point is closest to the tail_global coordinate among all text nodes. tail_global refers to the absolute coordinate position (x, y) of the arrow's starting end in the entire original image.

[0101] Child node: Find the text node whose center point is closest to the head_global coordinates among all text nodes. head_global refers to the absolute coordinates (x, y) of the arrow's pointing end (arrow tip) in the entire original image.

[0102] If the parent and child nodes are not the same node, establish a connection from the parent node to the child node (parent_node.add_child(child_node)).

[0103] 3) Paw Objective: "Claw" is a multi-way connector that branches from a "hub point" to a parent node and multiple child nodes.

[0104] Strategy: Find the "average orientation" of the claw by fitting multiple lines using RANSAC, and then use vector dot product to distinguish between "back" (parent node) and "front" (child node).

[0105] accomplish: Find the hub: Use the MeanShift clustering algorithm to find the region with the highest density among the pixels of the "claw" and regard its center (hub_global) as the "hub" of the connection.

[0106] Find the "fingers": Use the iterative RANSACRegressor algorithm to fit multiple straight lines (i.e., the "fingers" of the claw) from the pixels.

[0107] Calculate the average direction (avg_direction): For each fitted "finger" line, find the point farthest from hub_global, which represents the global coordinates of the focal point (hub point) of the claw symbol.

[0108] Create a "direction vector" pointing from the pivot point to this farthest point.

[0109] The average direction vectors of all the "fingers" are averaged to obtain an avg_direction, which represents the average direction vector (representing the overall direction of the claw symbol "opening"). This represents the overall direction of the claw "opening". Assigning father and child (core logic): Iterate through all nearby text nodes.

[0110] Create a node_vector for each node. The node_vector represents the node vector (the vector pointing from the hub point to the center of the candidate text node) (pointing from the hub to the node center).

[0111] Calculate the dot product between avg_direction and node_vector.

[0112] Parent node: If the dot product is less than 0, it means that the node is located on the opposite side of the "opening" direction of the claw (i.e., the position of the "palm" or "arm"), and it is considered the parent node.

[0113] Children: If the dot product is greater than or equal to 0, it means that the node is located in the direction the claw is "opening" (i.e., the position the "finger" is pointing), and they are considered children.

[0114] Fallback strategy: If the dot product does not find a parent node (e.g., all nodes are "in front"), z selects the node closest to the hub as the parent node.

[0115] 4) Line Objective: Lines are the most ambiguous connectors in terms of direction. They connect only two nodes, and direction is inferred here using heuristics.

[0116] Strategy: Prioritize using "connectivity" to determine the outcome. If that fails, then use "candidate states" established by other connectors (parentheses, arrows, etc.) to determine the outcome.

[0117] accomplish: Preprocessing: First, find the two nearest text nodes to the center of each line (a line represents a straight line object detected by the YOLO model) detection box and store them as (node_a, node_b). Simultaneously, calculate the number of lines connected to each node (node_degrees).

[0118] Rule A: Node Degree Takes Priority If degree_a > degree_b, then A is the parent node (because it connects more lines and is more like a "hub").

[0119] If degree_b > degree_a, then B is the parent node. degree_a and degree_b represent the connectivity of node A and the connectivity of node B (i.e., the number of lines connected to that node), respectively. Rule B: Fallback State If degree_a == degree_b (same connectivity), then proceed to the following rule judgment: Check the responses for node.is_parent_candidate and node.is_child_candidate: These are boolean values ​​set when dealing with parentheses, arrows, and claws. node.is_parent_candidate is the parent node candidate flag (i.e., the boolean state that the node is already a parent node), and node.is_child_candidate is the child node candidate flag (i.e., the boolean state that the node is already a child node).

[0120] Core idea: Try to connect a "leaf node" (i.e., is_a_leaf or is_b_leaf, defined as having no parent / child relationship and not being any candidate) to a "non-leaf node" (i.e., a node whose is_parent_candidate or is_child_candidate is True).

[0121] Exhaustively enumerate all possibilities of connecting node_a (non-leaf) to node_b (leaf) using the following rules 1-5.

[0122] Rule 1: A (candidate for both parent and child) + B (is a "true leaf") A is the father.

[0123] Rule 2: A (only the parent candidate) + B (is a "true leaf") A is the father.

[0124] Rule 3: A (only a sub-candidate) + B (is a "true leaf") A is the father.

[0125] Rule 4: A (candidate for both parent and child) + B (candidate for only parent) A is the father.

[0126] Rule 5: A (only a child candidate) + B (only a parent candidate) A is the parent. If none of these conditions are met, try connecting node_b (non-leaf) to node_a (leaf).

[0127] Final decision: If (node_a, node_b) matches either rule A or rule B, establish a parent-child relationship. If all rules fail, print a warning, and the relationship is ignored.

[0128] Step 3: Based on the text recognition results, mind map recognition results, and recorded behavioral characteristics, construct a multidimensional tensor scale; use a scientific creativity prediction model to perform regression prediction on the tensor scale to obtain a scientific creativity score.

[0129] The evaluation in this step includes two parts, one for the text-based response and the other for the mind map, for the two different types of content being identified. The technical approach used in both evaluations is the same; the difference lies in the quantification and encoding methods.

[0130] After data extraction, processing, and image recognition, a time-based CSV tensor table is obtained. Using 20-second time intervals, the percentage of writing speed, percentage of writing volume, and identified writing content within each interval are listed. Subsequently, the tensor table undergoes automated quantification and encoding across multiple dimensions. Two different encoding methods are used for the quantification standards of text responses and mind maps. Finally, the responses from a class are integrated into a single CSV table, with each row containing only the ID and code. The essence of encoding is to interpret the tensor table, extracting its features across six dimensions and assigning them values ​​in three tiers. The codes are then converted into algebraic values ​​for 50 features highly correlated with scientific creativity, fitted by a scientific creativity prediction model. These algebraic values ​​are input into the model, which outputs a score. The simplified workflow is: Tensor Table → Encoding → Scores of 50 Features → Model → Score. The scientific creativity prediction model must first be trained. Training uses the tensor table encoding and human ratings as annotations. After training, the scientific creativity prediction model will fit 50 features that are strongly correlated with scientific creativity, along with the corresponding weights of each feature within the regression model. The subsequent model scoring process involves calling upon the scientific creativity prediction model. First, the tensor table is converted into a coded form. Then, based on the coded form, scores for the 50 features fitted by the scientific creativity prediction model are generated. These 50 feature scores are input into the scientific creativity prediction model, which then calculates the scientific creativity score based on the fitted weights.

[0131] The automated coding method for text-based answers uses a 20-second time interval, representing the student's behavioral characteristics and answer content within that 20-second period. Each 20-second coding interval is separated by a vertical bar. Within a single 20-second interval, each dimension receives one code, separated by commas. Automated quantitative coding is performed on both content and behavioral characteristic dimensions. The content dimensions include six aspects: fluency, effectiveness, flexibility, originality, logic, and detailed explanation. 1) Fluency Behavior code definition: F- (Slow Writing): Writing time accounts for less than 15% of the total writing time. F0 (Normal Writing): 15% ≤ Writing time percentage < 28% F+ (Fluent Writing): Writing time ≥ 28% Reason: The typical concentration range of writing time is about 15%–26%, the upper quartile is about 24%–26%, the 90th percentile is about 30%–34%, and the maximum value is about 57.9%.

[0132] 2) Flexibility step: First, go directly to the final answer and use a large model (such as the deepseek-r1:14b open-source model) to break down the answer from different angles (to examine the multifaceted nature of students' thinking), dividing the content areas from different angles; then see which angle was written within each 20 seconds, and encode it in the following way: L+: Angle switched (encoded as soon as it appears). L0: Continue answering from the original perspective. L: As long as the large model is not divided into different angles, mark it with L. Note: One important point to note here is that if within a 20-second timeframe, the angle is maintained for the first 10 seconds and then changed for the next 10 seconds, it should be labeled L0 or L+. If two angles are specified within the 20-second timeframe, they should be labeled L+, L0, or L+.

[0133] 3) Originality The originality code should follow the L code, and it should not appear only once; there should be an originality code every 20 seconds.

[0134] step: First, we go to the final results of the answers and use a large model to analyze the angles of the answers (consistent with the "flexibility" mentioned above, so this can be directly used). Then, we statistically analyze the answers of all students, breaking down each student's answer into its angles. We use deepseek-r1:14b to force normalization of all angles, that is, answers with similar meanings are summarized and categorized using a concise sentence or phrase. We then analyze the proportion of different categories out of the total for each angle. The lower the frequency, the higher the originality.

[0135] In one embodiment, during encoding: Encoding as O+: Indicates a low-frequency scheme with a probability of occurrence <15%. Encoding O0: Indicates an intermediate frequency (IF) scheme, occurring in 15-50% of cases. The code O- indicates a high-frequency scheme, occurring at a frequency of 50%+. 4) Detailed explanation The detailed explanation of the word retrieval process is as follows: First, jieba word segmentation is used to segment the student's answer in the current 20 seconds. Then, the zh_core_web_trf model in spaCy is used to statistically analyze the parts of speech (e.g., content words, function words, verbs, adverbs) in the word segmentation.

[0136] In one embodiment, during encoding: Encoding E+: Content words ≥ 68% (What are content words? Nouns, verbs, adjectives, adverbs, classifiers, numerals, pronouns. What are function words? Prepositions, conjunctions, auxiliary words, etc.) && Adjectives and adverbs ≥ 22% && Verbs > 18% && The number of characters must be greater than 50 (meeting two or more of these conditions qualifies for E+ encoding) E0 is defined as follows: content words account for 55%–67% of the total word count, adjectives and adverbs account for 14%–21% of the total word count, verbs account for 12%–17% of the total word count, and the word count is between 20 and 39. (E0 is defined if two or more of these conditions are met.) Encoding E-: Content words ≤ 54% && Adjectives and adverbs ≤ 13% && Verbs ≤ 11% && Less than 19 characters (meeting two or more of these conditions qualifies for E- encoding) 5) Logicality The logical part is completed using word retrieval. First, jieba word segmentation is used to segment the student's answer in the current 20 seconds, and then the following types of words are retrieved: Time sequence words: first, second, then, next, subsequently, finally, initially, ultimately, at the same time, simultaneously.

[0137] Causal words: because, therefore, thus, consequently, due to, leading to, result, thus it can be seen, based on this.

[0138] Contrast and transition words: however, but, despite, although, on the contrary, on the contrary, yet, despite this.

[0139] Progressive and supplementary words: moreover, in addition, not only that, more importantly, especially, particularly.

[0140] Summary and inductive terms: In conclusion, to sum up, in short, in essence.

[0141] Conditional and hypothetical words: if, supposing, as long as, only, unless, otherwise.

[0142] Parallel and analogical words: similarly, in a similar way, on the one hand...on the other hand..., both...and..., not...but...

[0143] Labels: 1, 1.1, (1) 1) I, (I) etc. In one embodiment, during encoding: Encoding as G+: More than 2 logical words per 100 words Encoding G0: The number of logical words appearing is 1 per 100 words. Encoded as G-: No logical words present. 6) Effectiveness Validity is primarily assessed by searching for keywords (essentially nouns) and their derivatives in the question. For example, derivative nouns for "bicycle" include: pedals, handlebars, wheels, gearbox, brakes, etc., while derivative verbs include: ride, brake, pedal, etc. Derivative nouns for "giant panda" include: bamboo, etc. The validity of an answer is determined by searching for derivatives of the core names in the question within the answer search.

[0144] The steps for retrieving derived words are as follows: First, the question is segmented using jieba, then the nouns in the question are retrieved using the spaCy model. Next, the prompt word engineering is used to generate derived nouns, verbs, adjectives, and adverbs based on the retrieved nouns in the question using the deepseek-r1:14b model. Simultaneously, the student's answer is segmented in the same way, and the segmented words of the student's answer are compared with the derived words in the question.

[0145] In one embodiment, during encoding: Encoding as V+: Core nouns appear ≥3 times per 100 words && noun derivatives (wheels, bamboo, etc.) ≥5 times per 100 words && verb derivatives (ride, eat, etc.) ≥3 times per 100 words (achieving two or more of these qualifies for V+ encoding) V0: Core noun appears 1-2 times per 100 words && noun derivatives appear 2-4 times per 100 words && verb derivatives appear 1-2 times per 100 words (V0 can be used if two or more of these occur, or V0 can be used if V+, V0, or V- appear). V-: Core noun appears 0 times or only once && noun derivatives appear ≤1 time per 100 words && verb derivatives appear 0 times (V- can be used if two or more of these occur). 7) Behavioral characteristics In one embodiment, during encoding: Encoded as w- (low speed): Pen speed percentage < 80% Encoding as w0 (normal): 80% ≤ Pen speed percentage < 120% Encoding as W+ (High Speed): Pen speed percentage ≥ 120% Encoding N: Write irrelevant content that appears 5 or more "-" characters within a 20-second timeframe, ensuring there is output content and that "V-" appears. The code is P: Nothing was written within those 20 seconds. The following explains the automated coding method for mind map-based answers. Automated quantitative coding is performed from both content and behavioral characteristic dimensions. The content dimension includes six aspects: fluency, effectiveness, flexibility, originality, logic, and detailed explanation.

[0146] 1) Smoothness Maintain consistency with text-based responses. Use the percentage of writing time. Keep the segmentation threshold consistent with text-based responses.

[0147] F- (Slow Writing): Writing time accounts for less than 15% of the total writing time. F0 (Normal Writing): 15% ≤ Writing time percentage < 28% F+ (Fluent Writing): Writing time ≥ 28% 2) Flexibility When coding, if a new idea is written within the 20 seconds, it is marked with L+; if it's from the oldest to the newest, it's marked with L0 and L+; if there's only one, it's marked with L-. (Here, "idea" refers to a first-level branch.) L+: Switched to the first-level branch (coded as soon as it appears). L0: Continue answering on the original branch. L-: Mark as L- if there is only one branch. 3) Originality Perform statistics on all first-level node content. Use the same frequency threshold as in the text category. Perform normalization as well. After categorizing, if the writing occurred in a high-frequency first-level node within the last 20 seconds, encode it as O-; if in a mid-frequency first-level node, encode it as O0; and if in a high-frequency first-level node, encode it as O+. If both occur, take the highest frequency.

[0148] O+: Low-frequency first-level branch, occurrence probability <15% O0: Mid-frequency primary branch, occurring at 15-50% of frequencies. O-: High-frequency first-level branch, occurring in 50%+ of cases. 4) Detailed explanation The detailed explanation depends on the number of levels within a node. When writing a branch of a first-level node, if the branch extends more than 5 levels, it's marked as E+; if it extends 2-4 levels, it's marked as E0; and if it extends 1 level, it's marked as E-. This logic is the same as the originality rule above: as long as the writing occurs within that first-level node for 20 seconds, the same code applies. For example, if the writer spends 0-20 seconds writing within first-level node A, which has 6 branches, and continues writing within node A from 20-40 seconds until finishing at 100 seconds, then all detailed explanation codes from 0-20, 20-40, 40-60, and so on, are marked as E+. If, for example, the writer writes within both node A and node B within a 20-second period, then the detailed explanation code for that 20 seconds is taken as the one with the lowest branch extension level between nodes A and B.

[0149] E+: First-level branches extend to 4 levels or more. E0: The first-level branch extends to 2-3 levels. E-: First-level branch extends 1 level or has no extension. 5) Effectiveness Organize all extracted content from all nodes together, separated by spaces, and perform word segmentation. Perform derivative word lookup on the nouns in the question. Match the terms. Maintain consistency with the matching threshold and encoding rules for text classes.

[0150] In one embodiment, during encoding: V+ is defined as follows: core nouns appear ≥3 times per 100 words, noun derivatives appear ≥5 times per 100 words, and verb derivatives appear ≥3 times per 100 words (achieving two or more of these criteria qualifies for V+). V0: Core noun appears 1-2 times per 100 words && noun derivatives appear 2-4 times per 100 words && verb derivatives appear 1-2 times per 100 words (V0 can be used if two or more of these occur, or V0 can be used if V+, V0, or V- appear). V-: Core noun appears 0 times or only once && noun derivatives appear ≤1 time per 100 words && verb derivatives appear 0 times (V- can be used if two or more of these occur). 6) Logicality To judge the logic of a mind map, three aspects should be considered: the rationality of the structure, the relevance of cross branches, and the completeness of the logical chain.

[0151] Structural rationality: The score distribution for each first-level node is analyzed. Each second-level node is awarded 0.69 points, each third-level node 1.38 points, each fourth-level node 2.07 points, each fifth-level node 2.77 points, and each sixth-level and subsequent node 3.46 points (the weight is actually ln(2^(n-1))). This approach considers both complexity and the number of nodes. If an imbalance occurs between different first-level node branches, the structure is considered unreasonable. The score balance of each first-level node is compared (variance is calculated) to obtain the variance of the overall mind map's structural rationality.

[0152] Cross-branch correlation: Determine if there are logical relationships between the first-level nodes: category (containing and being contained), time relationship, causal relationship (progression). Here, the deepseek-r1:14b model is called, prompt words are set, and a large language model is used to determine whether it is a strong relationship, a medium relationship, or a weak relationship.

[0153] Integrity of the logical chain: The integrity of a logical chain refers to the logical relationships between nodes in a path (a path is a chain of nodes from the root node to every leaf node). The Deepseek-R1:14b model is used for this determination. It checks if the relationship is one of the following (between pairs of nodes): categorical relationship (containing or being contained), causal relationship, progressive relationship, etc. Encoding is done using 0s and 1s. If a relationship is satisfied, it is encoded as 1; otherwise, it is encoded as 0. The ratio of 0s to 1s is calculated.

[0154] In one embodiment, during encoding: Encoding as G+: The variance of the structural rationality score of each first-level branch is <1 / There is a strong relationship between each first-level node / The logical path ratio between all nodes with parent-child relationships is greater than 70%. Meeting two of these three conditions is sufficient to encode as G+. Encoding as G0: The variance of the structural rationality score of each first-level branch is between 1 and 10 / There is a moderate relationship between each first-level node / The logical path ratio between all nodes with parent-child relationships is between 40% and 70%. Meeting two of these three conditions is sufficient to encode as G0. Encoding as G-: The variance of the structural rationality score of each first-level branch is >10 / There is a weak relationship between each first-level node / The logical path ratio between all nodes with parent-child relationships is less than 40%. Meeting two of these three conditions is sufficient to encode as G-.

[0155] If each of the three cases is G+, G0, or G-, then the encoding is G0. 7) Behavioral characteristics In one embodiment, during encoding: Coded as W+: Writing speed percentage > 110% Coded as W0: Writing speed percentage is 85-110%. Coded as W-: Writing speed percentage <85% Encoding as N: Indicates invalid writing. If the above six dimensions contain five or more "-" characters within a 20-second interval, and there is guaranteed output content, and a "V-" appears... The code is P: nothing was written. Framework and Training Method of Ensemble Regression Model (Scientific Creativity Prediction Model) The embodiments of this invention employ a multi-level ensemble learning framework, combining advanced machine learning techniques such as feature engineering and stacked ensembles, to construct a scientific creativity prediction model specifically for continuous prediction of scientific creativity and feature importance analysis.

[0156] Core architecture features 1) Two-layer ensemble structure: Feature layer ensemble + Model layer ensemble (Model layer ensemble refers to combining all the regression models listed below through a stacking strategy. Feature layer ensemble refers to systematically generating a large number of cross-dimensional coupled features from the original time series of the tensor table through sliding window analysis and relation template matching. These features capture the temporal and logical relationships between different dimensions of creativity and basic writing behavior, forming a rich feature pool that provides an information foundation for subsequent model learning.) 2) Multi-algorithm fusion: Combining 10+ regression algorithms to leverage their strengths and compensate for their weaknesses. 3) Automated workflow: end-to-end automation from feature engineering to model evaluation Detailed composition of integrated regression model The scientific creativity prediction model constructed in this invention integrates multiple regression models. Its core lies in employing a stacked ensemble strategy. Instead of simply using multiple models in parallel, it constructs a hierarchical prediction framework designed to combine the advantages of different types of algorithms, aiming to achieve robust and accurate prediction performance surpassing any single model. This ensemble regression model includes a base learner layer and a meta-learner layer.

[0157] 1) Base learner set: Fusion of multi-paradigm algorithms The base learner layer integrates ten regression algorithms from different theoretical paradigms, forming a diverse pool of prediction models. These algorithms are categorized as follows based on their theoretical foundations and technical characteristics: Linear model family: This type of model is based on linear relationships, and prevents overfitting by introducing different regularization constraints, and has a certain feature selection capability.

[0158] Ridge Regression Model: Employing L2 regularization (weight decay) technology, the optimal regularization strength is automatically selected through cross-validation, which can effectively handle multicollinearity among features and improve model stability.

[0159] Lasso regression model: It uses L1 regularization, which tends to compress the coefficients of unimportant features to zero, thereby achieving automatic feature selection and simplifying the model structure.

[0160] Elastic network regression model: It combines the advantages of L1 and L2 regularization methods and is more robust when dealing with data with highly correlated features.

[0161] Support Vector Machines and Kernel Methods: These models excel at capturing nonlinear relationships through kernel function techniques.

[0162] Support Vector Regression (SVR): This model maps data to a high-dimensional feature space using a kernel function, searching for the optimal linear regression hyperplane within this space to handle complex nonlinear mapping relationships. The model preprocesses the input data using standardization and is configured with regularization and insensitive loss parameters.

[0163] Kernel Ridge Regression: This method combines ridge regression with kernel methods and is also used to handle nonlinear problems. It implicitly achieves linear regression in a high-dimensional feature space through the kernel function.

[0164] Tree model ensemble algorithms: These models achieve powerful nonlinear fitting capabilities by integrating multiple decision trees.

[0165] Extreme gradient boosting trees: By iteratively training a series of decision trees, each tree is designed to correct the prediction residuals of the previous tree combination, and the model is continuously optimized through a gradient boosting strategy.

[0166] Lightweight Gradient Boosting Machine: Another efficient implementation of gradient boosting trees, employing a histogram-based algorithm and a depth-limited leaf growth strategy, resulting in faster training speed and lower memory consumption.

[0167] Gradient boosting regression tree: A classic implementation of the gradient boosting algorithm, which builds a strong learner by integrating multiple weak learners (decision trees).

[0168] 2) Neural network model: Multilayer perceptron: A feedforward artificial neural network model containing multiple hidden layers, capable of learning complex nonlinear transformations between inputs and outputs. The model also undergoes data standardization and employs an adaptive moment estimation optimization algorithm for weight updates.

[0169] 3) Meta-learners: Stacked integration framework The meta-learner layer is the core coordination mechanism of the undergraduate creativity prediction model, specifically employing a stacked ensemble strategy. Its workflow is as follows: Phase 1 - Base Learner Prediction: Independently train the above-mentioned multiple base learners (such as ridge regression, extreme gradient boosting tree, gradient boosting regression tree, etc.) using all training data.

[0170] The second stage - meta-feature generation: The trained base learners are used to predict the original training data (overfitting can be avoided by cross-validation), and the prediction results of each base learner are used as new features, namely "meta-features".

[0171] The third stage - Meta-learner training: Using these meta-features as new inputs, a final meta-learner is trained (in this embodiment, a Lasso regression model is selected, and a set of regularization strength candidate values ​​are set for it, and the optimal value is automatically selected through cross-validation) to learn how to optimally weight and combine the prediction results of each base learner.

[0172] The final output of this stacked ensemble framework is the final prediction generated by the meta-learner based on the prediction results of all base learners. In this way, it can automatically learn and utilize the unique advantages of different base learners to form a unified prediction model with better overall performance.

[0173] In summary, the undergraduate science creativity prediction model, through a carefully designed stacked structure, organically combines multiple algorithms such as linear models, kernel methods, tree ensemble models, and neural networks, achieving effective complementarity and synergistic effects among heterogeneous models, and providing a core computing engine for accurate and stable prediction of scientific creativity.

[0174] To ensure the scientific creativity prediction model possesses excellent generalization ability and avoids overfitting to training data, thereby guaranteeing its accuracy and robustness in predicting scientific creativity scores on unknown data, a systematic model training and generalization assurance mechanism is adopted in one embodiment. The core of this mechanism lies in employing a unified training process under a cross-validation framework and incorporating multi-level, integrated anti-overfitting strategies.

[0175] 1) Robust training process based on cross-validation The training and evaluation of the scientific creativity prediction model does not employ the traditional method of one-time partitioning of the training / test set, but instead follows the K-fold cross-validation principle. The specific implementation method is as follows: Data partitioning: All available training data is randomly but uniformly divided into K (in one embodiment, K=5) mutually exclusive subsets, known as "folds". This process ensures that the partitioning results are reproducible for each experiment by using a fixed random number seed.

[0176] Training and Validation Iteration: In each round of training, one subset is used as the validation set, and the remaining K-1 subsets are used as the training set. This process is repeated K times to ensure that each subset is used as the validation set exactly once.

[0177] Performance Evaluation and Prediction: In each training round, a scientific creativity prediction model is fitted using the current training set, and predictions are made on the corresponding validation set. Finally, the predictions from all K validation sets are concatenated to obtain a nearly unbiased cross-validation prediction result for the entire training dataset. This result is used to objectively evaluate the model's generalization performance, rather than using the model's fit on the training set.

[0178] This cross-validation training process is the basic framework for training the scientific creativity prediction model in this embodiment of the invention. Its core advantage lies in making the most of limited data for model training and validation, thereby obtaining a more reliable estimate of the model's true generalization error.

[0179] 2) Integrated multi-level anti-overfitting strategy Within the cross-validation framework, this invention also integrates multiple technical means to work synergistically at different levels, constructing a depth-based overfitting prevention system: Algorithm-level regularization constraints: In the family of linear models (such as Ridge Regression, Lasso Regression, and Elastic Networks), explicitly introducing L1 and / or L2 regularization terms into the loss function effectively constrains the complexity of the model by penalizing excessively large model parameter values, preventing it from over-adapting to noise in the training data, thereby improving robustness.

[0180] Structural constraints and early stopping mechanisms in tree models: For various gradient boosting tree models (such as XGBoost and LightGBM), the expressive power and complexity of a single model are controlled by presetting the maximum depth of the tree, limiting the number of weak learners, and setting a low learning rate. At the same time, these algorithms typically integrate early stopping mechanisms to terminate training prematurely when the performance on the validation set no longer improves, thus avoiding unnecessary overfitting.

[0181] The inherent diversity advantage of ensemble learning: The stacked ensemble strategy employed in this embodiment of the invention is an effective means of preventing overfitting. By combining multiple base learners with different properties (heterogeneous), the scientific creativity prediction model can smooth out the overfitting tendency that may exist in individual models, reduce the variance of the overall model, and thus obtain more stable prediction performance.

[0182] A streamlined data usage strategy: As mentioned earlier, cross-validation itself is a key process to prevent overfitting. It ensures the objectivity of model performance evaluation, avoids optimistic estimates of model performance due to the randomness of a single data split, and thus guides the selection of models with true generalization ability.

[0183] In summary, the embodiments of this invention combine the cross-validation training framework with inherent techniques such as regularization, structural constraints, and ensemble diversity to form a rigorous and efficient model training and generalization assurance system. This system ensures that the final scientific creativity prediction model can not only learn the patterns in the training data well, but also be extended to new and unseen student behavior data, laying a solid technical foundation for achieving scientific and objective automated evaluation.

[0184] In the field of handwritten text recognition, although this invention adopts the traditional single-character segmentation → recognition → sorting scheme, it proposes numerous improvements and innovations in technical details, and uses non-standard fonts from primary and secondary school students for targeted training. This scheme performs better for the font characteristics of primary and secondary school students, such as non-standard font structure and irregular font layout. Simultaneously, a binary classification model is used for dual-branch recognition, employing single-character recognition models for digits and letters, and Chinese characters, for targeted recognition. Compared to the mixed training scheme, the recognition accuracy is 5.2% higher, achieving a handwritten text recognition accuracy of 94.6%. In the field of handwritten text recognition by primary and secondary school students, the recognition accuracy is higher than PaddleOCR.

[0185] In the field of handwritten mind map recognition, the technical solution adopted in this invention—mind map structure recognition → text region recognition → parent-child relationship determination using algorithms—solves the problem of mind map recognition in the daily teaching of primary and secondary school students. The overall mind map recognition accuracy reaches 87.45%.

[0186] In the field of process assessment, this invention combines two behavioral characteristics (percentage of writing speed and percentage of writing time) with the writing content to form a tensor scale for evaluation. In interpreting the tensor scale, based on the TTCT Torrance Creative Thinking Test, automated coding is performed on six dimensions (fluency, flexibility, validity, originality, elaboration, and logic) and behavioral characteristics. Subsequently, the coding and human scoring are used to train a scientific creativity prediction model for process assessment. During the automated coding process, traditional natural language processing methods are effectively combined with large-scale model reasoning analysis, forming a transparent, comprehensive, and objective coding system. Testing showed that the automated coding accuracy reached 98.7%, and the model training accuracy reached 87%.

[0187] In one embodiment, a scientific creativity evaluation system based on machine vision and natural language processing is provided, comprising the following units: The data extraction and processing unit is used to collect the content written by the user, obtain the handwritten text image, and record the behavioral characteristics within a preset time period, including the percentage of writing speed and the percentage of writing time. The recognition unit is used to perform text recognition on handwritten text images through the text recognition module, and to recognize the mind map components in the handwritten text images through the mind map recognition module; based on the behavioral features and recognition results, a tensor table of handwriting and content related to the time of the user's writing process is formed. An evaluation unit is used to encode the tensor table from the content dimension and the behavioral feature dimension, and input the encoding results into the scientific creativity prediction model for prediction to obtain a scientific creativity score; wherein, the scientific creativity prediction model includes a meta-learner and multiple base learners, the multiple base learners are different regression models, and the prediction results of each base learner are used as meta-features to input into the meta-learner to output the final prediction result.

[0188] In one embodiment, an apparatus is provided, the apparatus including a processor and a memory, the memory for storing instructions or computer programs, and the processor for executing the instructions or computer programs in the memory to cause the apparatus to perform the steps of the method described in the foregoing embodiments.

[0189] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing instructions that, when executed on a device, cause the device to perform the steps of the method described in the foregoing embodiments.

[0190] The system, device, and medium described herein have the same technical effects as those achieved by the methods described in the foregoing embodiments.

[0191] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined in this invention may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A scientific creativity evaluation method based on machine vision and natural language processing, characterized in that, Includes the following steps: Collect the content written by the user to obtain a handwritten text image, and record the behavioral characteristics within a preset time period, including the percentage of writing speed and the percentage of writing time. The text recognition module performs text recognition on the handwritten text image, and the mind map recognition module identifies mind map-related elements in the handwritten text image; based on the behavioral characteristics and recognition results, a tensor table of the user's handwriting process regarding time and content is formed. The tensor is encoded from the content dimension and the behavioral feature dimension. The encoding results are then predicted by the scientific creativity prediction model to obtain the scientific creativity score. The scientific creativity prediction model includes a meta-learner and multiple base learners. The multiple base learners represent different regression models. The prediction results of each base learner are used as meta-features input into the meta-learner to obtain the final prediction result.

2. The scientific creativity evaluation method based on machine vision and natural language processing according to claim 1, characterized in that, The text recognition module includes a single-character segmentation model, a binary classification model, a single-character recognition model with alphanumeric branches, and a single-character recognition model with Chinese character branches. The single-character segmentation model is used to retrieve and crop individual characters from a handwritten text image to obtain single-character regions. The binary classification model is used to recognize these single-character regions, obtaining a classification detection result indicating whether the content of the single-character region is a Chinese character or an alphanumeric character. The alphanumeric branch single-character recognition model is used to recognize single-character regions from the alphanumeric classification detection results, and the Chinese character branch single-character recognition model is used to recognize single-character regions from the Chinese character classification detection results.

3. The scientific creativity evaluation method based on machine vision and natural language processing according to claim 2, characterized in that, The text recognition module also includes a graph logic sorting model, which uses a spatial logic-based sorting algorithm to mark the single-character regions identified and retrieved by the single-character segmentation model on the graph.

4. The scientific creativity evaluation method based on machine vision and natural language processing according to claim 3, characterized in that, The spatial logic-based sorting algorithm is as follows: take the center point of each character box; take the center point of the top left character box as the starting point, and connect the center point of each character box in the figure to the starting point; select the points with an inclination angle less than a preset threshold, and determine the point with the smallest distance from the starting point as the next character of the starting point. Then, using the newly marked point as the starting point, iterates to the right sequentially until the connection relationship of the first line of characters is obtained. This process is then repeated line by line.

5. The scientific creativity evaluation method based on machine vision and natural language processing according to claim 2, characterized in that, The single-character segmentation model is built on the YOLOv8 object detection framework, including a backbone network and a detection head. A CBAM attention module is integrated into the key feature layer of the detection head, enabling the single-character segmentation model to adaptively optimize feature representation. The binary classification model uses an improved ResNet18 model, which is obtained by adding a CBAM module to the two BasicBlock layers of the first layer of the existing ResNet18 model. The CBAM module guides the binary classification model to focus on subtle traces crucial for distinguishing between Chinese characters and alphanumeric characters. The Chinese character branch single-character recognition model uses a modified ResNet50 model, which is improved by adding a channel attention mechanism to the second, third, and fourth layers of the existing ResNet50. The alphanumeric branch single-character recognition model uses the ResNet18 model.

6. The scientific creativity evaluation method based on machine vision and natural language processing according to claim 2, characterized in that, In the mind map recognition module, the mind map-related elements in the handwritten text image are first recognized, and then the text recognition module is called to recognize the content of the text area; finally, the spatial logic algorithm is used to determine the parent-child relationship between each text area.

7. The scientific creativity evaluation method based on machine vision and natural language processing according to any one of claims 1-6, characterized in that, The scientific creativity prediction model is trained and evaluated following the K-fold cross-validation principle.

8. A scientific creativity evaluation system based on machine vision and natural language processing, characterized in that, The system for implementing the method according to any one of claims 1-7 comprises the following units: The data extraction and processing unit is used to collect the content written by the user, obtain the handwritten text image, and record the behavioral characteristics within a preset time period, including the percentage of writing speed and the percentage of writing time. The recognition unit is used to perform text recognition on handwritten text images through the text recognition module, and to recognize mind map-related elements in handwritten text images through the mind map recognition module; based on the behavioral characteristics and recognition results, a tensor table of handwriting and content related to the time of the user's writing process is formed. The evaluation unit is used to predict the code of the tensor scale from the content dimension and the behavioral feature dimension through the scientific creativity prediction model to obtain the scientific creativity score. The scientific creativity prediction model includes a meta-learner and multiple base learners. The multiple base learners represent different regression models. The prediction results of each base learner are used as meta-features input into the meta-learner to obtain the final prediction result.

9. A device, characterized in that, The device includes a processor and a memory for storing instructions or computer programs, and the processor for executing the instructions or computer programs in the memory to cause the device to perform the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on the device, cause the device to perform the steps of the method according to any one of claims 1-7.

Images