Intelligent mental health assessment method and system based on text feature fusion

Abstract

The invention relates to the technical field of natural language processing, in particular to a mental health intelligent assessment method and system based on text feature fusion. The method comprises the following steps: acquiring a handwriting image of a subject, carrying out handwritten character recognition based on an OCR (Optical Character Recognition) technology to pre-process the image and text data, and collecting a psychological questionnaire after the handwriting image of the subject; according to the preprocessed image and text data, multi-scale handwriting features are extracted based on an improved MobileVit model, personalized handwriting features are extracted based on standard word stock comparison, text semantic features are extracted based on an improved Bert model, and then cross-modal feature fusion is carried out; cross-modal fusion features are analyzed and predicted through a large-scale pre-training model, and intelligent assessment of mental health is achieved; and in combination with the psychological questionnaire condition, the psychological health intelligent assessment result is verified and evaluated, so that the accuracy and reliability of the assessment result are ensured. According to the invention, intelligent and accurate evaluation of the mental health state of the subject can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for intelligent mental health assessment based on text feature fusion. Background Art

[0002] With the increasing awareness of mental health in society, mental health issues have become one of the public health issues of increasing concern worldwide. Mental health not only affects the quality of life and happiness of individuals, but also has an important impact on social development and stability. Traditional mental health assessment methods, such as questionnaires and face-to-face psychological counseling, may be affected by subjectivity and are difficult to implement on a large scale with limited resources. Although these methods can help individuals understand their own mental state to a certain extent, because they rely on self-reports of the assessed, the results are often interfered by personal subjective factors.

[0003] With the rapid development of natural language processing (NLP), computer vision technology (CV) and artificial intelligence, as well as the increasing maturity of cross-modal model technology, new opportunities have emerged for using these technologies to identify mental health. These technologies provide more objective and efficient tools that can make up for the shortcomings of traditional assessment methods. For example, natural language processing technology can extract semantic information from text and reveal the inner emotions and psychological state of the person being assessed; computer vision technology can analyze handwriting images, capture handwriting behavior characteristics, and further infer the individual's emotional and mental health status. However, single-modality assessment methods are still limited in comprehensiveness and accuracy. Summary of the invention

[0004] In order to solve the technical problems of the existing solutions' focus on mental health issues and the limitations of traditional mental health assessment methods, the embodiments of the present invention provide a method and system for intelligent mental health assessment based on text feature fusion. The technical solution is as follows:

[0005] On the one hand, a method for intelligent assessment of mental health based on text feature fusion is provided, characterized in that the method is applicable to an intelligent assessment system for mental health based on text feature fusion, and the system comprises: a handwriting data acquisition module, a preprocessing module, a multimodal feature extraction module, a cross-modal feature fusion module, a decision integration module and a classification result evaluation module;

[0006] Methods include:

[0007] S1. Collect the handwriting image data of the tester to obtain the handwriting image data set and the text semantic data set; pre-process the handwriting image data set and the text semantic data set through the pre-processing module to extract the content information of the handwritten text;

[0008] S2. Use the improved Bert model to extract semantic features from the preprocessed text semantic data set; use the improved MobileVit model to extract visual features from the preprocessed handwriting image data set; use the handwriting feature extractor to extract handwriting features from the handwriting image data set; obtain the features of different modalities after extraction;

[0009] S3, input the features of different modalities into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data that integrates the domain knowledge of vision, semantics, and handwriting psychology;

[0010] S4, through multiple classifiers, make classification decisions on the fused features, and integrate the prediction results of each classifier through a soft voting strategy;

[0011] S5. Analyze the multi-dimensional characteristics of the prediction results to obtain the user's mental health assessment results; collect psychological questionnaire data to verify and evaluate the user's mental health assessment results.

[0012] Optionally, in S1, handwriting image data of the tester is collected to obtain a handwriting image dataset and a text semantic dataset; the handwriting image dataset and the text semantic dataset are preprocessed by a preprocessing module to extract content information of the handwritten text, including:

[0013] Obtain the tester's handwritten notes, the contents of which are: covering all aspects of the tester's daily life and work;

[0014] The number of words collected for each topic shall not be less than 50; the collection environment factors shall be consistent;

[0015] Obtaining the psychological status-related questionnaire filled out by the tester and extracting the evaluation results of the questionnaire;

[0016] Based on the collected handwritten notes of the testers and the questionnaire evaluation results, a handwriting image dataset and a text semantic dataset are obtained;

[0017] The handwriting image dataset and text semantic dataset are preprocessed through the preprocessing module, and the content information of the handwritten text is extracted through optical character recognition (OCR) technology.

[0018] Optionally, in S1, the handwriting image dataset and the text semantic dataset are preprocessed, and the content information of the handwritten text is extracted by optical character recognition (OCR) technology, including:

[0019] Preprocess the handwriting image dataset and text semantic dataset;

[0020] The preprocessing of the handwriting image dataset includes: image grayscale processing, image noise processing, binarization processing, and image rotation correction processing;

[0021] The preprocessing of text semantic data sets includes: text cleaning, optical character recognition (OCR), word segmentation, and natural language processing.

[0022] Optionally, S2, using an improved Bert model to extract semantic features from a text semantic dataset, including:

[0023] Load the pre-trained improved BERT model and its corresponding vocabulary;

[0024] Convert the text sequence after word segmentation into an input format acceptable to the improved Bert model to obtain input data; the acceptable input formats include input IDs, token type IDs, and attention mask formats;

[0025] The input data is passed into the Bert model, and the context representation of each word is output through forward propagation calculation;

[0026] Extract features of each word from the output context representation;

[0027] Use the vector corresponding to the CLS tag as the semantic representation of the entire sentence, or obtain the semantic representation of the entire sentence through pooling operation;

[0028] The extracted semantic features are processed to adapt to the requirements of cross-fusion tasks.

[0029] Optionally, in S2, an improved MobileVit model is used to extract visual features from a handwriting image dataset, including:

[0030] The preprocessed image is input into the MobileVit model, and features of different scales are extracted through the hybrid architecture of the MobileVit model; a feature vector integrating features at different levels is output.

[0031] Optionally, in S2, the handwriting feature extractor includes:

[0032] Based on the handwriting feature definition of handwriting psychology and the comparison idea between handwritten Chinese characters and Chinese characters in the GB2313 standard character library, a single handwritten Chinese character handwriting feature extractor, a line of handwritten Chinese character handwriting feature extractor, and a multi-line handwritten Chinese character handwriting feature extractor are constructed.

[0033] Optionally, in S3, the features of different modalities are input into a cross-modal fusion module for comprehensive processing to obtain comprehensive feature data integrating domain knowledge of vision, semantics, and handwriting psychology, including:

[0034] Obtaining the image visual feature coding, text semantic feature coding and handwriting feature coding, aligning and synchronizing the three feature codings, and merging the aligned and synchronized feature codings; wherein, cross-attention is paid between the image visual feature and the text semantic feature, and between the image visual feature and the handwriting feature;

[0035] The merged feature codes are then processed for feature enhancement.

[0036] Optionally, in S4, a classification decision is made on the fused features through multiple classifiers, and the prediction results of each classifier are integrated through a soft voting strategy, including:

[0037] Integrate different machine learning algorithms into the multi-classifier joint voting decision integration mechanism;

[0038] Train each classifier model independently so that each classifier model learns the mapping from input features to predict the target variable;

[0039] A joint voting mechanism of soft voting is adopted, the probability prediction of each classifier output is considered, the predicted probability of each category is weighted averaged, and the category with the highest average probability is selected to obtain the prediction result.

[0040] On the other hand, a mental health intelligent assessment system based on text feature fusion is provided, which is applied to a mental health intelligent assessment method based on text feature fusion, and the system comprises:

[0041] A handwriting data collection module is used to collect the tester's handwriting image data and obtain a handwriting image data set and a text semantic data set;

[0042] A preprocessing module is used to preprocess the handwriting image dataset and the text semantic dataset to extract the content information of the handwritten text;

[0043] The multimodal feature extraction module is used to extract semantic features from the preprocessed text semantic data set using the improved Bert model; to extract visual features from the preprocessed handwriting image data set using the improved MobileVit model; to extract handwriting features from the handwriting image data set using the handwriting feature extractor; and to obtain features of different modalities after extraction;

[0044] A cross-modal feature fusion module is used to input features of different modalities into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data that integrates domain knowledge of vision, semantics, and handwriting psychology;

[0045] The decision integration module uses multiple classifiers to make classification decisions on the fused features and integrates the prediction results of each classifier through a soft voting strategy;

[0046] The classification result evaluation module is used to analyze the multi-dimensional characteristics of the prediction results to obtain the user's mental health evaluation results; collect psychological questionnaire data to verify and evaluate the user's mental health evaluation results.

[0047] On the other hand, a mental health intelligent assessment device based on text feature fusion is provided, and the mental health intelligent assessment device based on text feature fusion includes: a processor; a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned mental health intelligent assessment methods based on text feature fusion is implemented.

[0048] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored. The at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned intelligent mental health assessment methods based on text feature fusion.

[0049] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0050] In the embodiment of the present invention, the cross-modal model for fusing handwriting features with text semantic features is faster than traditional mental health assessment methods, such as questionnaire surveys and face-to-face psychological counseling. The cross-modal model for fusing handwriting features with text semantic features inputs features of different modalities into a cross-modal fusion module for comprehensive processing, integrating domain knowledge of vision, semantics, and handwriting psychology, and the model effect is better than that of a single modality model.

[0051] The present invention combines handwriting features and text semantic features to provide a more comprehensive and multi-dimensional mental health assessment. This method not only takes into account the changes in writing behavior, but also analyzes the emotions and semantics of the written content, thereby providing a more accurate assessment of the mental state. Single-feature assessment methods may be limited by noise or single-dimensional information. By integrating multiple features, single errors can be reduced and the accuracy and reliability of the assessment results can be improved.

[0052] The cross-modal model method and system for the fusion of handwriting features and text semantic features can be applied to the fields of education such as student mental health monitoring, medical fields such as clinical psychological auxiliary assessment, remote mental health monitoring, etc., workplaces such as corporate employee mental health auxiliary management, judicial fields such as criminal psychology analysis, criminal psychology, etc., and personal health management such as self-mental health auxiliary monitoring; by combining a variety of technical means such as handwriting psychology, text semantic analysis and image visual analysis, a comprehensive assessment of the individual's mental health status is achieved. This method not only improves the accuracy and reliability of mental health identification, but also provides a scientific basis for mental health intervention, which helps public health agencies to formulate effective mental health intervention strategies. With the continuous advancement of technology, it is believed that this method will play an increasingly important role in the field of mental health assessment and intervention, and contribute to improving the overall mental health level of society. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 It is a flow chart of a method for intelligent assessment of mental health based on text feature fusion provided by an embodiment of the present invention;

[0055] Figure 2 It is a schematic diagram of the algorithm network structure for extracting text semantic information based on the improved Bert model provided in an embodiment of the present invention;

[0056] Figure 3 It is a schematic diagram of the network structure of an algorithm for extracting handwriting visual information based on the improved MobileVit provided in an embodiment of the present invention;

[0057] Figure 4 It is a module structure diagram of a handwriting feature extractor based on handwriting psychology domain knowledge provided by an embodiment of the present invention;

[0058] Figure 5 It is a schematic diagram of the algorithm structure of the cross-modal feature fusion mechanism provided by the implementation example of the present invention;

[0059] Figure 6 It is a block diagram of a mental health intelligent assessment system based on text feature fusion provided by an embodiment of the present invention;

[0060] Figure 7 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0062] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0063] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0064] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0065] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0066] The embodiment of the present invention provides a method for intelligent mental health assessment based on text feature fusion, which can be implemented by an intelligent mental health assessment device based on text feature fusion, and the intelligent mental health assessment device based on text feature fusion can be a terminal or a server. Figure 1 The flowchart of the method for intelligent mental health assessment based on text feature fusion is shown, and the method is applicable to an intelligent mental health assessment system based on text feature fusion, and the system includes: a handwriting data acquisition module, a preprocessing module, a multimodal feature extraction module, a cross-modal feature fusion module, a decision integration module, and a classification result evaluation module;

[0067] The processing flow of the method may include the following steps:

[0068] S1. Collect the handwriting image data of the tester to obtain the handwriting image data set and the text semantic data set; pre-process the handwriting image data set and the text semantic data set through the pre-processing module to extract the content information of the handwritten text;

[0069] In a feasible implementation, in S1, the handwriting image data of the tester is collected to obtain a handwriting image data set and a text semantic data set; the handwriting image data set and the text semantic data set are preprocessed by a preprocessing module to extract the content information of the handwritten text, including:

[0070] Obtain the tester's handwritten notes, the content of which is: covering all aspects of the tester's daily life and work; this module records the subject's handwriting process through a variety of handwriting input devices (such as digital tablets, touch screens, smart pens, etc.), and generates data including handwriting tracks and pictures to ensure the accuracy and completeness of the data.

[0071] The number of words collected for each topic shall not be less than 50; the collection environment factors shall be consistent;

[0072] Obtaining the psychological status-related questionnaire filled out by the tester and extracting the evaluation results of the questionnaire;

[0073] Based on the collected handwritten notes of the testers and the questionnaire evaluation results, a handwriting image dataset and a text semantic dataset are obtained;

[0074] The handwriting image dataset and text semantic dataset are preprocessed through the preprocessing module, and the content information of the handwritten text is extracted through optical character recognition (OCR) technology.

[0075] In a feasible implementation, the collection environment should be as consistent as possible to avoid the influence of external light, writing surface and other factors on the collection results. The written content should cover topics such as daily life, study, scientific research or interpersonal communication to ensure the diversity and authenticity of the corpus; the number of words should not be less than 50; the specific content is not limited, and it is recommended to include: scenes, characters, environment, mood and other information; and complete 100 questionnaires on mood, personality, and life, and obtain mental health assessment results through the questionnaire. Collect handwriting image data and questionnaire results.

[0076] In a feasible implementation, in S1, the handwriting image dataset and the text semantic dataset are preprocessed, and the content information of the handwritten text is extracted by optical character recognition (OCR) technology, including:

[0077] Preprocess the handwriting image dataset and text semantic dataset;

[0078] The preprocessing of the handwriting image dataset includes: image grayscale processing, image noise processing, binarization processing, and image rotation correction processing;

[0079] The preprocessing of text semantic data sets includes: text cleaning, optical character recognition (OCR), word segmentation, and natural language processing.

[0080] In a feasible implementation method, the handwriting image data set is preprocessed, such as grayscale processing of the handwriting image, converting the color image into a grayscale image to simplify processing and reduce the amount of calculation; processing the image noise, using methods such as median filtering to remove the noise in the image and improve the image quality; applying a binarization algorithm to convert the grayscale image into a binary image to make the text part more prominent; detecting the tilt angle of the text and performing rotation correction to ensure the text is level.

[0081] Preprocess the text semantic data set, such as cleaning the text semantic data, removing noise characters in the OCR results, correcting typos and misrecognized characters, and improving the accuracy of the text; use word segmentation tools to segment the text and extract keywords and phrases; based on natural language processing technology, perform semantic analysis such as sentiment analysis and topic extraction to understand the emotional tendency and main content of the text.

[0082] In a feasible implementation, through the three main steps of image preprocessing, optical character recognition and text cleaning, the text content and specific meaning can be accurately extracted from the handwriting image.

[0083] S2. Use the improved Bert model to extract semantic features from the preprocessed text semantic data set; use the improved MobileVit model to extract visual features from the preprocessed handwriting image data set; use the handwriting feature extractor to extract handwriting features from the handwriting image data set; obtain the features of different modalities after extraction;

[0084] In a feasible implementation, S2, using an improved Bert model to extract semantic features from a text semantic dataset, including:

[0085] Load the pre-trained improved BERT model and its corresponding vocabulary;

[0086] Convert the text sequence after word segmentation into an input format acceptable to the improved Bert model to obtain input data; the acceptable input formats include input IDs, token type IDs, and attention mask formats;

[0087] The input data is passed into the Bert model, and the context representation of each word is output through forward propagation calculation;

[0088] Extract features of each word from the output context representation;

[0089] Use the vector corresponding to the CLS tag as the semantic representation of the entire sentence, or obtain the semantic representation of the entire sentence through pooling operation;

[0090] The extracted semantic features are processed to adapt to the requirements of cross-fusion tasks.

[0091] In a feasible implementation, a pre-trained improved Bert model and its corresponding vocabulary are loaded to ensure that the model can correctly parse the input text; the segmented text sequence is converted into an input format acceptable to the model, including input IDs, token type IDs, attention masks, etc.; the processed input data is passed into the Bert model, and the context representation (embedding) of each word is obtained through forward propagation calculation; features are extracted from the representation output by the model, usually using the vector corresponding to the [CLS] tag as the semantic representation of the entire sentence, or obtaining the overall representation of the sentence through pooling operations; the extracted semantic features are further processed, such as normalization, dimensionality reduction, etc., to meet the needs of cross-fusion fusion tasks.

[0092] In a feasible implementation, in S2, an improved MobileVit model is used to extract visual features from a handwriting image dataset, including:

[0093] The preprocessed image is input into the MobileVit model, and features of different scales are extracted through the hybrid architecture of the MobileVit model; a feature vector integrating features at different levels is output.

[0094] In a feasible implementation, the MobileVit model can effectively extract features of different scales through its hybrid architecture: coarse-level features, intermediate-level features, and fine-level features. The coarse-level features: quickly extract basic features of the image, such as edges and simple shapes, through the MobileNet part; the intermediate-level features: use the self-attention mechanism of the Transformer part to further analyze the complex relationship between these basic features and enhance the contextual understanding ability of the features; the fine-level features: in a deeper network, carefully capture high-level semantic information and detailed structural features, such as the thickness changes and connection methods of the handwriting. The MobileVit model outputs a set of feature vectors that integrate features at different levels, which can be directly used for subsequent image recognition, classification or cross-modal fusion tasks. Through this series of steps, MobileVit can efficiently extract rich visual features from handwritten handwriting images, supporting more in-depth image analysis and processing.

[0095] In a feasible implementation manner, in S2, the handwriting feature extractor includes:

[0096] Based on the handwriting feature definition of handwriting psychology and the comparison idea between handwritten Chinese characters and Chinese characters in the GB2313 standard character library, a single handwritten Chinese character handwriting feature extractor, a line of handwritten Chinese character handwriting feature extractor, and a multi-line handwritten Chinese character handwriting feature extractor are constructed.

[0097] In a feasible implementation manner, the handwriting features of a single handwritten Chinese character include inclination, absolute size of Chinese characters, relative size of Chinese characters, squareness of Chinese characters, relative inclination of Chinese characters, degree of connection of Chinese characters, degree of offset of coordinates of the center of gravity of Chinese characters, length of character image contour, area of ​​character image contour, perimeter of convex hull of character image, curvature of character image, degree of curvature of strokes, etc.; the handwriting features of a line of handwritten Chinese characters include slope of Chinese character baseline, degree of offset of coordinates of the center of gravity of a line, etc.; the handwriting features of multiple lines of handwritten Chinese characters include top margin, bottom margin, left margin, right margin, degree of offset of coordinates of the center of gravity of multiple lines, etc.

[0098] S3, input the features of different modalities into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data that integrates the domain knowledge of vision, semantics, and handwriting psychology;

[0099] In a feasible implementation, in S3, the features of different modalities are input into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data integrating the domain knowledge of vision, semantics and handwriting psychology, including:

[0100] Obtaining the image visual feature coding, text semantic feature coding and handwriting feature coding, aligning and synchronizing the three feature codings, and merging the aligned and synchronized feature codings; wherein, cross-attention is paid between the image visual feature and the text semantic feature, and between the image visual feature and the handwriting feature;

[0101] The merged feature codes are then processed for feature enhancement.

[0102] In a feasible implementation, the image visual feature encoding, text semantic feature encoding, and handwriting feature encoding are aligned and synchronized to ensure the consistency of the features at the spatial and semantic levels, and the features from these three different sources are merged; cross-attention is paid to the image visual features and text semantic features, as well as the image visual features and handwriting features; the fused features are further processed to enhance the representation ability of the features, so as to improve the expression ability of cross-modal features and ensure that the fused features can effectively support subsequent decision-making integration tasks; through such a cross-modal fusion framework, the model can integrate multiple information such as visual information, text content and handwriting style to provide richer and more accurate analysis results.

[0103] S4, through multiple classifiers, make classification decisions on the fused features, and integrate the prediction results of each classifier through a soft voting strategy;

[0104] In a feasible implementation, in S4, multiple classifiers are used to make classification decisions on the fused features, and the prediction results of each classifier are integrated through a soft voting strategy, including:

[0105] Integrate different machine learning algorithms into the multi-classifier joint voting decision integration mechanism;

[0106] Train each classifier model independently so that each classifier model learns the mapping from input features to predict the target variable;

[0107] A joint voting mechanism of soft voting is adopted, the probability prediction of each classifier output is considered, the predicted probability of each category is weighted averaged, and the category with the highest average probability is selected to obtain the prediction result.

[0108] In a feasible implementation, in the multi-classifier joint voting decision integration mechanism, the complementary characteristics of different machine learning algorithms such as KNN, SVM, XGBoost, etc. are used to improve the accuracy and robustness of the overall decision integration model. Each model is trained independently, and each model learns to predict the mapping of the target variable from the input features; a joint voting mechanism of soft voting is adopted, the probability prediction of each classifier output is considered, the predicted probability of each category is weighted averaged, and the category with the highest average probability is selected. The multi-classifier joint voting decision integration mechanism combines classifiers of different algorithms and uses their respective advantages to enhance the accuracy and generalization ability of prediction.

[0109] S5. Analyze the multi-dimensional characteristics of the prediction results to obtain the user's mental health assessment results; collect psychological questionnaire data to verify and evaluate the user's mental health assessment results.

[0110] In a feasible implementation, the entire data set is randomly divided into K subsets of equal size, each subset tries to maintain the consistency of data distribution, K-1 subsets are used as training data, and the remaining subset is used as verification data. This process is repeated K times, each time a different subset is selected as verification data, and the rest are used as training data; the evaluation indicators of each iteration, such as accuracy, recall, etc., are calculated, and the average of these indicators is calculated to estimate the overall performance of the model. Use multiple evaluation indicators such as accuracy, recall, F1 score and confusion matrix to comprehensively evaluate the performance of the model on the classification task. Apply the optimal model obtained in the actual mental health assessment task, continuously monitor the performance of the model and adjust the model in time to cope with possible data drift or new evaluation requirements. K-fold cross validation provides an evaluation of model stability and generalization ability; multiple evaluation indicators reflect the classification performance of the model from different angles to ensure the practicality and accuracy of the model. Through the above steps, the performance of the mental health assessment model can be evaluated and optimized.

[0111] In a feasible implementation, Figure 2 This is a schematic diagram of the algorithm network structure for extracting text semantic information based on the improved Bert model; Figure 3This is a schematic diagram of the algorithm network structure for extracting handwriting visual information based on the improved MobileVit; Figure 4 The module structure diagram of the handwriting feature extractor based on the knowledge of handwriting psychology; Figure 5 This is a schematic diagram of the algorithm structure of the cross-modal feature fusion mechanism. This embodiment includes a flow chart of text semantics, image vision, and handwriting feature extraction. Different algorithms are used to extract multimodal features, and ultimately the information of multiple modalities is integrated through encoding, cross-attention, and fusion methods.

[0112] The semantic features of text are extracted using an algorithm based on the improved Bert model. The input contains a special [CLS], multiple words (word1, word2, ...) and a separator (SEP); Token Embeddings: Each input word (including special [CLS] and [SEP] tags) is converted into a corresponding word embedding vector (E[CLS], Eword1, Eword2, ..., E[SEP]); Segment Embeddings: Used to distinguish different parts of the input, each word corresponds to a segment embedding vector (Ea or Eb). In this example, words in the same segment have the same embedding vector; Position Embeddings: Each word is assigned a position embedding vector (E0, E1, E2, ..., En) according to its position in the sentence. Multi-HeadAttention: Multiple attention heads obtain attention representation through weighted summation of query, key and value; Add & LayerNormalization: The output of multi-head attention is added to the input and layer normalization is performed; Feed Forward Network: Nonlinear transformation is performed through feedforward neural network; Add & Layer Normalization: The output of feedforward neural network is added to the input and layer normalization is performed. The above encoding layer operations are stacked multiple times, and the multi-head attention, feedforward neural network and normalization operations are repeated each time. Output BERT feature embedding: The final feature representation is the output of the BERT model and can be used for various downstream tasks such as classification, sequence labeling, etc.

[0113] Load the pre-trained improved Bert model and its corresponding vocabulary to ensure that the model can correctly parse the input text; convert the segmented text sequence into an input format acceptable to the model, including input IDs, token type IDs, attention masks, etc.; pass the processed input data into the Bert model, and obtain the context representation (embedding) of each word through forward propagation calculation; extract features from the representation output by the model, usually using the vector corresponding to the [CLS] tag as the semantic representation of the entire sentence, or obtaining the overall representation of the sentence through pooling operations; further process the extracted semantic features, such as normalization and dimensionality reduction, to meet the needs of cross-fusion tasks.

[0114] The improved Bert model algorithm is as follows in Table 2:

[0115] Table 2 Algorithm based on the improved Bert model

[0116]

[0117] In a feasible implementation, the improved MobileVit model is used to extract the coarse, medium and fine-grained visual features of the image, and the preprocessed image is input into the MobileVit model. The MobileVit model can effectively extract features of different scales through its hybrid architecture: coarse features, medium features and fine features, among which coarse features: the basic features of the image, such as edges and simple shapes, are quickly extracted through the MobileNet part; medium features: the self-attention mechanism of the Transformer part is used to further analyze the complex relationship between these basic features and enhance the contextual understanding ability of the features; fine features: in a deeper network, high-level semantic information and detailed structural features are carefully captured, such as the thickness changes and connection methods of the handwriting. The MobileVit model outputs a set of feature vectors that integrate features at different levels, which can be directly used for subsequent image recognition, classification or cross-modal fusion tasks. Through this series of steps, MobileVit can efficiently extract rich visual features from handwritten handwriting images, supporting more in-depth image analysis and processing.

[0118] Extract coarse, medium and fine-grained visual features of the image based on the improved MobileVit model. Receive the input handwriting image and perform a 3x3 convolution operation on the input image, which is the standard initial feature extraction step. Feature extraction branch: contains multiple MV2 (MobileNetV2) modules. These modules are lightweight convolutional layers designed to reduce computational costs while maintaining performance. Coarse feature extraction: contains a MobileViT module followed by an MV2 module. The MobileViT module introduces the Vision Transformer element into the model to capture the global dependencies of the handwriting in the handwriting image. Medium feature extraction: similar to coarse feature extraction, contains another MobileViT module and an MV2 module. Focuses on extracting the medium-level features of the handwriting from the handwriting image. Fine feature extraction: contains another MobileViT module and an MV2 module. Aims to capture the detailed features in the handwriting image. Connection layer: performs a 1x1 convolution operation. It is followed by a global pooling and linear layer. This layer integrates all the extracted features in preparation for the final classification. The improved MobileViT model integrates MobileNetV2 modules and MobileViT modules at different stages, enabling the model to effectively capture local and global features while maintaining computational efficiency.

[0119] The process algorithm for extracting coarse, medium and fine-grained visual features of images based on the improved MobileVit model is as follows in Table 3:

[0120] Table 3 Process algorithm for extracting coarse, medium and fine-grained visual features of images based on the improved MobileVit model

[0121]

[0122] The handwriting feature extractor is used to extract handwriting features from the handwriting image dataset, which mainly includes: the handwriting feature definition based on handwriting psychology, and the comparison between handwritten Chinese characters and Chinese characters in the GB2313 standard character library, and the handwriting feature extractors for single handwritten Chinese characters, a line of handwritten Chinese characters, and multiple lines of handwritten Chinese characters. The handwriting features of a single handwritten Chinese character include inclination, absolute size of Chinese characters, relative size of Chinese characters, squareness of Chinese characters, relative inclination of Chinese characters, degree of connected strokes of Chinese characters, degree of offset of coordinates of the center of gravity of Chinese characters, length of the outline of the character image, area of ​​the outline of the character image, perimeter of the convex hull of the character image, curvature of the character image, degree of curvature of the strokes, etc.; the handwriting features of a line of handwritten Chinese characters include the slope of the baseline of Chinese characters, degree of offset of coordinates of the center of gravity of the line, etc.; the handwriting features of multiple lines of handwritten Chinese characters include top margin, bottom margin, left margin, right margin, degree of offset of coordinates of the center of gravity of multiple lines, etc.

[0123] Finally, through encoding, cross-attention and fusion methods, the information of multiple modalities is integrated. X-Language: language data input; X-Image: image data input; X-Knowlage: knowledge data input in the field of handwriting psychology. The language input data is encoded to generate the language feature representation H-Language; the image input data is encoded to generate the image feature representation H-Image; the knowledge input data is encoded to generate the knowledge feature representation H-Knowlage. The language feature H-Language and the image feature H-Image are input to the Cross Attention module to generate the attention feature H-Attention1; the image feature H-Image and the knowledge feature H-Knowlage are input to another Cross Attention module to generate the attention feature H-Attention2. The attention features H-Attention from the two Cross Attention modules are averaged and fused to generate the fusion feature H-Fusion. The fusion feature H-Fusion is input to the Transformer Decode module for decoding to generate output. This is a multimodal (language, image, and knowledge) feature fusion architecture that integrates information from multiple modalities through encoding, cross-attention, and fusion methods, and ultimately generates prediction results through the decoder.

[0124] In an embodiment of the present invention, the domain knowledge of handwriting psychology, text semantic analysis, and image visual analysis are combined to construct a cross-modal model. This model can comprehensively analyze the psychological state of an individual from different dimensions and improve the accuracy and reliability of recognition. By extracting the characteristics of handwritten handwriting, the emotional fluctuations in the writing process can be captured; by analyzing the semantics of the text, the emotional expression and psychological state of the person being assessed in the written content can be understood; through image visual analysis, the handwriting behavior characteristics of the individual in the writing process can be observed. The combination of these multimodal data can form a more comprehensive mental health assessment system.

[0125] In a feasible implementation, the following Table 1 shows the test results of some algorithms:

[0126] Table 1 Test results of some algorithms

[0127]

[0128] In summary, this application proposes to deeply integrate NLP, CV and handwriting psychology knowledge to build an innovative and comprehensive mental health intelligent recognition system. The system aims to overcome the shortcomings of traditional evaluation methods through the complementary advantages of cross-modal data and achieve more accurate, efficient and objective mental health evaluation.

[0129] Figure 6 1 is a block diagram of a mental health intelligent assessment system based on text feature fusion according to an exemplary embodiment, wherein the system is used for a mental health intelligent assessment method based on text feature fusion. Figure 6 The system includes a handwriting data collection module 310, a preprocessing module 320, a multimodal feature extraction module 330, a cross-modal feature fusion module 340, a decision integration module 350 and a classification result evaluation module 360. Among them:

[0130] The handwriting data collection module 310 is used to collect the handwriting image data of the tester and obtain the handwriting image data set and the text semantic data set;

[0131] The preprocessing module 320 is used to preprocess the handwriting image data set and the text semantic data set to extract the content information of the handwritten text;

[0132] The multimodal feature extraction module 330 is used to extract semantic features from the preprocessed text semantic data set using the improved Bert model; extract visual features from the preprocessed handwriting image data set using the improved MobileVit model; extract handwriting features from the handwriting image data set using the handwriting feature extractor; and obtain features of different modes after extraction;

[0133] A cross-modal feature fusion module 340 is used to input features of different modalities into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data that integrates domain knowledge of vision, semantics, and handwriting psychology;

[0134] A decision integration module 350 is used to make classification decisions on the fused features through multiple classifiers, and to integrate the prediction results of each classifier through a soft voting strategy;

[0135] The classification result evaluation module 360 ​​is used to analyze the multi-dimensional characteristics of the prediction results to obtain the user's mental health evaluation results; collect psychological questionnaire data, and verify and evaluate the user's mental health evaluation results.

[0136] Optionally, the handwriting data collection module 310 is used to obtain the tester's handwritten notes, the contents of which are: all-round topics covering the tester's daily life and work;

[0137] The number of words collected for each topic shall not be less than 50; the collection environment factors shall be consistent;

[0138] Obtaining the psychological status-related questionnaire filled out by the tester and extracting the evaluation results of the questionnaire;

[0139] Based on the collected handwritten notes of the testers and the questionnaire evaluation results, a handwriting image dataset and a text semantic dataset are obtained.

[0140] Optionally, the preprocessing module 320 is used to preprocess the handwriting image dataset and the text semantic dataset through a preprocessing module, and extract the content information of the handwritten text through optical character recognition (OCR) technology.

[0141] Optionally, the handwriting image dataset and the text semantic dataset are preprocessed, and the content information of the handwritten text is extracted by optical character recognition (OCR) technology, including:

[0142] Preprocess the handwriting image dataset and text semantic dataset;

[0143] The preprocessing of the handwriting image dataset includes: image grayscale processing, image noise processing, binarization processing, and image rotation correction processing;

[0144] The preprocessing of text semantic data sets includes: text cleaning, optical character recognition (OCR), word segmentation, and natural language processing.

[0145] Optionally, a multimodal feature extraction module 330 is used to load a pre-trained improved Bert model and its corresponding vocabulary;

[0146] Convert the text sequence after word segmentation into an input format acceptable to the improved Bert model to obtain input data; the acceptable input formats include input IDs, token type IDs, and attention mask formats;

[0147] The input data is passed into the Bert model, and the context representation of each word is output through forward propagation calculation;

[0148] Extract features of each word from the output context representation;

[0149] Use the vector corresponding to the CLS tag as the semantic representation of the entire sentence, or obtain the semantic representation of the entire sentence through pooling operation;

[0150] The extracted semantic features are processed to adapt to the requirements of cross-fusion tasks.

[0151] Optionally, the multimodal feature extraction module 330 is used to input the preprocessed image into the MobileVit model, extract features of different scales through the hybrid architecture of the MobileVit model, and output a feature vector that integrates features at different levels.

[0152] Optionally, a handwriting feature extractor comprises:

[0153] Based on the handwriting feature definition of handwriting psychology and the comparison idea between handwritten Chinese characters and Chinese characters in the GB2313 standard character library, a single handwritten Chinese character handwriting feature extractor, a line of handwritten Chinese character handwriting feature extractor, and a multi-line handwritten Chinese character handwriting feature extractor are constructed.

[0154] Optionally, a cross-modal feature fusion module 340 is used to obtain image visual feature coding, text semantic feature coding and handwriting feature coding, perform feature alignment and synchronization processing on the three feature codings, and merge the aligned and synchronized feature codings; wherein, cross-attention is performed between the image visual feature and the text semantic feature, and between the image visual feature and the handwriting feature;

[0155] The merged feature codes are then processed for feature enhancement.

[0156] Optionally, the decision integration module 350 is used to integrate different machine learning algorithms into the multi-classifier joint voting decision integration mechanism;

[0157] Train each classifier model independently so that each classifier model learns the mapping from input features to predict the target variable;

[0158] A joint voting mechanism of soft voting is adopted, the probability prediction of each classifier output is considered, the predicted probability of each category is weighted averaged, and the category with the highest average probability is selected to obtain the prediction result.

[0159] In the embodiment of the present invention, the cross-modal model system for the fusion of handwriting features and text semantic features can be applied to the fields of education such as student mental health monitoring, medical fields such as clinical psychological auxiliary assessment, remote mental health monitoring, etc., workplaces such as corporate employee mental health auxiliary management, judicial fields such as criminal psychology analysis, criminal psychology, etc., personal health management such as self-mental health auxiliary monitoring; by combining a variety of technical means such as handwriting psychology, text semantic analysis and image visual analysis, a comprehensive assessment of the individual's mental health status is achieved. This method not only improves the accuracy and reliability of mental health identification, but also provides a scientific basis for mental health intervention, which helps public health institutions to formulate effective mental health intervention strategies. With the continuous advancement of technology, it is believed that this method will play an increasingly important role in the field of mental health assessment and intervention, and contribute to improving the overall mental health level of society.

[0160] Figure 3 is a schematic diagram of the structure of a mental health intelligent assessment device based on text feature fusion provided by an embodiment of the present invention, such as Figure 7 As shown, the mental health intelligent assessment device based on text feature fusion may include the above Figure 6The mental health intelligent assessment system based on text feature fusion is shown. Optionally, the mental health intelligent assessment device 410 based on text feature fusion may include a first processor 2001 .

[0161] Optionally, the mental health intelligent assessment device 410 based on text feature fusion may also include a memory 2002 and a transceiver 2003 .

[0162] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0163] Combine the following Figure 7 The components of the mental health intelligent assessment device 410 based on text feature fusion are specifically introduced as follows:

[0164] The first processor 2001 is the control center of the mental health intelligent assessment device 410 based on text feature fusion, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).

[0165] Optionally, the first processor 2001 can perform various functions of the mental health intelligent assessment device 410 based on text feature fusion by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.

[0166] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 7 CPU0 and CPU1 are shown in FIG.

[0167] In a specific implementation, as an embodiment, the mental health intelligent assessment device 410 based on text feature fusion may also include multiple processors, such as Figure 7The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0168] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0169] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be accessed through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0170] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0171] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 7 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0172] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 7 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0173] It should be noted that Figure 7 The structure of the mental health intelligent assessment device 410 based on text feature fusion shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0174] In addition, the technical effects of the mental health intelligent assessment device 410 based on text feature fusion can refer to the technical effects of the mental health intelligent assessment method based on text feature fusion described in the above method embodiment, which will not be repeated here.

[0175] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0176] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0177] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0178] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0179] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0180] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0181] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.

[0182] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An intelligent mental health assessment method based on text feature fusion, It is characterized in that The method is applicable to a mental health intelligent assessment system based on text feature fusion, the system comprising: a handwriting data acquisition module, a preprocessing module, a multimodal feature extraction module, a cross-modal feature fusion module, a decision integration module and a classification result assessment module; The method comprises: S1, collecting the handwriting image data of the tester, obtaining a handwriting image data set and a text semantic data set; preprocessing the handwriting image data set and the text semantic data set by the preprocessing module, and extracting the content information of the handwritten text; S2. Use the improved Bert model to extract semantic features from the preprocessed text semantic data set; use the improved MobileVit model to extract visual features from the preprocessed handwriting image data set; use the handwriting feature extractor to extract handwriting features from the handwriting image data set; obtain the features of different modalities after extraction; S3, input the features of different modalities into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data that integrates the domain knowledge of vision, semantics, and handwriting psychology; S4, through multiple classifiers, make classification decisions on the fused features, and integrate the prediction results of each classifier through a soft voting strategy; S5. Analyze the multi-dimensional characteristics of the prediction results to obtain the user's mental health assessment results; collect psychological questionnaire data to verify and evaluate the user's mental health assessment results.

2. According to the method for intelligent assessment of mental health based on text feature fusion according to claim 1, It is characterized in that In S1, the handwriting image data of the tester is collected to obtain a handwriting image dataset and a text semantic dataset; Preprocessing the handwriting image data set and the text semantic data set by the preprocessing module to extract content information of the handwritten text includes: Obtaining the tester's handwritten notes, wherein the contents of the handwritten notes are: covering all aspects of the tester's daily life and work; The number of words collected for each topic shall not be less than 50; the collection environment factors shall be consistent; Obtaining a psychological status-related questionnaire filled out by the test subject, and extracting the evaluation results of the questionnaire; Based on the collected handwritten notes of the tester and the questionnaire evaluation results, a handwriting image dataset and a text semantic dataset are obtained; The handwriting image data set and the text semantic data set are preprocessed by the preprocessing module, and the content information of the handwriting text is extracted by the optical character recognition (OCR) technology.

3. According to the method for intelligent assessment of mental health based on text feature fusion according to claim 2, It is characterized in that In S1, the handwriting image dataset and the text semantic dataset are preprocessed, and the content information of the handwritten text is extracted by optical character recognition (OCR) technology, including: Preprocess the handwriting image dataset and text semantic dataset; The preprocessing of the handwriting image data set includes: image grayscale processing, image noise processing, binarization processing and image rotation correction processing; The preprocessing of the text semantic data set includes: text cleaning processing, optical character recognition (OCR) processing, word segmentation processing and natural language processing.

4. According to claim 3, the method for intelligent assessment of mental health based on text feature fusion, It is characterized in that S2 uses an improved Bert model to extract semantic features from a text semantic dataset, including: Load the pre-trained improved BERT model and its corresponding vocabulary; Convert the text sequence after word segmentation processing into an input format acceptable to the improved Bert model to obtain input data; the acceptable input format includes input IDs, token type IDs and attention mask format; The input data is passed into the Bert model, and the context representation of each word is output through forward propagation calculation; Extract features of each word from the output context representation; Use the vector corresponding to the CLS tag as the semantic representation of the entire sentence, or obtain the semantic representation of the entire sentence through pooling operation; The extracted semantic features are processed to adapt to the requirements of cross-fusion tasks.

5. According to claim 4, the method for intelligent assessment of mental health based on text feature fusion, It is characterized in that In S2, the improved MobileVit model is used to extract visual features from the handwriting image dataset, including: The preprocessed image is input into the MobileVit model, and features of different scales are extracted through the hybrid architecture of the MobileVit model; a feature vector integrating features at different levels is output.

6. According to claim 5, the method for intelligent assessment of mental health based on text feature fusion, It is characterized in that In S2, the handwriting feature extractor comprises: Based on the handwriting feature definition of handwriting psychology and the comparison idea between handwritten Chinese characters and Chinese characters in the GB2313 standard character library, a single handwritten Chinese character handwriting feature extractor, a line of handwritten Chinese character handwriting feature extractor, and a multi-line handwritten Chinese character handwriting feature extractor are constructed.

7. The method for intelligent assessment of mental health based on text feature fusion according to claim 6, It is characterized in that In S3, the features of different modalities are input into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data integrating the domain knowledge of vision, semantics and handwriting psychology, including: Obtaining the image visual feature coding, text semantic feature coding and handwriting feature coding, aligning and synchronizing the three feature codings, and merging the aligned and synchronized feature codings; wherein, cross-attention is paid between the image visual feature and the text semantic feature, and between the image visual feature and the handwriting feature; The merged feature codes are then processed for feature enhancement.

8. According to claim 7, the method for intelligent assessment of mental health based on text feature fusion, It is characterized in that In S4, multiple classifiers are used to make classification decisions on the fused features, and the prediction results of each classifier are integrated through a soft voting strategy, including: Integrate different machine learning algorithms into the multi-classifier joint voting decision integration mechanism; Train each classifier model independently so that each classifier model learns the mapping from input features to predict the target variable; A joint voting mechanism of soft voting is adopted, the probability prediction of each classifier output is considered, the predicted probability of each category is weighted averaged, and the category with the highest average probability is selected to obtain the prediction result.

9. A mental health intelligent assessment system based on text feature fusion, the mental health intelligent assessment system based on text feature fusion is used to implement the mental health intelligent assessment method based on text feature fusion as claimed in any one of claims 1 to 8, It is characterized in that The system comprises: A handwriting data collection module is used to collect the tester's handwriting image data and obtain a handwriting image data set and a text semantic data set; A preprocessing module is used to preprocess the handwriting image dataset and the text semantic dataset to extract the content information of the handwritten text; The multimodal feature extraction module is used to extract semantic features from the preprocessed text semantic data set using the improved Bert model; to extract visual features from the preprocessed handwriting image data set using the improved MobileVit model; to extract handwriting features from the handwriting image data set using the handwriting feature extractor; and to obtain features of different modalities after extraction; A cross-modal feature fusion module is used to input features of different modalities into the cross-modal fusion module for comprehensive processing to obtain comprehensive feature data that integrates domain knowledge of vision, semantics, and handwriting psychology; The decision integration module uses multiple classifiers to make classification decisions on the fused features and integrates the prediction results of each classifier through a soft voting strategy; The classification result evaluation module is used to analyze the multi-dimensional characteristics of the prediction results to obtain the user's mental health evaluation results; collect psychological questionnaire data to verify and evaluate the user's mental health evaluation results.

10. A mental health intelligent assessment device based on text feature fusion, It is characterized in that The mental health intelligent assessment device based on text feature fusion includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

Images