Artificial intelligence psychological assessment method and system based on multi-modal input

Abstract

The application discloses an artificial intelligence psychological evaluation method and system based on multi-modal input, and belongs to the technical field of artificial intelligence, and solves the problem that the existing system does not consider individual differences of the tested person to generate a customized evaluation strategy, resulting in insufficient evaluation targeting of the tested person, the method comprises the following steps: generating a personalized evaluation psychological evaluation strategy of the tested person based on basic information of the tested person, collecting multi-modal evaluation information of the tested person in real time, and identifying and analyzing a comprehensive state characteristic vector based on a pre-constructed psychological evaluation model; the application utilizes basic information to construct an initial feature portrait, uses test appeal as a constraint, adopts a collaborative attention network to generate strategy demand information, and individually adjusts item priorities based on an adjusted feature portrait, so that an evaluation strategy that adapts to individual cognitive preferences and emotional characteristics is generated, different individual differences can be adapted to, and implicit and defensive psychological problems of the tested person can be accurately captured, and the accuracy and efficiency of psychological evaluation are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an artificial intelligence psychological assessment method and system based on multimodal input. Background Technology

[0002] With rapid societal development and an accelerated pace of life, mental health issues are becoming increasingly prominent, with a profound impact on individuals and society. Traditional psychological assessment methods mainly utilize subjective scales, behavioral observation and interviews, and computerized psychological testing. Among these, subjective scales are currently the most mainstream and widely used method. However, test takers may provide inaccurate answers due to concealment, self-perception bias, or a desire to present a positive image. Furthermore, subjective scales are less suitable for children, the elderly, or individuals with language impairments. Behavioral observation and interviews, on the other hand, are conducted by professional psychological counselors or psychiatrists. Observing the behavior, facial expressions, and body language of test subjects, and combining this with structured or unstructured interviews for assessment, requires a significant amount of time and human resources for a single complete assessment. This makes it impossible to meet the needs of large-scale screening and difficult to monitor subjects continuously for extended periods in natural settings. With the development of computer technology, online assessment platforms that digitize paper scales have emerged, which has improved the efficiency of administration and the convenience of data management to some extent. However, their core principle is still based on self-reported questions and answers, so they have not fundamentally overcome the inherent defects of traditional scale methods and still face issues such as subjectivity and social expectations.

[0003] Chinese patent CN116705212B discloses a psychological assessment method and system based on the Internet of Things (IoT). The IoT-based psychological assessment system includes a page setting unit, an IoT device, a prediction unit, a page adjustment unit, and a determination unit. This system adjusts the psychological assessment questions displayed on the page to the current test taker based on predictions of the attention focus coordinate range and the credibility of the answers to the psychological assessment questions. This allows for the repeated display of some psychological assessment questions instead of all of them, thus at least solving the technical problem of showing all psychological assessment questions to the same test taker twice. However, existing systems adjust the question display based on attention coordinates and answer credibility without considering individual differences in test takers to generate customized assessment strategies. Because different test takers exhibit specific psychological states, simple question adjustments cannot adapt to deeper needs, resulting in insufficient targeted assessment and difficulty in capturing latent or defensive psychological problems. To address these issues, we propose an artificial intelligence psychological assessment method and system based on multimodal input. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing an artificial intelligence psychological assessment method and system based on multimodal input. This solves the problems of existing systems adjusting the display of test questions based on attention coordinates and answer credibility, failing to consider individual differences among test takers to generate customized assessment strategies, and failing to adapt to deeper needs due to the specific psychological states of different test takers. This results in insufficient targeted assessment for test takers and difficulty in capturing latent or defensive psychological problems.

[0005] This invention is implemented as follows: an artificial intelligence psychological assessment method based on multimodal input, the method comprising:

[0006] Collect basic information about the test subjects and generate personalized psychological assessment strategies based on that information.

[0007] The system collects multimodal assessment information from test subjects in real time during the assessment based on personalized psychological assessment strategies, performs noise suppression preprocessing on the multimodal assessment information, and outputs multimodal alignment information with a unified time granularity.

[0008] Multimodal alignment information is obtained, mapped to a cross-domain feature association network, and coupled and associated with the multimodal alignment information in the cross-domain feature association network to obtain a comprehensive state feature vector that represents the overall state of the test subject.

[0009] Based on a pre-built psychological assessment model, the comprehensive state feature vector is identified and analyzed to generate psychological assessment results, including risk level and psychological state type. Based on the psychological assessment results, a psychological assessment report for the test subject is generated.

[0010] Preferably, the method for generating personalized psychological assessment strategies based on the test subject's basic information includes:

[0011] Obtain basic information about the test subjects and construct an initial feature profile representing the attributes and characteristics of the test subjects based on the basic information. The initial feature profile is constructed based on the test subjects' age, gender, educational background, occupation type, cognitive preference characteristics, and emotional characteristics.

[0012] Load the initial feature profile of the test subject, and adjust the attribute feature weights in the initial feature profile according to the test subject's testing needs to obtain the adjusted feature profile after feature weight adjustment. Use the adjusted feature profile as the strategy trigger engine, and convert the adjusted feature profile into standard strategy requirement information through a collaborative attention network.

[0013] Obtain strategy requirement information, use strategy requirement information as index to traverse the multimodal strategy assessment question bank, trigger the push of at least one set of standard assessment strategies, and map the standard assessment strategies and strategy requirement information into structured vectors and strategy requirement vectors, respectively.

[0014] Obtain the structured vector corresponding to the standard evaluation strategy, calculate the weighted similarity between the structured vector and the strategy requirement vector using the weighted Hellinger-cosine product similarity function, and use the standard evaluation strategy corresponding to the highest weighted similarity as the candidate standard evaluation strategy.

[0015] The weighted similarity calculation formula between the structured vector and the strategy requirement vector is expressed as follows:

[0016]

[0017] in, Represents structured vectors With strategy demand vector Weighted similarity, Structured vectors With strategy demand vector Cosine similarity, structured vectors With strategy demand vector Hellinger similarity, This represents the attribute feature weights in the initial feature profile;

[0018]

[0019]

[0020] in, It is a very small constant;

[0021] The standard assessment strategy is loaded with candidates. Based on the adjusted feature profile after feature weight adjustment, the priority of multimodal items in the standard assessment strategy is adjusted to obtain the personalized assessment strategy for the test taker.

[0022] Preferably, the method for prioritizing multimodal items in the standard assessment strategy based on the adjusted feature profile after feature weight adjustment includes:

[0023] Candidate standard assessment strategies are obtained, multimodal items in the standard assessment strategies are identified, and they are parsed based on a sparse Transformer architecture with embedded attention mechanism to obtain item multimodal features including modality type attributes, cognitive level, emotional arousal intensity, assessment target contribution rate, and assessment time consumption.

[0024] Load the multimodal features of the multimodal items, construct a personalized priority scoring function based on the adjusted feature profile after feature weight adjustment, and score the priority of the multimodal features of the items through the personalized priority scoring function to obtain the item priority value of the multimodal features of the items;

[0025] Based on the priority values ​​of the multimodal features of the items, the multimodal items in the standard assessment strategy are sorted in descending order to obtain the initial adjustment assessment strategy;

[0026] An initial adjustment assessment strategy is obtained. A sequential adjustment reward function is constructed with constraints of minimizing modal type mutual exclusion, maximizing modal equilibrium, and minimizing time budget. An improved NSGA-III algorithm is used to solve the sequential adjustment reward function to generate a Pareto front solution set. The optimal solution is selected as the final personalized assessment psychological assessment strategy.

[0027] Preferably, the method for noise suppression preprocessing of multimodal evaluation information includes:

[0028] The priority values ​​and modality type attributes of multimodal items in the personalized assessment psychological testing strategy of the test subjects are retrieved. Based on the improved Relief algorithm, the priority values ​​and modality type attributes of multimodal items are coupled to obtain the filtering priority of multimodal items.

[0029] The formula for calculating the filtering priority of multimodal items is as follows:

[0030]

[0031]

[0032] in, Representing multimodal items Filtering priority These are multimodal items. Item priority values, all multimodal items Item priority average, all multimodal items Standard deviation of item preference values These are multimodal items. The modality type attribute weights, the mean of the modality type attribute weights, and the standard deviation of the modality type attribute weights. These are the priority values ​​for the question item and the priority coefficients for the modal type attribute, respectively. This is a correction factor;

[0033] Acquire multimodal assessment information, determine whether the filtering priority of the multimodal items corresponding to the multimodal assessment information exceeds the preset filtering threshold, and perform adaptive noise suppression on multimodal assessment information with different filtering priorities based on the submodal noise suppression mechanism;

[0034] If the filtering priority of the multimodal items does not exceed the preset filtering threshold, the first filter is used to adaptively filter the multimodal evaluation information to obtain low-priority modal information.

[0035] If the filtering priority of a multimodal item exceeds the preset filtering threshold, a second filter is used to adaptively filter the multimodal assessment information to obtain high-priority modal information.

[0036] Low-priority modal information and high-priority modal information are loaded. The physiological signal stream time axis in the multimodal evaluation information is used as the master clock. The speech signal stream is inserted into the master clock using a phase vocoder. The text stream and video signal stream are inserted into the master clock using linear interpolation to obtain multimodal alignment information with uniform time granularity.

[0037] Preferably, the first filter includes a first filter input layer, a first filter parallel module, and a first filter output layer. The first filter parallel module includes a Jieba layer and a blind source separation layer based on accelerometer data, which are configured in parallel. The second filter includes a second filter input layer, a second filter parallel module, and a second filter output layer. The second filter input layer is connected to the second filter parallel module, and the second filter parallel module is connected to the second filter output layer. The second filter parallel module includes an NLMS adaptive filtering layer and an MTCNN layer, which are configured in parallel. The NLMS adaptive filtering layer and the MTCNN layer are respectively connected to the U-Net deep learning segmentation layer.

[0038] Preferably, the method for coupling and associating multimodal alignment information within a cross-domain feature association network includes:

[0039] The multimodal alignment information is obtained and mapped to the modal cross-projection layer of the cross-domain feature association network. During the mapping process, the multimodal alignment information is subjected to nonlinear transformation and information compression.

[0040] A bidirectional cross-attention mechanism is employed to perform bidirectional interaction on the multimodal alignment information within the modal cross-projection layer, thereby extracting the context vector of the multimodal alignment information.

[0041] The context vector of the multimodal alignment information is represented as follows:

[0042]

[0043]

[0044]

[0045] in, These are the current modality information and the interaction modality information in the multimodal alignment information, respectively. These are the interaction information vectors of the current modal information relative to the interaction modal information, and the interaction information vectors of the interaction modal information relative to the current modal information, respectively. These are the Query matrix obtained by linear mapping of the current modality information and the Query matrix obtained by linear mapping of the interaction modality information, respectively. These are the Key matrices obtained by linear mapping of the current modality information and the Key matrices obtained by linear mapping of the interaction modality information, respectively. These are the Value matrix obtained by linear mapping of the current modality information and the Value matrix obtained by linear mapping of the interaction modality information, respectively. The vector length representing the Query matrix;

[0046] Information aggregation is performed on the multimodal alignment information and context vector, and the aggregated multimodal alignment information is decomposed into low-rank Tucker decomposition to obtain the comprehensive state feature vector representing the overall state of the test subject.

[0047] Preferably, when the psychological assessment model is pre-built, it is based on a graph neural network model. The psychological assessment model also includes an assessment input layer and an assessment output layer. A multimodal attention mechanism network is set between the assessment input layer and the graph neural network model. The multimodal attention mechanism network includes front-end processing embedding layers, multi-head cross-modal attention layers, and a Transformer classification head. The graph neural network model includes a mental state graph builder, a three-layer GNN layer, an MLP layer, and a conditional cross-attention layer. A saliency analysis layer is set between the graph neural network model and the assessment output layer.

[0048] Preferably, the method for identifying and analyzing comprehensive state feature vectors based on a pre-constructed psychological assessment model includes:

[0049] Obtain the comprehensive state feature vector that represents the overall state of the test subject, and dynamically calculate the label weights of multimodal features in the comprehensive state feature vector based on the multi-head attention mechanism;

[0050] The formula for calculating the label weights of multimodal features is as follows:

[0051]

[0052]

[0053] in, Representing multimodal features Tag weight, The number of attention heads in a multi-head attention mechanism. Indicates the current attention head Label weights of mid-modal features These represent the dimensions of the multimodal feature query projection matrix, the multimodal feature key projection matrix, and the multimodal feature key projection matrix, respectively. Indicates matrix transpose. These are the label embedding vector and the multimodal feature vector, respectively.

[0054] Based on the Transformer classification head, the label weights among the multimodal features are weighted and fused to generate a coarse classification risk level for evaluating the comprehensive state feature vector;

[0055] Obtain the coarse classification level corresponding to the comprehensive state feature vector, generate a feature enhancement mask based on the coarse classification level, create a fine multi-label classification based on the conditional cross attention mechanism, and perform dimensional logistic regression scoring on the multi-label classification results;

[0056] The system uses dimensional logistic regression scores as the risk level assessment for test subjects, outputting the risk level assessment of test subjects. Based on the gradient class significance analysis tool XGBoost model, the system compares the fine multi-label classification with the preset psychological state classification interval, outputting the psychological state type of the test subjects. Based on the psychological assessment results, the system generates a psychological assessment report for the test subjects.

[0057] On the other hand, the present invention also provides an artificial intelligence psychological assessment system based on multimodal input, the artificial intelligence psychological assessment system based on multimodal input comprising:

[0058] The personalized strategy generation module is used to collect basic information of test subjects and generate personalized psychological assessment strategies based on the basic information of test subjects.

[0059] The multimodal information acquisition module is used to collect multimodal assessment information of test subjects in real time based on personalized assessment psychological assessment strategies, perform noise suppression preprocessing on the multimodal assessment information, and output multimodal alignment information with uniform time granularity.

[0060] The information coupling and association module is used to acquire multimodal alignment information, map the multimodal alignment information to the cross-domain feature association network, and couple and associate the multimodal alignment information in the cross-domain feature association network to obtain a comprehensive state feature vector that represents the overall state of the test subject.

[0061] The assessment module identifies and analyzes the comprehensive state feature vector based on a pre-built psychological assessment model, generates psychological assessment results including risk level and psychological state type, and generates a psychological assessment report for the test subject based on the psychological assessment results.

[0062] Preferably, the personalized strategy generation module includes:

[0063] The profile generation unit is used to acquire the basic information of the test subject and construct an initial feature profile representing the attribute characteristics of the test subject based on the basic information of the test subject;

[0064] The strategy requirement acquisition unit is used to load the initial feature profile of the test subject, adjust the attribute feature weights in the initial feature profile based on the test subject's testing needs, and obtain the adjusted feature profile after feature weight adjustment. The adjusted feature profile is used as the strategy triggering engine, and the adjusted feature profile is converted into standard strategy requirement information through a collaborative attention network.

[0065] The standard strategy push unit is used to obtain strategy requirement information. Using the strategy requirement information as an index, it traverses the multimodal strategy assessment question bank. The multimodal strategy assessment question bank triggers the push of at least one set of standard assessment strategies. The standard assessment strategies and strategy requirement information are mapped into structured vectors and strategy requirement vectors, respectively.

[0066] The standard strategy screening unit is used to obtain the structured vector corresponding to the standard evaluation strategy. The weighted similarity between the structured vector and the strategy requirement vector is calculated using the weighted Hellinger-cosine product similarity function. The standard evaluation strategy corresponding to the highest weighted similarity is selected as the candidate standard evaluation strategy.

[0067] The standard strategy adjustment unit is used to load candidate standard assessment strategies. Based on the adjusted feature profile after feature weight adjustment, the multimodal items in the standard assessment strategy are prioritized to obtain a personalized psychological assessment strategy for the test taker.

[0068] Compared with the prior art, the embodiments of this application have the following main advantages:

[0069] In this embodiment of the invention, an initial feature profile is constructed using basic information. Constrained by testing requirements, a collaborative attention network is used to generate strategy requirement information. Based on the adjusted feature profile, the priority of questions is personalized, thereby generating an assessment strategy that adapts to individual cognitive preferences and emotional characteristics. This strategy can accommodate different individual differences, accurately capturing the test-taker's latent and defensive psychological problems, improving the accuracy and efficiency of psychological assessment. Furthermore, the assessment process is guaranteed to be completed in one go, eliminating the need for repeated question answering. Modal equalization optimization reduces video / physical wearing time, lowering the test-taker's workload. Ultimately, this embodiment of the invention can be directly applied to psychological assessment scenarios in schools, hospitals, enterprises, and other settings.

[0070] In this embodiment of the invention, when generating a personalized psychological assessment strategy based on the test subject's basic information, an initial profile is constructed using the test subject's age, gender, educational background, occupation type, cognitive preferences, and emotional characteristics. The attribute weights are dynamically adjusted in conjunction with the test requirements, and then converted into strategy requirements through a collaborative attention network. Candidate strategies are then accurately screened through structured vector mapping and weighted Hellinger-cosine product similarity. Finally, based on the adjusted profile, the priority of multimodal items is optimized to achieve a leap from a general template to personalized customization in the assessment strategy, significantly improving personalization adaptability, assessment accuracy, and efficiency.

[0071] In this embodiment of the invention, by identifying the modal type attributes of the items and constructing a scoring function in conjunction with the adjusted feature profile, the items are accurately scored and ranked. Furthermore, the improved NSGA-III algorithm can efficiently search for multi-objective optimal solutions, thereby generating a Pareto front solution set with high priority total score and few constraint violations. Then, combined with the real-time feedback from the test takers, a balanced optimal sequence is selected from the solution set, so that the item sequence is adjusted from single-dimensional priority ranking to multi-objective collaborative optimization. This significantly improves the test takers' assessment experience and enhances the integrity of modal data, systematically solving the shortcomings of traditional item ranking that emphasizes individual items over collaboration and commonalities over individual characteristics.

[0072] In this embodiment of the invention, the improved Relief algorithm couples the items with modal attributes, avoiding over-processing of low-value items or insufficient filtering of high-value items. Furthermore, it performs modal adaptive filtering on multimodal assessment information, ensuring the quality of high-value modal data while reducing the noise reduction cost of low-value modal data. This provides a clean, synchronous, and high-value data foundation for subsequent accurate identification of psychological states and risk assessment.

[0073] In this embodiment of the invention, the psychological assessment model is based on a graph neural network model and incorporates a multimodal attention mechanism network and a saliency analysis layer. The graph neural network model can represent the psychological state of the test subject in the form of a graph, thereby more intuitively displaying the psychological state of the test subject. Based on the coarse classification level, a feature enhancement mask is generated, and a fine multi-label classification is built through cross-attention, realizing hierarchical feature enhancement and fine multi-label classification, ensuring the reliability of the risk level quantification and subdivision results. The saliency analysis layer can clearly display the decision-making process of the model based on saliency analysis, enabling the test subject to better understand the output of the model. Attached Figure Description

[0074] Figure 1 A schematic diagram illustrating the implementation process of an AI-based psychological assessment method based on multimodal input is shown.

[0075] Figure 2 This diagram illustrates the implementation process of a psychological assessment strategy that generates personalized assessments based on the test subject's basic information.

[0076] Figure 3 This paper illustrates the implementation process of a method for prioritizing multimodal items in a standard assessment strategy based on an adjusted feature profile with adjusted feature weights.

[0077] Figure 4 A schematic diagram of the implementation process of a noise suppression preprocessing method for multimodal evaluation information is shown.

[0078] Figure 5 A schematic diagram of the implementation process of the method for coupling and associating multimodal alignment information in a cross-domain feature association network is shown.

[0079] Figure 6 The diagram illustrates the implementation process of a method for identifying and analyzing comprehensive state feature vectors based on a pre-built psychological assessment model.

[0080] Figure 7 The diagram shows the architecture of an AI-based psychological assessment system based on multimodal input. Detailed Implementation

[0081] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0082] Existing systems adjust test item presentation based on attention coordinates and answer credibility, failing to consider individual differences among test takers to generate customized assessment strategies. Because different test takers exhibit specific psychological states, simple test item adjustments cannot meet deeper needs, resulting in insufficient targeted assessment and difficulty in capturing latent or defensive psychological problems. To address these issues, we propose an AI-based psychological assessment method and system based on multimodal input. In short, the method first generates a personalized psychological assessment strategy based on the test taker's basic information. Then, it collects multimodal assessment information in real-time during the test based on the personalized strategy. This information undergoes noise suppression preprocessing. Next, the multimodal alignment information is mapped into a cross-domain feature association network. The multimodal alignment information within this network is then coupled and correlated to obtain a comprehensive state feature vector representing the test taker's overall state. Finally, based on a pre-constructed psychological assessment model, the comprehensive state feature vector is identified and analyzed to generate psychological assessment results, including risk level and psychological state type. A psychological assessment report is then generated based on these results. In this embodiment of the invention, an initial feature profile is constructed using basic information. Constrained by testing requirements, a collaborative attention network is used to generate strategy requirement information. Based on the adjusted feature profile, the priority of questions is personalized, thereby generating an assessment strategy that adapts to individual cognitive preferences and emotional characteristics. This strategy can accommodate different individual differences, accurately capturing the test-taker's latent and defensive psychological problems, improving the accuracy and efficiency of psychological assessment. Furthermore, the assessment process is guaranteed to be completed in one go, eliminating the need for repeated question answering. Modal equalization optimization reduces video / physical wearing time, lowering the test-taker's workload. Ultimately, this embodiment of the invention can be directly applied to psychological assessment scenarios in schools, hospitals, enterprises, and other settings.

[0083] This invention provides an artificial intelligence-based psychological assessment method based on multimodal input. Figure 1 A schematic diagram illustrating the implementation process of an AI-based psychological assessment method based on multimodal input is shown. This method specifically includes:

[0084] S10. Collect basic information of the test subjects and generate personalized psychological assessment strategies based on the basic information of the test subjects. The test subjects can be preschool children, students, new professionals / white-collar workers / senior executives, special industry personnel, the elderly, pregnant women, and clinical subgroups. The basic information of the test subjects includes, but is not limited to, the test subjects' age, gender, ethnicity, nationality / region, marital status, height and weight, education level (current grade / highest degree), occupation type (national standard GB / T 6565), job stress level, income range, social security type, history of mental / neurological diseases, family history, list of chronic diseases, current medication (psychiatric / chronic drugs), smoking and drinking amount, sleep rhythm type (morning / night), assessment scenario (school / company / hospital / home), assessment time (morning / afternoon / after night shift), noise environment (dB), network type, equipment level, name, relationship, and telephone number.

[0085] S20: Real-time acquisition of multimodal assessment information of test subjects during the assessment based on personalized psychological assessment strategies; noise suppression preprocessing of multimodal assessment information; outputting multimodal alignment information with uniform time granularity.

[0086] S30, obtain multimodal alignment information, map the multimodal alignment information into the cross-domain feature association network, couple and associate the multimodal alignment information in the cross-domain feature association network, and obtain the comprehensive state feature vector that represents the comprehensive state of the test subject;

[0087] S40 identifies and analyzes the comprehensive state feature vector based on a pre-built psychological assessment model, generates psychological assessment results including risk level and psychological state type, and generates a psychological assessment report for the test subject based on the psychological assessment results.

[0088] In this embodiment, the psychological assessment report may also include the test subject's basic information, coarse risk level, detailed psychological state type, psychological dimension profile, personalized intervention suggestions, and privacy and data authorization.

[0089] In this embodiment of the invention, an initial feature profile is constructed using basic information, and the strategy requirement information is generated by using a collaborative attention network under the constraint of testing requirements. The priority of the items is then adjusted in a personalized manner based on the adjusted feature profile, thereby generating an assessment strategy that is adapted to individual cognitive preferences and emotional characteristics. This strategy can adapt to different individual differences, thereby accurately capturing the hidden and defensive psychological problems of the test subjects and improving the accuracy and efficiency of psychological assessment.

[0090] Considering that traditional assessment techniques rely solely on simple demographic information of test subjects and fail to account for individual differences, this invention provides a method for generating personalized psychological assessment strategies based on basic test subject information. Figure 2This diagram illustrates the implementation flow of a method for generating personalized psychological assessment strategies based on the basic information of test subjects. Specifically, this method includes:

[0091] S101. Obtain the basic information of the test subject and construct an initial feature profile representing the attribute characteristics of the test subject based on the basic information of the test subject. The initial feature profile is constructed based on the test subject's age, gender, educational background, occupation type, cognitive preference characteristics, and emotional characteristics. The initial feature profile of the test subject is constructed with six core attributes: age, gender, educational background, occupation type, cognitive preference characteristics, and emotional characteristics. This achieves full-dimensional coverage of the test subject's static attributes (age / occupation) and dynamic traits (cognitive preference / emotional characteristics), thereby providing a comprehensive and three-dimensional individual profile for subsequent strategy adjustments.

[0092] S102, load the initial feature profile of the test subject, adjust the attribute feature weights in the initial feature profile based on the test subject's testing needs, and obtain the adjusted feature profile after feature weight adjustment. Use the adjusted feature profile as the strategy trigger engine, and convert the adjusted feature profile into standard strategy requirement information through a collaborative attention network. The constraint of the test subject's testing needs ensures that strategy generation changes from system-led to test subject-system collaboration, thereby ensuring that the evaluation content is highly matched with the test subject's real needs.

[0093] S103, obtain strategy requirement information, use strategy requirement information as index to traverse the multimodal strategy assessment question bank, trigger the push of at least one set of standard assessment strategies, and map the standard assessment strategies and strategy requirement information into structured vectors and strategy requirement vectors respectively.

[0094] S104. Obtain the structured vector corresponding to the standard assessment strategy. Calculate the weighted similarity between the structured vector and the strategy requirement vector using the weighted Hellinger-cosine product similarity function. Select the standard assessment strategy corresponding to the highest weighted similarity as the candidate standard assessment strategy. The standard assessment strategy is a baseline template containing multimodal item metadata vectors. Multimodal items include, but are not limited to, text modality, voice modality, visual scene modality, video-micro-expression modality, and physiological modality. Assessment dimensions include, but are not limited to, depression-anhedonia, social anxiety, trauma cueing, cognitive preference, attention deficit, emotion regulation, loneliness, suicide risk, burnout, and internet addiction.

[0095] The weighted similarity calculation formula between the structured vector and the strategy requirement vector is expressed as follows:

[0096]

[0097] in, Represents structured vectors With strategy demand vector Weighted similarity, Structured vectors With strategy demand vector Cosine similarity, structured vectors With strategy demand vector Hellinger similarity, This represents the attribute feature weights in the initial feature profile;

[0098]

[0099]

[0100] in, It is a very small constant;

[0101] S105, Load the candidate standard assessment strategy, and adjust the priority of the multimodal items in the standard assessment strategy based on the adjusted feature profile after feature weight adjustment, so as to generate a personalized assessment strategy for the test taker, thereby avoiding the test taker's frustration or single-modality fatigue caused by doing the difficult items first and then the easy items.

[0102] In this embodiment of the invention, when generating a personalized psychological assessment strategy based on the test subject's basic information, an initial profile is constructed using the test subject's age, gender, educational background, occupation type, cognitive preferences, and emotional characteristics. The attribute weights are dynamically adjusted in conjunction with the test requirements, and then converted into strategy requirements through a collaborative attention network. Candidate strategies are then accurately screened through structured vector mapping and weighted Hellinger-cosine product similarity. Finally, based on the adjusted profile, the priority of multimodal items is optimized to achieve a leap from a general template to personalized customization in the assessment strategy, significantly improving personalization adaptability, assessment accuracy, and efficiency.

[0103] This invention provides a method for prioritizing multimodal items in a standard assessment strategy based on an adjusted feature profile with adjusted feature weights. Figure 3 This document illustrates a flowchart illustrating a method for prioritizing multimodal items in a standard assessment strategy based on an adjusted feature profile with adjusted feature weights. The method specifically includes:

[0104] S1051, obtain candidate standard assessment strategies, identify multimodal items in the standard assessment strategies and parse them based on a sparse Transformer architecture with embedded attention mechanism to obtain item multimodal features including modality type attributes, cognitive level, emotional arousal intensity, assessment target contribution rate and assessment time consumption.

[0105] S1052, Load the multimodal features of the multimodal items, construct a personalized priority scoring function based on the adjusted feature profile after feature weight adjustment, and score the priority of the multimodal features of the items through the personalized priority scoring function to obtain the item priority value of the multimodal features of the items;

[0106] S1053, based on the priority value of the multimodal features of the items, the multimodal items in the standard assessment strategy are sorted in descending order to obtain the initial adjustment assessment strategy;

[0107] S1054: Obtain the initial adjustment assessment strategy. Considering that simply sorting items in descending order of priority value may lead to modality type mutual exclusion, modality imbalance, and time limit exceeding, a sequence adjustment reward function is constructed with constraints of minimizing modality type mutual exclusion, maximizing modality balance, and minimizing time budget. An improved NSGA-III algorithm is used to solve the sequence adjustment reward function to generate a Pareto front solution set. The optimal solution is selected as the final personalized assessment psychological assessment strategy. The improved NSGA-III algorithm can efficiently search for multi-objective optimal solutions, thereby generating a Pareto front solution set with high priority total score and few constraint violations. Then, combined with the real-time feedback of the test takers, a balanced optimal sequence is selected from the solution set, so that the item sequence is adjusted from single-dimensional priority sorting to multi-objective collaborative optimization, which significantly improves the test takers' assessment experience and enhances the integrity of modality data.

[0108] In this embodiment of the invention, by identifying the modal type attributes of the items and constructing a scoring function in conjunction with the adjusted feature profile, the items are accurately scored and ranked. Furthermore, the improved NSGA-III algorithm can efficiently search for multi-objective optimal solutions, thereby generating a Pareto front solution set with high priority total score and few constraint violations. Then, combined with the real-time feedback from the test takers, a balanced optimal sequence is selected from the solution set, so that the item sequence is adjusted from single-dimensional priority ranking to multi-objective collaborative optimization. This significantly improves the test takers' assessment experience and enhances the integrity of modal data, systematically solving the shortcomings of traditional item ranking that emphasizes individual items over collaboration and commonalities over individual characteristics.

[0109] Considering the ease with which noise can arise in multimodal assessment information collected from multiple heterogeneous sources, and the potential for neglecting the differences in importance of different items and the inherent noise characteristics of different modes, this invention provides a noise suppression preprocessing method for multimodal assessment information. Figure 4 This diagram illustrates the implementation flow of a noise suppression preprocessing method for multimodal evaluation information. The noise suppression preprocessing method specifically includes:

[0110] S201: Retrieve the item priority value and modality type attribute of the multimodal items in the personalized assessment psychological assessment strategy of the test subject. Based on the improved Relief algorithm, couple the item priority value and modality type attribute of the multimodal items to obtain the filtering priority of the multimodal items.

[0111] The formula for calculating the filtering priority of multimodal items is as follows:

[0112]

[0113]

[0114] in, Representing multimodal items Filtering priority These are multimodal items. Item priority values, all multimodal items Item priority average, all multimodal items Standard deviation of item preference values These are multimodal items. The modality type attribute weights, the mean of the modality type attribute weights, and the standard deviation of the modality type attribute weights. These are the priority values ​​for the question item and the priority coefficients for the modal type attribute, respectively. This is a correction factor;

[0115] S202, Obtain multimodal evaluation information, determine whether the filtering priority of the multimodal items corresponding to the multimodal evaluation information exceeds the preset filtering threshold, and perform adaptive noise suppression on multimodal evaluation information with different filtering priorities based on the submodal noise suppression mechanism;

[0116] S203, If the filtering priority of the multimodal items does not exceed the preset filtering threshold, the first filter is used to adaptively filter the multimodal evaluation information to obtain low-priority modal information;

[0117] S204. If the filtering priority of the multimodal items exceeds the preset filtering threshold, the second filter is used to adaptively filter the multimodal evaluation information to obtain high-priority modal information.

[0118] The first filter comprises a first filter input layer, a first filter parallel module, and a first filter output layer. The first filter parallel module includes a parallel Jieba layer and a blind source separation layer based on accelerometer data. The first filter can lightly process and remove explicit noise, improving the computational efficiency of non-core items. The second filter comprises a second filter input layer, a second filter parallel module, and a second filter output layer. The second filter input layer is connected to the second filter parallel module, and the second filter parallel module is connected to the second filter output layer. The second filter parallel module includes a parallel NLMS adaptive filtering layer and an MTCNN layer. The NLMS adaptive filtering layer and the MTCNN layer are respectively connected to the U-Net deep learning segmentation layer. The second filter can accurately remove implicit noise (speech reverberation, video illumination interference, and physiological signal electromyography artifacts) through a deep learning model, ensuring the denoising accuracy of core items.

[0119] S205 loads low-priority modal information and high-priority modal information, uses the time axis of the physiological signal stream in the multimodal evaluation information as the main clock, uses a phase vocoder to insert the voice signal stream into the main clock, and uses linear interpolation to insert the text stream and video signal stream into the main clock to obtain multimodal alignment information with uniform time granularity.

[0120] In this embodiment of the invention, the improved Relief algorithm couples the items with modal attributes, avoiding over-processing of low-value items or insufficient filtering of high-value items. Furthermore, it performs modal adaptive filtering on multimodal assessment information, ensuring the quality of high-value modal data while reducing the noise reduction cost of low-value modal data. This provides a clean, synchronous, and high-value data foundation for subsequent accurate identification of psychological states and risk assessment.

[0121] This invention provides a method for coupling and associating multimodal alignment information within a cross-domain feature association network. Figure 5 This diagram illustrates the implementation flow of a method for coupling and associating multimodal alignment information within a cross-domain feature association network. The method specifically includes:

[0122] S301: Obtain multimodal alignment information, map the multimodal alignment information into the modality cross-projection layer of the cross-domain feature association network, and perform nonlinear transformation and information compression processing on the multimodal alignment information during the mapping process. Nonlinear transformation (ReLU function or Tanh function) can capture the complex relationship between modalities, so that the features of different modalities are better aligned in high-dimensional space, while information compression (PCA or autoencoder) reduces redundant information and improves the compactness and representativeness of features.

[0123] S302 employs a bidirectional cross-attention mechanism to perform bidirectional interaction on the multimodal alignment information within the modal cross-projection layer, extracting the context vector of the multimodal alignment information. The context vector extracted through the bidirectional cross-attention mechanism can capture the interrelationship of multimodal information in the global scope. The context vector not only contains the information of the current modality, but also integrates the interaction information of other modalities, making the features of each modality richer and more comprehensive.

[0124] The context vector of the multimodal alignment information is represented as follows:

[0125]

[0126]

[0127]

[0128] in, These are the current modality information and the interaction modality information in the multimodal alignment information, respectively. These are the interaction information vectors of the current modal information relative to the interaction modal information, and the interaction information vectors of the interaction modal information relative to the current modal information, respectively. These are the Query matrix obtained by linear mapping of the current modality information and the Query matrix obtained by linear mapping of the interaction modality information, respectively. These are the Key matrices obtained by linear mapping of the current modality information and the Key matrices obtained by linear mapping of the interaction modality information, respectively. These are the Value matrix obtained by linear mapping of the current modality information and the Value matrix obtained by linear mapping of the interaction modality information, respectively. The vector length representing the Query matrix;

[0129] S303, information aggregation is performed on the multimodal alignment information and context vector, and the aggregated multimodal alignment information is decomposed into low-rank Tucker decomposition to obtain the comprehensive state feature vector representing the overall state of the test subject.

[0130] In this embodiment of the invention, multimodal alignment information and context vector are aggregated, and a comprehensive state feature vector is obtained through low-rank Tucker decomposition. Low-rank decomposition can further compress the feature dimension while retaining the most important information, reducing computational complexity and storage requirements.

[0131] It should be noted that the psychological assessment model is pre-built based on a graph neural network model. The model includes an assessment input layer and an assessment output layer. A multimodal attention mechanism network is established between the assessment input layer and the graph neural network model. This network includes front-end processing embedding layers, a multi-head cross-modal attention layer, and a Transformer classification head. The graph neural network model includes a psychological state graph builder, a three-layer GNN layer, an MLP layer, and a conditional cross-attention layer. A saliency analysis layer is established between the graph neural network model and the assessment output layer. The multimodal attention mechanism network is used to obtain a comprehensive state feature vector representing the overall state of the test taker. Based on the multi-head attention mechanism, the label weights of the multimodal features in the comprehensive state feature vector are dynamically calculated. Based on the Transformer classification head, the label weights among the multimodal features are weighted and fused to generate an assessment comprehensive state feature vector. The coarse classification risk level is obtained by the graph neural network model, which is used to obtain the coarse classification level corresponding to the comprehensive state feature vector. Based on the coarse classification level, a feature enhancement mask is generated, and a fine multi-label classification is created based on the conditional cross-attention mechanism. The multi-label classification results are then scored by dimensional logistic regression, and the dimensional logistic regression score is used as the subject's assessed risk level. The assessed risk level of the subject is then output. The saliency analysis layer compares the fine multi-label classification with the preset psychological state classification interval based on the gradient class saliency analysis tool XGBoost model, and outputs the subject's psychological state type. A psychological assessment report of the subject is generated based on the psychological assessment results.

[0132] In this embodiment of the invention, the psychological assessment model is based on a graph neural network model and incorporates a multimodal attention mechanism network and a saliency analysis layer. The graph neural network model can represent the psychological state of the test subject in the form of a graph, thereby more intuitively displaying the psychological state of the test subject. Based on the coarse classification level, a feature enhancement mask is generated, and a fine multi-label classification is built through cross-attention, realizing hierarchical feature enhancement and fine multi-label classification, ensuring the reliability of the risk level quantification and subdivision results. The saliency analysis layer can clearly display the decision-making process of the model based on saliency analysis, enabling the test subject to better understand the output of the model.

[0133] This invention provides a method for identifying and analyzing comprehensive state feature vectors based on a pre-constructed psychological assessment model. Figure 6 The diagram illustrates the implementation process of a method for identifying and analyzing comprehensive state feature vectors based on a pre-built psychological assessment model. Specifically, this method includes:

[0134] S401, obtain the comprehensive state feature vector representing the overall state of the test subject, and dynamically calculate the label weights of multimodal features in the comprehensive state feature vector based on the multi-head attention mechanism;

[0135] The formula for calculating the label weights of multimodal features is as follows:

[0136]

[0137]

[0138] in, Representing multimodal features Tag weight, The number of attention heads in a multi-head attention mechanism. Indicates the current attention head Label weights of mid-modal features These represent the dimensions of the multimodal feature query projection matrix, the multimodal feature key projection matrix, and the multimodal feature key projection matrix, respectively. Indicates matrix transpose. These are the label embedding vector and the multimodal feature vector, respectively.

[0139] S402, based on the Transformer classification head, performs label weighted fusion on the label weights among multimodal features to generate a coarse classification risk level for evaluating the comprehensive state feature vector, where the coarse classification risk level can be high risk, medium risk, or low risk;

[0140] S403, obtain the coarse classification level corresponding to the comprehensive state feature vector, generate a feature enhancement mask based on the coarse classification level, and create a fine multi-label classification based on the conditional cross-attention mechanism. Then, perform dimensional logistic regression scoring on the multi-label classification results. The fine multi-label can be labeled as "anxiety (0.4), mild depression (0.3), sleep disorder (0.3)".

[0141] S404 uses dimensional logistic regression scores as the risk level assessment for test takers, outputting the risk level. It then uses the XGBoost model, a gradient-based saliency analysis tool, to compare the refined multi-label classification with predefined psychological state classification intervals, outputting the test taker's psychological state type. Based on the psychological assessment results, it generates a psychological assessment report for the test taker. The XGBoost model, as a powerful gradient boosting tree model, can effectively handle classification problems and provide saliency analysis. For example, when assessing patients with depression, the model not only outputs the label "depression" but also uses saliency analysis to indicate that low mood and decreased interest are the main salient features.

[0142] On the other hand, embodiments of the present invention also provide an artificial intelligence psychological assessment system based on multimodal input. Figure 7 The diagram illustrates the architecture of an AI-based psychological assessment system with multimodal input. This system specifically includes:

[0143] The personalized strategy generation module 100 is used to collect basic information of the test subjects and generate personalized psychological assessment strategies for the test subjects based on the basic information of the test subjects.

[0144] The personalized strategy generation module 100 includes:

[0145] The portrait generation unit 110 is used to acquire the basic information of the test subject and construct an initial feature portrait representing the attribute characteristics of the test subject based on the basic information of the test subject.

[0146] The strategy requirement acquisition unit 120 is used to load the initial feature profile of the test subject, adjust the attribute feature weights in the initial feature profile based on the test subject's test requirements, and obtain the adjusted feature profile after feature weight adjustment. The adjusted feature profile is used as the strategy triggering engine, and the adjusted feature profile is converted into standard strategy requirement information through a collaborative attention network.

[0147] The standard strategy push unit 130 is used to obtain strategy requirement information, and traverse the multimodal strategy assessment question bank with the strategy requirement information as an index. The multimodal strategy assessment question bank triggers the push of at least one set of standard assessment strategies, and maps the standard assessment strategies and strategy requirement information into structured vectors and strategy requirement vectors, respectively.

[0148] The standard strategy screening unit 140 is used to obtain the structured vector corresponding to the standard evaluation strategy, and to calculate the weighted similarity between the structured vector and the strategy requirement vector using the weighted Hellinger-cosine product similarity function. The standard evaluation strategy corresponding to the highest weighted similarity is used as the candidate standard evaluation strategy.

[0149] The standard strategy adjustment unit 150 is used to load candidate standard assessment strategies. Based on the adjusted feature profile after feature weight adjustment, the multimodal items in the standard assessment strategy are prioritized to obtain a personalized psychological assessment strategy for the test taker.

[0150] The multimodal information acquisition module 200 is used to collect multimodal assessment information of test subjects in real time during the assessment based on personalized assessment psychological assessment strategies, perform noise suppression preprocessing on the multimodal assessment information, and output multimodal alignment information with uniform time granularity.

[0151] The information coupling and association module 300 is used to acquire multimodal alignment information, map the multimodal alignment information to the cross-domain feature association network, and couple and associate the multimodal alignment information in the cross-domain feature association network to obtain a comprehensive state feature vector that represents the comprehensive state of the test subject.

[0152] The assessment module 400 identifies and analyzes the comprehensive state feature vector based on a pre-built psychological assessment model, generates psychological assessment results including risk level and psychological state type, and generates a psychological assessment report for the test subject based on the psychological assessment results.

[0153] In summary, this invention provides an AI-based psychological assessment method and system based on multimodal input. In the embodiments of this invention, an initial feature profile is constructed using basic information, and strategic requirement information is generated using a collaborative attention network constrained by testing needs. Based on the adjusted feature profile, the priority of the items is personalized, thereby generating an assessment strategy that adapts to individual cognitive preferences and emotional characteristics. This strategy can adapt to different individual differences, thereby accurately capturing the latent and defensive psychological problems of the test subjects and improving the accuracy and efficiency of psychological assessment.

[0154] It should be noted that, for the sake of simplicity, the foregoing embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to the present invention. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0155] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on these embodiments, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can still combine, add, delete, or otherwise adjust the features of the various embodiments of the present invention according to the circumstances without conflict or creative effort, thereby obtaining different technical solutions that do not fundamentally depart from the concept of the present invention. These technical solutions also fall within the scope of protection of the present invention.

Claims

1. An artificial intelligence psychological assessment method based on multimodal input, characterized in that, The AI-based psychological assessment method based on multimodal input includes: Collect basic information about the test subjects and generate personalized psychological assessment strategies based on that information. The system collects multimodal assessment information from test subjects in real time during the assessment based on personalized psychological assessment strategies, performs noise suppression preprocessing on the multimodal assessment information, and outputs multimodal alignment information with a unified time granularity. Multimodal alignment information is obtained, mapped to a cross-domain feature association network, and coupled and associated with the multimodal alignment information in the cross-domain feature association network to obtain a comprehensive state feature vector that represents the overall state of the test subject. Based on a pre-built psychological assessment model, the comprehensive state feature vector is identified and analyzed to generate psychological assessment results including risk level and psychological state type. Based on the psychological assessment results, a psychological assessment report for the test subject is generated. Personalized psychological assessment strategies are generated based on the test taker's basic information, including: Obtain basic information about the test subjects and construct an initial feature profile representing the attributes and characteristics of the test subjects based on the basic information. The initial feature profile is constructed based on the test subjects' age, gender, educational background, occupation type, cognitive preference characteristics, and emotional characteristics. Load the initial feature profile of the test subject, and adjust the attribute feature weights in the initial feature profile according to the test subject's test requirements to obtain the adjusted feature profile after the attribute feature weights are adjusted. Use the adjusted feature profile as the strategy trigger engine, and convert the adjusted feature profile into standard strategy requirement information through a collaborative attention network. Obtain strategy requirement information, and use the strategy requirement information as an index to traverse the multimodal strategy assessment question bank. The multimodal strategy assessment question bank triggers the push of at least one set of standard assessment strategies. Map the standard assessment strategies and strategy requirement information into structured vectors and strategy requirement vectors, respectively. The standard assessment strategy is a baseline template containing multimodal question item metadata vectors. Multimodal questions include text modality, voice modality, visual scene modality, video-micro-expression modality, and physiological modality. Obtain the structured vector corresponding to the standard evaluation strategy, calculate the weighted similarity between the structured vector and the strategy requirement vector using the weighted Hellinger-cosine product similarity function, and use the standard evaluation strategy corresponding to the highest weighted similarity as the candidate standard evaluation strategy. Load candidate standard assessment strategies, and adjust the priority of multimodal items in the standard assessment strategies based on the adjusted feature profile after adjusting the attribute feature weights, to generate personalized psychological assessment strategies for test takers; Based on the adjusted feature profile after attribute feature weighting, the priority of multimodal items in the standard assessment strategy is adjusted, including: Candidate standard assessment strategies are obtained, multimodal items in the standard assessment strategies are identified, and they are parsed based on a sparse Transformer architecture with embedded attention mechanism to obtain item multimodal features including modality type attributes, cognitive level, emotional arousal intensity, assessment target contribution rate, and assessment time consumption. Load the multimodal features of multimodal items, construct a personalized priority scoring function based on the adjusted feature profile after adjusting the attribute feature weights, and score the priority of the multimodal features of the items through the personalized priority scoring function to obtain the item priority value of the multimodal features of the items; Based on the priority value of the multimodal features of the items, the multimodal items in the standard assessment strategy are sorted in descending order to obtain the initial adjustment assessment strategy; An initial adjustment assessment strategy is obtained. A sequential adjustment reward function is constructed with constraints of minimizing modal type mutual exclusion, maximizing modal equilibrium, and minimizing time budget. An improved NSGA-III algorithm is used to solve the sequential adjustment reward function to generate a Pareto front solution set. The optimal solution is selected as the final personalized assessment psychological assessment strategy. Noise suppression preprocessing is performed on the multimodal evaluation information, including: The priority values ​​and modality type attributes of multimodal items in the personalized assessment psychological testing strategy of the test subjects are retrieved. Based on the improved Relief algorithm, the priority values ​​and modality type attributes of multimodal items are coupled to obtain the filtering priority of multimodal items. Acquire multimodal assessment information, determine whether the filtering priority of the multimodal items corresponding to the multimodal assessment information exceeds the preset filtering threshold, and perform adaptive noise suppression on multimodal assessment information with different filtering priorities based on the submodal noise suppression mechanism; If the filtering priority of the multimodal items does not exceed the preset filtering threshold, the first filter is used to adaptively filter the multimodal evaluation information to obtain low-priority modal information. If the filtering priority of a multimodal item exceeds the preset filtering threshold, a second filter is used to adaptively filter the multimodal assessment information to obtain high-priority modal information. Low-priority modal information and high-priority modal information are loaded. The physiological signal stream time axis in the multimodal evaluation information is used as the master clock. The speech signal stream is inserted into the master clock using a phase vocoder. The text stream and video signal stream are inserted into the master clock using linear interpolation to obtain multimodal alignment information with uniform time granularity.

2. The AI-based psychological assessment method based on multimodal input as described in claim 1, characterized in that: The first filter includes a first filter input layer, a first filter parallel module, and a first filter output layer. The first filter parallel module includes a Jieba layer and a blind source separation layer based on accelerometer data, which are set in parallel. The second filter includes a second filter input layer, a second filter parallel module, and a second filter output layer. The second filter input layer is connected to the second filter parallel module, and the second filter parallel module is connected to the second filter output layer. The second filter parallel module includes an NLMS adaptive filtering layer and an MTCNN layer, which are set in parallel. The NLMS adaptive filtering layer and the MTCNN layer are respectively connected to the U-Net deep learning segmentation layer.

3. The artificial intelligence psychological assessment method based on multimodal input as described in claim 2, characterized in that: Coupling and associating multimodal alignment information within a cross-domain feature association network includes: The multimodal alignment information is obtained and mapped to the modal cross-projection layer of the cross-domain feature association network. During the mapping process, the multimodal alignment information is subjected to nonlinear transformation and information compression. A bidirectional cross-attention mechanism is employed to perform bidirectional interaction on the multimodal alignment information within the modal cross-projection layer, thereby extracting the context vector of the multimodal alignment information. Information aggregation is performed on the multimodal alignment information and context vector, and the aggregated multimodal alignment information is decomposed into low-rank Tucker decomposition to obtain the comprehensive state feature vector representing the overall state of the test subject.

4. The artificial intelligence psychological assessment method based on multimodal input as described in claim 3, characterized in that: When the psychological assessment model is pre-built, it is based on a graph neural network model. The psychological assessment model also includes an assessment input layer and an assessment output layer. A multimodal attention mechanism network is set between the assessment input layer and the graph neural network model. The multimodal attention mechanism network includes front-end processing embedding layers, multi-head cross-modal attention layers, and a Transformer classification head. The graph neural network model includes a mental state graph builder, a three-layer GNN layer, an MLP layer, and a conditional cross-attention layer. A saliency analysis layer is set between the graph neural network model and the assessment output layer.

5. The artificial intelligence psychological assessment method based on multimodal input as described in claim 4, characterized in that: The comprehensive state feature vector is identified and analyzed based on a pre-built psychological assessment model, including: Obtain the comprehensive state feature vector that represents the overall state of the test subject, and dynamically calculate the label weights of multimodal features in the comprehensive state feature vector based on the multi-head attention mechanism; Based on the Transformer classification head, the label weights among the multimodal features are weighted and fused to generate a coarse classification risk level for evaluating the comprehensive state feature vector; Obtain the coarse classification risk level corresponding to the comprehensive state feature vector, generate a feature enhancement mask based on the coarse classification risk level, create a fine multi-label classification based on the conditional cross attention mechanism, and perform dimensional logistic regression scoring on the fine multi-label classification. The system uses dimensional logistic regression scores as the risk level assessment for test subjects, outputting the risk level assessment of test subjects. Based on the gradient class significance analysis tool XGBoost model, the system compares the fine multi-label classification with the preset psychological state classification interval, outputting the psychological state type of the test subjects. Based on the psychological assessment results, the system generates a psychological assessment report for the test subjects.

6. An AI-based psychological assessment system based on multimodal input, used to implement the AI-based psychological assessment method based on multimodal input as described in any one of claims 1-5, characterized in that: The AI-based psychological assessment system based on multimodal input includes: The personalized strategy generation module is used to collect basic information of test subjects and generate personalized psychological assessment strategies based on the basic information of test subjects. The multimodal information acquisition module is used to collect multimodal assessment information of test subjects in real time based on personalized assessment psychological assessment strategies, perform noise suppression preprocessing on the multimodal assessment information, and output multimodal alignment information with uniform time granularity. The information coupling and association module is used to acquire multimodal alignment information, map the multimodal alignment information to the cross-domain feature association network, and couple and associate the multimodal alignment information in the cross-domain feature association network to obtain a comprehensive state feature vector that represents the overall state of the test subject. The assessment module identifies and analyzes the comprehensive state feature vector based on a pre-built psychological assessment model, generates psychological assessment results including risk level and psychological state type, and generates a psychological assessment report for the test subject based on the psychological assessment results.

7. The AI-based psychological assessment system based on multimodal input as described in claim 6, characterized in that: The personalized strategy generation module includes: The profile generation unit is used to acquire the basic information of the test subject and construct an initial feature profile representing the attribute characteristics of the test subject based on the basic information of the test subject; The strategy requirement acquisition unit is used to load the initial feature profile of the test subject, adjust the attribute feature weights in the initial feature profile based on the test subject's test requirements, and obtain the adjusted feature profile after the attribute feature weights are adjusted. The adjusted feature profile is used as the strategy triggering engine, and the adjusted feature profile is converted into standard strategy requirement information through a collaborative attention network. The standard strategy push unit is used to obtain strategy requirement information. Using the strategy requirement information as an index, it traverses the multimodal strategy assessment question bank. The multimodal strategy assessment question bank triggers the push of at least one set of standard assessment strategies. The standard assessment strategies and strategy requirement information are mapped into structured vectors and strategy requirement vectors, respectively. The standard strategy screening unit is used to obtain the structured vector corresponding to the standard evaluation strategy. The weighted similarity between the structured vector and the strategy requirement vector is calculated using the weighted Hellinger-cosine product similarity function. The standard evaluation strategy corresponding to the highest weighted similarity is selected as the candidate standard evaluation strategy. The standard strategy adjustment unit is used to load candidate standard assessment strategies. Based on the adjusted feature profile after attribute feature weight adjustment, the multimodal items in the standard assessment strategy are prioritized to generate a personalized psychological assessment strategy for the test taker.

Images