A multi-modal art therapy modeling and privacy protection method based on federated learning

Abstract

A kind of multi-modal art healing modeling and privacy protection method based on federal learning, comprising: constructing federal learning framework, including global aggregation server and N higher vocational college client nodes, each client collects local multi-modal data, multi-modal data includes psychological scale evaluation data, online behavior log data and artistic image preference data, and the three types of data are standardized pretreated and feature encoded;Deploy active learning module on each client, semi-automatic labeling of artistic image preference data based on sample uncertainty, generate labeled artistic preference training set, and control the labeling cost within the preset threshold value.The present application obtains psychological scale evaluation, online behavior log and artistic image preference three types of heterogeneous data by deploying data acquisition module on each higher vocational college client node, and carries out standardization normalization, word embedding encoding and visual feature extraction.

Description

Technical Field

[0001] This invention belongs to the field of mental health analysis technology, specifically a multimodal art therapy modeling and privacy protection method based on federated learning. Background Technology

[0002] Federated learning, multimodal modeling, and art therapy-based psychological intervention belong to three sub-sectors: distributed machine learning, multi-source data processing, and campus mental health services. A standardized implementation system for these three technologies, tailored to vocational college students, has long been lacking. Multimodal modeling relies on multiple information carriers to characterize psychological preferences; federated learning utilizes distributed node collaborative training to avoid cross-node transmission of raw data; and art therapy uses visual art materials to alleviate emotions. Combining these three technologies can build a non-invasive psychological service system for colleges and universities. Currently, the digital construction of campus mental health services generally involves deploying these three technologies separately, lacking an integrated implementation path.

[0003] First, commercially available multimodal federated modeling frameworks only support the fusion of similar data types, failing to synchronously accommodate three types of heterogeneous data: scale values, time-series operation records, and image visual features. Modality fusion merely employs simple vector concatenation, unable to establish a mapping between psychological states and aesthetic or operational behaviors. Furthermore, the feature vectors generated locally by various institutions have inconsistent dimensions and numerical scales, making it impossible to uncover deep multimodal relationships during global server aggregation. The output user demand vectors only carry surface-level independent information, unable to generate hierarchical and quantifiable representations of art therapy needs, and lack cross-modal attention fusion logic based on psychological numerical modalities as the query benchmark.

[0004] Secondly, the general federated learning gradient privacy protection only adds fixed-amplitude noise, lacking a gradient norm constraint step. The gradient value fluctuation range from a single sample is uncontrolled, and the noise cannot fully cover the sample feature clues carried within the gradient. Global model aggregation only assigns fixed weights according to the number of samples at each node, without introducing local model fitting metrics to participate in dynamic weight adjustment. The gradient contribution of nodes with small data volume but good fitting performance is weakened. At the same time, model iteration only adopts a static one-time training mode, without corresponding incremental updates and historical parameter fixation logic. The dynamic changes in students' psychological aesthetic state cannot be continuously synchronized to the global model. Summary of the Invention

[0005] In view of the above situation and to overcome the shortcomings of the prior art, the present invention provides a multimodal art therapy modeling and privacy protection method based on federated learning, so as to at least partially solve the above technical problems.

[0006] The technical solution adopted in this invention is as follows: This invention proposes a multimodal art therapy modeling and privacy protection method based on federated learning, comprising the following steps: Step S1: Construct a federated learning framework, including a global aggregation server and N vocational college client nodes. Each client collects local multimodal data, including psychological scale assessment data, online behavior log data, and art image preference data. The three types of data are then standardized, preprocessed, and feature-encoded. Step S2: Deploy the active learning module on each client to perform semi-automatic annotation of art image preference data based on sample uncertainty, generate an annotated art preference training set, and control the annotation cost within a preset threshold; Step S3: Construct a cross-modal collaborative attention fusion network locally on each client, and perform cross-modal interactive modeling of psychological scale features, behavioral log features and visual features of art preferences to output a personalized art therapy needs profile vector for students. Step S4: In the local model gradient calculation stage of federated training, the gradient vector is pruned by norm and Laplacian noise that satisfies differential privacy constraints is added to generate privacy-preserving gradients, and the privacy-preserving gradients are uploaded to the global aggregation server. Step S5: The global aggregation server calculates adaptive aggregation weights based on the data volume ratio of each client and the performance of the local model on the validation set, generates a global demand profile model through weighted averaging, and distributes the global model to each client node; Step S6: Based on the continuous interaction between students and the art therapy recommendation system, periodically trigger the incremental update process of the federated model, fine-tune the global model based on the newly added interaction data, and realize the dynamic evolution of the demand profile. Step S7: Based on the updated demand profile vector, recommend personalized art therapy solutions to students through the healing content matching module, including three types of recommendation results: color therapy, composition therapy, and style therapy.

[0007] In one embodiment of the present invention, step S1 specifically includes: Each vocational college client collects psychological scale assessment data through the campus mental health management system and collects students' explicit ratings or implicit feedback data on art images through the art image preference annotation system. The art image annotation data covers three dimensions: color preference, composition preference, and style category preference. The psychological scale assessment data were normalized using Min-Max, mapping the scores of each scale to the [0,1] interval:

[0008] in, The original scale scores, The minimum possible score for this scale. This represents the maximum possible score for the scale. These are the normalized characteristic values ​​of the scale; Sliding window statistics and word embedding encoding are performed on online behavior log data to transform continuous behavior sequences into fixed-dimensional behavior feature vectors.

[0009] in, For behavior sequences, As a behavioral feature dimension, For learnable embedding matrices, This is a behavioral feature vector; Visual features are extracted from artistic images using a pre-trained convolutional neural network to extract color histogram features, compositional texture features, and style category features, generating an art preference visual feature vector.

[0010] in, Input for artistic images, For pre-trained CNN parameters, As a dimension of art preference characteristics, This is a visual feature vector representing artistic preferences.

[0011] In one embodiment of the present invention, the active learning module in step S2 is implemented as follows: Calculate the prediction uncertainty of unlabeled artistic image samples, and use the entropy maximization criterion to select high-value labeled samples:

[0012] in, For the sample Uncertainty, This represents the total number of art preference categories (color, composition, style). For the current model to samples Category The predicted probability, These are the parameters of the current client's local model. Sort the samples by uncertainty from high to low, select the top-K samples for manual labeling, and use the prediction results of the current model as pseudo-labels for the remaining samples to achieve semi-automatic labeling; The cost constraint for labeling is:

[0013] in, This represents the number of samples that require manual labeling. The total number of artistic image samples, The cost threshold is typically set to 0.1-0.3.

[0014] In one embodiment of the present invention, the cross-modal collaborative attention fusion network in step S3 is implemented as follows: The features of psychological scales, behavioral logs, and visual features of art preferences are input into the encoders of their respective modalities to obtain the modal latent representations:

[0015] in, These are the learnable parameters for the three modal encoders; Calculate cross-modal collaborative attention weights, using psychological scale features as queries and behavioral log features and art preference features as keys and values, to perform cross-modal attention computation:

[0016] in, The projection matrix of the attention mechanism, The dimension of the key vector. The feature vector for the demand profile after cross-modal fusion; The fused feature vectors are input into a multilayer perceptron, which outputs the final art therapy needs profile vector:

[0017] in, This is the vector dimension for demand profiles, typically ranging from 64 to 256. We create personalized art therapy needs profile vectors for students, with each dimension corresponding to a different intensity of therapy needs.

[0018] In one embodiment of the present invention, the differential privacy-preserving gradient processing in step S4 includes: Before uploading the local model gradients to the global server, the gradient information is made privacy-preserving, specifically including: Gradient norm clipping: Calculates the gradient vector of the current training batch. And based on a preset cropping threshold Perform L2 norm clipping on it, that is: when When the gradient is scaled to To limit the sensitivity of a single sample to the influence of the gradient and prevent gradient leakage from leading to the reverse inference of student's original data features, the specific formula is as follows:

[0019] in, This is the original gradient vector. This is the gradient clipping threshold. Describing the L2 norm, The clipped gradient vector; Laplace noise injection: based on a preset privacy budget Sensitivity to gradient after clipping Random noise that satisfies a Laplace distribution is generated and then superimposed onto the clipped gradient vector to form a result that satisfies... -Perturbation gradient for differential privacy:

[0020] in, This is the L2 sensitivity of the gradient (the upper limit of the gradient difference between two adjacent datasets). For privacy budget ( The smaller the value, the stronger the privacy protection (typically 0.5-5). Position parameters The Laplace distribution of the scale parameter is the privacy-preserving gradient; Privacy-preserving gradients are uploaded to the global aggregation server, while local raw data and local model parameters do not leave the client node.

[0021] In one embodiment of the present invention, the adaptive aggregation weight calculation method in step S5 is as follows: After receiving the privacy-preserving gradients uploaded by each client, the global aggregation server calculates the aggregation weights for each client based on the proportion of data volume and model performance.

[0022] in, For the first The aggregate weight of each client, This represents the number of local training samples for this client. The total number of samples from all clients. For the first The F1 score of the client-side local model on the validation set. This is a tradeoff factor between data volume and model performance, typically set to 0.5; The formula for updating global model parameters is:

[0023] in, For the first Round global model parameters, The global learning rate, For the first Privacy protection gradient for each client upload.

[0024] In one embodiment of the present invention, the dynamic demand profile update mechanism in step S6 includes: ① Set the profile update trigger condition: When the cumulative number of interactions between students and the art therapy recommendation system reaches a threshold. Or the time interval since the last update exceeds the threshold. At that time, the federated model is triggered to update incrementally; ② Each client collects newly added interaction behavior data, including art content click records, dwell time, rating feedback and sharing behavior, to build an incremental training dataset; ③ Employing an elastic weight fixation mechanism for incremental learning, while updating the model to adapt to new data, protects previously learned historical knowledge from catastrophic forgetting:

[0025] in, Let the loss function be the current task. The diagonal elements of the Fisher information matrix (measurement parameters) (Importance of historical mission) These are the historically optimal parameters. This is the curing strength coefficient.

[0026] In one embodiment of the present invention, the healing content matching method in step S7 is as follows: Based on demand profile vector The matching degree between it and various types of content in the art therapy content library is calculated using cosine similarity measurement:

[0027] in, For the first The feature vector of art therapy content (including color features, composition features, and style features). A score for the degree of matching; Sort by matching score from high to low, select the top-N art therapy content to recommend to students, where N typically ranges from 5 to 10.

[0028] The beneficial effects of the technical solution of this invention are as follows: This invention addresses the problem of unified representation of multi-source heterogeneous data by deploying data acquisition modules at client nodes in various higher vocational colleges, acquiring three types of heterogeneous data: psychological scale assessments, online behavioral logs, and artistic image preferences, and then performing standardization, normalization, word embedding encoding, and visual feature extraction. This invention introduces an active learning strategy based on uncertainty sampling to automatically select high-information samples for manual annotation, while generating pseudo-labels for the remaining samples using the current model. This reduces the amount of manual annotation to 10%-30% of the total while ensuring annotation accuracy. It also employs a cross-modal attention mechanism that uses psychological scale features as queries and behavioral logs and art preference features as key-value pairs to achieve deep interactive modeling of the three modalities and generate personalized profile vectors that contain multi-dimensional healing needs. This invention, through gradient norm pruning and adaptive Laplacian noise injection during the gradient upload stage of federated training, ensures that even if an attacker intercepts the gradient information, they cannot reverse-engineer the student's original data; the global server comprehensively evaluates the data scale and model quality of each client, dynamically allocates aggregation weights, and generates a high-performance global demand profile model. This invention triggers incremental updates of the federated model based on students' continuous interactive behavior, employs an elastic weight solidification mechanism to avoid catastrophic forgetting, and recommends personalized art therapy programs to students based on the latest profile vector.

[0029] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0030] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the multimodal art therapy modeling and privacy protection method based on federated learning proposed in this invention. Detailed Implementation

[0031] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0032] The following describes an embodiment of the present invention, with reference to the accompanying drawings, a method for multimodal art therapy modeling and privacy protection based on federated learning.

[0033] like Figure 1 As shown, this embodiment of the invention provides a multimodal art therapy modeling and privacy protection method based on federated learning, including the following method steps: Step S1: Construct a federated learning framework, including a global aggregation server and N vocational college client nodes. Each client collects local multimodal data, including psychological scale assessment data, online behavior log data, and art image preference data. The three types of data are then standardized, preprocessed, and feature-encoded. Step S2: Deploy the active learning module on each client to perform semi-automatic annotation of art image preference data based on sample uncertainty, generate an annotated art preference training set, and control the annotation cost within a preset threshold; Step S3: Construct a cross-modal collaborative attention fusion network locally on each client, and perform cross-modal interactive modeling of psychological scale features, behavioral log features and visual features of art preferences to output a personalized art therapy needs profile vector for students. Step S4: In the local model gradient calculation stage of federated training, the gradient vector is pruned by norm and Laplacian noise that satisfies differential privacy constraints is added to generate privacy-preserving gradients, and the privacy-preserving gradients are uploaded to the global aggregation server. Step S5: The global aggregation server calculates adaptive aggregation weights based on the data volume ratio of each client and the performance of the local model on the validation set, generates a global demand profile model through weighted averaging, and distributes the global model to each client node; Step S6: Based on the continuous interaction between students and the art therapy recommendation system, periodically trigger the incremental update process of the federated model, fine-tune the global model based on the newly added interaction data, and realize the dynamic evolution of the demand profile. Step S7: Based on the updated demand profile vector, recommend personalized art therapy solutions to students through the healing content matching module, including three types of recommendation results: color therapy, composition therapy, and style therapy.

[0034] In practical applications, the entire chain of this invention runs on two levels of hardware nodes: the local client of the institution and the central aggregation server. Only gradient information processed by privacy perturbation is transmitted between the two types of nodes. The original multimodal data of the institution is stored in the local storage medium throughout the process. The first step of the chain completes the construction of the global training framework and the unified transformation and processing of multi-source heterogeneous data. The global framework is divided into an independent central computing node and multiple sets of local computing nodes of the institution. Each local computing node of the institution is simultaneously connected to three types of data acquisition channels. The three types of raw data are respectively connected to a dedicated standardized transformation unit to complete the numerical range unification, sequence vector conversion and image visual feature extraction operations. The feature vectors of the three types after unified format conversion are simultaneously sent to the local interactive computing network to complete the data format unification preprocessing before the cross-operation of multi-type data information, providing a unified dimension input data carrier for subsequent cross-modal feature interaction operations.

[0035] Furthermore, in the second step of the link, a sample screening and labeling unit is installed on the local computing node. The unit completes the hierarchical sorting of unlabeled artistic image samples based on the fluctuation range of the predicted values ​​of the samples. The samples with higher ranking are sent to the manual labeling channel to generate standard labels, while the samples with lower ranking are used as alternative labels by the predicted values ​​output by the local computing network. The two types of labeled samples are merged to generate a complete local image training dataset. The unit has built-in numerical constraints to control the total proportion of samples with manual labeling. The manual labeling operation only covers high information gain samples, and low-value samples rely on the model to generate labels to supplement the training data. The entire labeling process relies on the image visual features output in the first step to carry out the operation. Based on the image features after the standardization process in the first step, the sample fluctuation range is directly calculated, and the total number of samples covered by manual labeling is reduced simultaneously. The complete image training dataset after labeling is directly connected to the local interactive computing network in the third step, and continuously participates in feature interactive computing as input data for the network image branch.

[0036] Furthermore, the third step of the process relies on a local interactive computing network to extract cross-information from three types of standardized features. The network is divided into three independent feature encoding branches that receive psychological scale features, behavioral sequence features, and visual features of artistic images, respectively. The hidden vectors output by the three encoding branches enter the cross-modal attention computing unit. The unit uses the psychologically related hidden vector as the query benchmark, and the behavioral and image hidden vectors are concatenated as the key-value matching benchmark. The unit has three independent projection matrices built in to complete the unified mapping of vector dimensions. The fused vector after mapping is sent to the multilayer perception unit to compress and generate a fixed-dimensional user demand vector. Each set of values ​​in the vector corresponds to the strength of a type of art therapy demand. The entire network operation only calls the local feature data after the standardization and annotation processing in the first two steps, and does not transmit any original features and annotation samples to external nodes. The user demand vector generated by the operation is stored locally for subsequent gradient calculation and material matching. Cross-modal interactive computing can capture the inherent correspondence between psychological values, operational behaviors, and image aesthetics, and provide a unified standard vector output result for subsequent gradient privacy processing, global model aggregation, and material matching.

[0037] Furthermore, the fourth step of the link takes the original gradient vector produced by the backpropagation of the network in the third step. The gradient vector is then sent to the gradient scaling unit and the noise superposition unit to complete the privacy transformation. The gradient scaling unit fixes the maximum value range of the gradient vector. Gradient vectors exceeding the range are scaled proportionally. The scaled gradient vector is then sent to the noise superposition unit. The unit generates a random noise vector based on the fixed privacy control value and the gradient sensitivity value. The noise vector and the scaled gradient vector are superimposed bit by bit to generate the privacy gradient vector. The original data of the institution, the local network parameter storage channel and the gradient external transmission channel are physically isolated. Only the gradient vector that has undergone privacy perturbation processing can be transmitted to the central aggregation server. The gradient scaling operation reduces the influence range of a single sample on the gradient value, and the noise superposition operation destroys the original data features implicit in the gradient. After the two types of operations are executed simultaneously, external nodes cannot rely on the gradient information to reverse infer the student's private psychological data. The privacy gradient serves as the only data transmission between nodes, connecting the model parameter update link between the local client and the central server. The privacy gradient does not carry any original user data features, meeting the basic operational requirement that the data does not leave the local area.

[0038] Furthermore, in the fifth step of the process, the central aggregation server uniformly receives the privacy gradient vectors uploaded by all university nodes. Internally, the server synchronously retrieves two types of basic values ​​for weight calculation: one type is the ratio of the total number of local training samples at a single university to the total number of samples from all universities; the other type is the ratio of the local model validation set fitting index at a single university to the total fitting index of all universities. These two ratios are weighted and summed using fixed weight coefficients to obtain the single-university gradient aggregation weight. The privacy gradient vectors of all universities are multiplied by their corresponding weight values ​​and then accumulated. The accumulated result, combined with the global learning rate, completes the iterative update of the central model parameters. The newly generated full-scale model is then generated. The global model parameters are synchronously distributed to all local client nodes of all institutions. After receiving the global model, each institution node replaces its old local network parameters and conducts a new round of local multimodal data training and gradient generation operations. The weight calculation takes into account both the scale of the institution's data and the local model's fitting effect. Institution nodes with less data but better model fitting effect can obtain higher aggregate weights, avoiding the global model fitting bias caused by relying solely on the number of samples to allocate weights. The global model distribution operation synchronously completes the synchronous update of model parameters across multiple institutions, unifies the basic network structure of all local clients, and ensures that the user demand vectors output by different institution nodes have a unified measurement standard.

[0039] Furthermore, the sixth step of the process continuously monitors user interaction data between the university's client and the art therapy material recommendation system. This interaction includes four types of information: image click records, page dwell time, user rating feedback, and material sharing operations. The monitoring unit sets two sets of trigger values: the cumulative number of interaction operations reaches a corresponding value, and the interval between two model updates reaches a corresponding value. Once either condition is met, the incremental training process is initiated. Each university node collects new interaction data to construct an incremental training dataset. This dataset is then fed into the incremental learning module, which incorporates a historical parameter constraint unit for computation. The module adds a historical parameter constraint loss to the training loss based on the new data. The constraint loss is generated based on the historical best network parameters and parameter importance numerical calculations. After incremental training, the local client re-executes the entire process of gradient calculation, privacy perturbation, gradient upload, central server weight aggregation, and global model distribution. The incremental training module can retain the user feature association logic learned by the model in the early stage. New interaction data only fine-tunes the model parameters. Periodic incremental updates rely on real-time user interaction data to continuously correct the demand vector output by the model, reducing the deviation between the model's stored vector and the user's current psychological and aesthetic state. The incremental training process fully reuses the gradient privacy processing and adaptive aggregation links in steps four and five, reducing the resource overhead caused by additional architecture construction.

[0040] Furthermore, the seventh step of the process receives the user demand vector output after the incremental update in the sixth step. This vector is then fed into the material matching unit. The unit synchronously stores the user's multi-dimensional demand vector and all material feature vectors within the art material library. The unit calculates the matching degree between a single user demand vector and a single material feature vector based on the cosine of the angle between the vectors. All materials are sorted in descending order according to their matching degree values, and a fixed number of materials at the top of the sorted list are selected as recommended output content. The unit divides into three independent output channels to push materials matching the color dimension, the composition dimension, and the art style dimension, respectively. These three types of materials are pushed to the front-end interactive interface for users to view. The matching operation is carried out based on the complete multi-dimensional demand vector output by the model. The demand intensity values ​​of all dimensions within the vector are called simultaneously to complete the material comparison. The single-dimensional material classification output can correspond to different levels of healing needs of users. The user demand vector is not transmitted to third-party material platforms. The material matching operation reuses the standardized vector format generated by the preceding steps. The vector dimensions are unified and no additional conversion is required. It fully inherits the user feature data produced by the preceding federated modeling steps, forming a complete closed-loop processing system from multi-modal data collection, model training, privacy transmission, dynamic updates to personalized material push.

[0041] In one specific implementation, step S1 specifically includes: Each vocational college client collects psychological scale assessment data through the campus mental health management system and collects students' explicit ratings or implicit feedback data on art images through the art image preference annotation system. The art image annotation data covers three dimensions: color preference, composition preference, and style category preference. The psychological scale assessment data were normalized using Min-Max, mapping the scores of each scale to the [0,1] interval:

[0042] in, The original scale scores, The minimum score on the scale. The maximum score on the scale. These are the normalized characteristic values ​​of the scale; Sliding window statistics and word embedding encoding are performed on online behavior log data to transform continuous behavior sequences into fixed-dimensional behavior feature vectors.

[0043] in, For behavior sequences, As a behavioral feature dimension, For learnable embedding matrices, This is a behavioral feature vector; Visual features are extracted from artistic images using a pre-trained convolutional neural network to extract color histogram features, compositional texture features, and style category features, generating an art preference visual feature vector.

[0044] in, Input for artistic images, For pre-trained CNN parameters, As a dimension of art preference characteristics, This is a visual feature vector representing artistic preferences.

[0045] In practical applications, the embodiments of this invention involve four parallel processing branches running synchronously. These four branches share the same local computing resources on the client side. After each branch completes its operation, it synchronously outputs standardized feature vectors with uniform dimensions and consistent numerical ranges. All vectors are uniformly sent to the input port of the cross-modal collaborative attention fusion network. The four branches form a complete front-end data processing link. The starting point of the link is the two types of business acquisition systems and the behavior log storage unit, and the ending point is the three types of modal feature vector output channels. The four branches cooperate with each other to complete the unified mapping of heterogeneous data. The standardized vectors produced by each branch need to be jointly used by the output vectors of the other two branches to participate in subsequent modal interaction operations. The output of a single branch cannot independently support the model training process.

[0046] Furthermore, the first collection branch is responsible for collecting and retrieving two types of raw data. The campus mental health management system reads the questionnaire responses completed by students on campus and outputs raw score data in pure numerical format. The art image preference annotation system retrieves the interaction records generated by students browsing, liking, scoring, and collecting images. The system simultaneously decomposes the three types of labeling information contained in the records: color tendency, picture layout tendency, and art style tendency. The two collection systems are deployed on the same local server of the higher vocational college client. During the collection process, no student raw responses or image interaction records are transmitted to external nodes. The two types of raw data are sent to the corresponding numerical conversion unit and image feature extraction unit, respectively. The online behavior log storage unit simultaneously retrieves the full time-series operation records generated by students operating the art therapy platform. The time-series records are simultaneously sent to the sliding window statistics unit. The collection operations of the three types of raw data are started simultaneously. The collected data directly enters the corresponding conversion branch. The entire collection process serves as the prerequisite for all standardized operations. Without the raw data output by the collection branch, subsequent normalization, sequence encoding, and image extraction operations cannot be started.

[0047] Furthermore, the second numerical transformation branch receives the raw scores of the scales output by the campus mental health management system. The branch has a built-in fixed mapping calculation formula to complete the interval scaling operation. The formula reads the lowest and highest score boundaries corresponding to a single scale and converts the raw scores into continuous values ​​between 0 and 1. The converted values ​​are uniformly used as psychological modal features. All scale data share the same numerical range after conversion. The score weights of different scales will not be shifted due to differences in the original score range. The entire conversion operation is completed in the local memory of the client. The standardized values ​​generated by the conversion are not transmitted externally separately, but are only stored in the local cache to be merged with the other two types of modal features and input into the fusion network. The branch produces uniform interval values, which can eliminate the scale differences between different scales within the psychological data and set the same numerical benchmark for the three types of modal features, avoiding the dominance of one type of scale score in the overall feature.

[0048] Furthermore, the third sequence encoding branch receives the complete behavioral time-series record output by the behavioral log storage unit. Within the branch, a sliding window statistical operation is first initiated. The window truncates continuous operation segments according to a fixed length, splitting the ultra-long time-series record into long subsequences. The subsequences processed by the window are sent to the embedding encoding operation unit. The encoding unit retrieves the learnable embedding matrix to complete the sequence vectorization mapping. The mapping operation outputs a fixed-length one-dimensional vector. All behavioral sequences have the same vector dimension after window truncation and encoding. All original operation sequences of varying lengths are transformed into feature carriers of the same specification. The encoding operation is completed based on the embedding matrix stored locally on the client. The matrix parameters are updated synchronously with the global model distribution process. The behavioral feature vectors generated by encoding are stored in the local cache and synchronously undergo modal fusion operations with normalized psychological values ​​and image visual vectors to unify the vector specifications of the behavioral data and form input materials with the other two types of modal features that match the specifications.

[0049] Furthermore, the fourth image feature extraction branch receives the image file and corresponding three-dimensional preference labels output by the art image preference annotation system. The branch calls the pre-trained convolutional network to complete the image pixel information decomposition and calculation. The network outputs pixel color distribution statistics, image texture layout information, and image style classification values ​​in layers. The three types of information are spliced ​​and integrated into a single image visual vector. After all input images are processed by the same convolutional network, the output vector dimension remains consistent. The original pixel dot matrix information is compressed into a low-dimensional feature carrier after network calculation. After the original image file is processed, it is directly deleted from the cache, and only the compressed visual vector is retained to participate in the subsequent process. The vector extracted by the branch is synchronously recorded into the local cache and summarized with the psychological and behavioral feature vectors. The convolutional network synchronously outputs three corresponding feature components of color, composition, and style. The component content and the three-dimensional preference labels obtained in the acquisition stage form a corresponding matching relationship, providing image-side subdivision dimension information for subsequent cross-modal attention calculation.

[0050] In one specific implementation, the active learning module in step S2 is implemented as follows: Calculate the prediction uncertainty of unlabeled artistic image samples, and use the entropy maximization criterion to select high-value labeled samples:

[0051] in, For the sample Uncertainty, This represents the total number of art preference categories (color, composition, style). For the current model to samples Category The predicted probability, These are the parameters of the current client's local model. Sort the samples by uncertainty from high to low, select the top-K samples for manual labeling, and use the prediction results of the current model as pseudo-labels for the remaining samples to achieve semi-automatic labeling; The cost constraint for labeling is:

[0052] in, This represents the number of samples that require manual labeling. The total number of artistic image samples, The cost threshold is typically set to 0.1-0.3.

[0053] In specific applications, the active learning module of this invention is divided into three consecutive computational processes. The three processes share the standardized art preference visual feature vector output in step S1 as a unified input material. The three processes are executed sequentially in a fixed order. The numerical results produced by the previous process are directly sent to the next process as the basis for computation. All three processes are deployed locally on the client side of the higher vocational college. The computation process does not call up data from external servers. The three processes jointly generate a complete labeled image dataset. The dataset is directly connected to the image input channel of the cross-modal collaborative attention fusion network. A single process cannot produce complete and usable training samples when running independently. The output of each computational unit must be combined with the other two units to complete the complete sample labeling task.

[0054] Furthermore, the first computational unit reads all unlabeled art image feature vectors stored locally. The unit retrieves the model parameters that have been initialized locally on the client side. The parameters participate in the unified entropy calculation formula to perform the calculation. The formula simultaneously reads the predicted probability values ​​corresponding to the three preference categories of color, composition, and style. Each image sample corresponds to a complete probability distribution array. After substituting all the values ​​in the array into the summation operation, the uncertainty value corresponding to a single image sample is obtained. After all unlabeled samples have completed the value conversion one by one, a complete set of uncertainty values ​​is formed. The unit operation reuses the visual feature vectors extracted by the convolutional network in step S1 throughout the entire process. The local computing power usage scale remains within a stable range. The entropy calculation logic synchronously covers the three preference classification dimensions. The uncertainty value output by a single sample synchronously reflects the fluctuation degree of the model judgment of the image in all classification dimensions of color, composition, and style. The general screening logic that only performs entropy calculation for a single classification dimension cannot synchronously obtain the comprehensive fluctuation data of the three dimensions. The output value cannot fully match the multi-label classification requirements required for art therapy modeling.

[0055] Furthermore, the second sorting and splitting unit receives the set of uncertainty values ​​corresponding to all samples output by the first operation unit. Within the unit, a binding mapping relationship is established between samples and corresponding values. The mapping entries are rearranged according to their values ​​from largest to smallest. The rearranged entry sequence is split into two sets of data: a fixed number of entries at the front end and the remaining entries at the back end. The image samples corresponding to the front end entries are transferred to the manual annotation channel. The image samples corresponding to the back end entries retrieve the classification prediction values ​​output by the local model as sample labels. The manual annotation channel outputs sample entries with manually labeled real labels. The label entries generated from the prediction values ​​are simultaneously stored locally. The storage areas for both types of entries maintain a unified data format, and the arrangement rules of image feature vectors and label information within both types of entries are completely consistent. The splitting operation relies on the comprehensive fluctuation value calculated from the entropy value as the sole basis for splitting. The sample splitting rules are bound to the three-dimensional image preference comprehensive judgment of fluctuation. Samples with higher values ​​correspond to a larger fluctuation range in the model's judgment of the three-class image preferences. The manual annotation operation is concentrated on these samples. Low-fluctuation samples rely on model prediction values ​​to fill in the label content.

[0056] Furthermore, the third-stage quantity constraint judgment unit synchronously reads the total number of manually labeled sample items and the total number of all local art image samples. The unit inputs a fixed ratio value to complete the numerical comparison calculation. The ratio value is set to a fixed value in the range of 0.1 to 0.3. The total number of manually labeled sample items, the ratio value, and the total number of full samples maintain a size comparison relationship. The total number of manually labeled sample items does not exceed the product value. Before the sample diversion operation is executed, the unit will calculate the maximum number of manually labeled samples that can be accommodated based on the total number of full samples and the ratio value. The number of front-end items selected by the diversion unit matches the maximum number of items. The ratio value can be adjusted according to the number of labelers within the institution. The unit continuously verifies the total number of manually labeled samples. When the number of samples exceeds the upper limit, the unit automatically reduces the number of front-end manually labeled items and supplements more sample items with pseudo-labels generated by model prediction. The unit outputs a constraint signal and synchronously transmits it to the second-stage sorting and diversion unit to complete the dynamic control of the sample selection scale. The entire set of numerical constraint logic limits the sample volume corresponding to the manual operation. Institutions can adjust the ratio value according to their own personnel configuration to adapt to the labeling resource supply of institutions of different sizes.

[0057] In one specific implementation, the cross-modal collaborative attention fusion network in step S3 is implemented as follows: Characteristics of psychological scales Behavioral log characteristics Visual characteristics of artistic preferences Input the encoders for each modality separately to obtain the modal implicit representations:

[0058] in, These are the learnable parameters for the three modal encoders; Calculate cross-modal collaborative attention weights, using psychological scale features as queries and behavioral log features and art preference features as keys and values, to perform cross-modal attention computation:

[0059] in, The projection matrix of the attention mechanism, The dimension of the key vector. The feature vector for the demand profile after cross-modal fusion; The fused feature vectors are input into a multilayer perceptron, which outputs the final art therapy needs profile vector:

[0060] in, This is the vector dimension for demand profiles, typically ranging from 64 to 256. We create personalized art therapy needs profile vectors for students, with each dimension corresponding to a different intensity of therapy needs.

[0061] In specific applications, the cross-modal collaborative attention fusion network of this invention consists of three sequentially connected structures: a modality coding unit, a cross-modal attention operation unit, and a multilayer perceptron unit. The three structures share the three types of standardized modal vectors produced in steps S1 and S2 as unified input materials. The vector data generated by the previous structure is completely transmitted to the next structure as the operation input. The three structures are synchronously deployed on the local hardware environment of the vocational college client. All parameter updates and vector operation processes are completed in local memory. The three structures jointly operate to generate a unified specification of demand profile vector. The vector is directly sent to the gradient calculation unit for backpropagation operation. A single structure cannot produce a complete feature carrier that is adapted to the subsequent process when running alone. The output vector of each structure requires the participation of the other two structures in the complete feature fusion process.

[0062] Furthermore, the modality coding unit is divided into three sets of parallel independent coding branches. These three branches correspond to three types of input vectors: psychological scale features, behavioral log features, and visual features of artistic preferences. Each branch carries a dedicated set of learnable parameters. After receiving the original standardized vector of the corresponding modality, each branch performs single-layer or multi-layer nonlinear mapping operations. The mapping operation compresses redundant information in the original vector and outputs modality latent representation vectors of uniform specifications. The three coding branches start operations synchronously, and the three types of original vectors enter the corresponding branches simultaneously to complete the transformation. The latent vectors output by the three branches are stored in the local cache for retrieval by the attention operation unit. The three independent coding branches set exclusive mapping rules for different attribute modalities. Psychological numerical vectors, temporal behavioral vectors, and image visual vectors each complete dimensionality reduction and purification according to the transformation logic adapted to their own information distribution, unifying the basic size of various types of vectors. The general single encoder parallel structure does not distinguish between the three types of heterogeneous data (psychological, behavioral, and image) and set independent parameter systems. Applying the same mapping rules to all modalities will weaken the unique information features of each modality.

[0063] Furthermore, the cross-modal attention processing unit reads the latent vectors produced by the three encoding branches. Internally, the unit performs a vector concatenation operation between the behavioral and artistic latent vectors. The concatenated result serves as the original input material for keys and values. The psychological modality latent vector is used separately as query material and input into the processing branch. The unit has three sets of independent projection matrices corresponding to the query, key, and value data paths, respectively. After the vector input for each path, a dimension scaling mapping is performed. The mapped query vector and key vector undergo matrix multiplication. The result is matched with the key vector dimension value to perform scaling. The scaling result is fed into the softmax function to generate the weight allocation coefficients corresponding to each set of vectors. The weight coefficients are then matrix-multiplied with the concatenated value vector. The multiplication operation generates a fused feature vector. The entire operational logic uses psychological dimension information as the criterion for weight allocation. External information, including behavior and images, is assigned corresponding weights according to their correlation with psychological state. The vector concatenation operation integrates all information from both external modalities. Three sets of projection matrices simultaneously align the three-way vector dimensions. The weight allocation values ​​directly reflect the strength of the correlation between psychological state and aesthetics and operational behavior. Conventional attention networks often use images or text as the query subject, which cannot prioritize capturing the constraint relationship between psychological state and aesthetic preferences in the corresponding scenario. The combination of vector concatenation with a dedicated three-way projection matrix has not formed a standardized application solution in the field of psychological-art multimodal modeling.

[0064] Furthermore, the multilayer perceptron unit receives the fused feature vector output by the attention operation unit. The unit contains multiple continuous nonlinear transformation layers, each with an independent set of learnable parameters. The fused vector undergoes feature purification and dimensionality compression layer by layer. The unit ultimately outputs a fixed-dimensional one-dimensional profile vector, with the vector dimension ranging from 64 to 256. Each set of values ​​within the vector corresponds to a strength indicator of a type of art therapy need. All dimensional values ​​are synchronously stored in a local cache. During the forward propagation phase, these values ​​serve as a carrier of the need profile and are sent to the therapy content matching module. During the backward propagation phase, they participate in gradient value solving. The multiple continuous nonlinear transformation layers decompose the interwoven multimodal correlation information within the fused vector, breaking down the comprehensive correlation features into independent sub-item need intensity values, thus completing the conversion of the fused features into a standardized need label vector. An attention network without a perceptron unit can only output an unlayered comprehensive fused vector, where various therapy need information is intertwined, making it impossible to directly perform layered matching operations for the three types of therapy materials: color, composition, and style.

[0065] In one specific implementation, the differential privacy-preserving gradient processing in step S4 includes: Before uploading the local model gradients to the global server, the gradient information is made privacy-preserving, specifically including: Gradient norm clipping: Calculates the gradient vector of the current training batch. And based on a preset cropping threshold Perform L2 norm clipping on it, that is: when When the gradient is scaled to To limit the sensitivity of a single sample to the influence of the gradient and prevent gradient leakage from leading to the reverse inference of student's original data features, the specific formula is as follows:

[0066] in, This is the original gradient vector. This is the gradient clipping threshold. Describing the L2 norm, The clipped gradient vector; Laplace noise injection: based on a preset privacy budget Sensitivity to gradient after clipping Random noise that satisfies a Laplace distribution is generated and then superimposed onto the clipped gradient vector to form a result that satisfies... -Perturbation gradient for differential privacy:

[0067] in, This is the L2 sensitivity of the gradient (the upper limit of the gradient difference between two adjacent datasets). For privacy budget ( The smaller the value, the stronger the privacy protection (typically 0.5-5). Position parameters Scale parameters The Laplace distribution, For privacy protection gradients; Privacy protection gradient Uploaded to the global aggregation server, the local raw data and local model parameters do not leave the client node.

[0068] In specific applications, the gradient privacy processing flow of this invention is divided into three consecutive execution modules: a gradient norm clipping unit, a Laplace noise superposition unit, and a gradient transport isolation unit. These three modules rely on the original gradient vector generated by the backpropagation operation in step S3 as unified input material. The three modules complete the computation in a fixed order, with the vector generated by the previous module being completely fed into the next module for computation. All three modules are deployed in the local computing environment of the vocational college client. The entire computation process does not access the global server's storage content. The three modules collaboratively generate perturbed gradient vectors, which are then sent to the global aggregation server as the only data carrier that can be transmitted externally. A single module cannot generate gradient vectors that satisfy differential privacy constraints when running alone; the output data of each module requires the cooperation of the other two modules to complete the complete privacy protection process.

[0069] Furthermore, the gradient norm clipping unit receives the original gradient vector generated by backpropagation from the cross-modal collaborative attention fusion network. The unit reads a locally preset clipping threshold constant, calculates the L2 norm value corresponding to the original gradient vector, and compares the norm value with the clipping threshold constant. The comparison result is substituted into a fixed conversion formula to complete the gradient scaling conversion. When the L2 norm value of the original gradient vector exceeds the threshold constant, the entire vector is scaled proportionally according to the ratio of the threshold to the norm. When the L2 norm value of the original gradient vector is lower than the threshold constant, the vector is directly output with its original numerical specification. The converted gradient vector is stored in a local temporary buffer for noise superposition. Unit retrieval and the entire conversion operation uniformly limit the range of gradient value changes that any sample can cause within a single batch. The upper limit of the difference in gradient values ​​between adjacent sample sets is fixed, and the upper limit of values ​​is synchronously used as the basis for sensitivity calculation in subsequent noise calculation stages. The pruning operation only applies to the gradient vector value level. The original multimodal training data, modality encoder parameters, and attention projection matrix parameters stored on the client do not participate in gradient scaling operations throughout the process. The general gradient processing flow does not set a fixed upper limit to constrain the influence of single-sample gradients. The uncontrolled gradient value fluctuation range will increase the computational load of the noise stage, and the corresponding sensitivity values ​​cannot be uniformly calibrated.

[0070] Furthermore, the Laplacian noise superposition unit reads the scaled gradient vector output by the clipping unit. The unit performs gradient L2 sensitivity conversion based on the clipping threshold constant, uniformly setting the sensitivity value to twice the clipping threshold constant. The unit retrieves the locally configured privacy budget constant, and divides the sensitivity value by the privacy budget constant to obtain the noise distribution scale parameter. The unit generates a Laplacian random noise vector of the same dimension according to the zero-position parameter and the converted scale parameter. The noise vector and the scaled gradient vector are added element-wise. The resulting vector is stored as a perturbation gradient vector in the transmission buffer. The privacy budget constant is set to 0. The selectable values ​​are in the range of 0.5 to 5. Adjusting the values ​​synchronously changes the noise distribution scale parameter. When the privacy budget constant is lowered, the corresponding scale parameter is simultaneously amplified. The range of noise values ​​superimposed within the vector is simultaneously widened. The entire noise generation logic relies on the gradient sensitivity calibrated in the pruning stage to complete parameter matching. The two units form a numerical linkage relationship. Changing the pruning threshold will synchronously adjust all the operation parameters of the noise generation stage. The general privacy scheme with independently configured noise parameters does not have the linkage logic between the parameters of the two units. Setting the sensitivity value separately will result in a mismatch between the noise magnitude and the gradient fluctuation range, making it impossible to stably maintain the differential privacy constraint conditions.

[0071] Furthermore, the gradient transport isolation unit reads the perturbation gradient vector produced by the noise superposition unit. The unit is divided into two independent cache areas: a local storage partition and an external transmission partition. The local storage partition retains the original gradient vector, the complete multimodal training dataset, the parameters of the three modal encoders, the attention projection matrix parameters, and the multilayer perceptron parameters. The external transmission partition only stores the perturbation gradient vector. A one-way data transmission rule is set between the two partitions, allowing only the perturbation gradient vector to be transferred from the local storage partition to the external transmission partition. The original gradient vector, local model parameters, and student original multimodal data do not have the channel permission to flow into the external transmission partition. The perturbation gradient vector in the external transmission partition is pushed to the data receiving port of the global aggregation server according to the communication protocol. After the data push is completed, the cache content of the external transmission partition is cleared. All original data and model parameters in the local storage partition are continuously stored on the client's local hardware and will not be transmitted to external nodes along the communication link. The partition isolation rule cuts off the channel for the outward flow of original sensitive information. The data transmission logic of the general federated learning node does not divide into independent storage partitions, and there is a path for the original gradient and model parameters to leak outward.

[0072] In one specific implementation, the adaptive aggregation weight calculation method in step S5 is as follows: After receiving the privacy-preserving gradients uploaded by each client, the global aggregation server calculates the aggregation weights for each client based on the proportion of data volume and model performance.

[0073] in, For the first The aggregate weight of each client, This represents the number of local training samples on the client side. The total number of samples from all clients. For the first The F1 score of the client-side local model on the validation set. This is a tradeoff factor between data volume and model performance, typically set to 0.5; The formula for updating global model parameters is:

[0074] in, For the first Round global model parameters, The global learning rate, For the first Privacy protection gradient for each client upload.

[0075] In practical applications, the adaptive weighted aggregation process of this invention is divided into three consecutively running modules: a weight parameter acquisition unit, a client weight calculation unit, and a global model iteration unit. These three modules uniformly receive the privacy-preserving gradient vectors transmitted by each client in step S4 as the basic computational material. The three modules follow a fixed execution order, and all numerical results output by the previous module are completely transmitted to the next module for subsequent calculations. All three modules are deployed within the global aggregation server hardware. The computation process does not retrieve the original data and model parameters stored locally by each client. The three modules jointly perform computations to generate iterated new global model parameters. Once the parameters are completed, they are synchronously distributed to all client nodes to replace the old local parameters. A single module cannot complete the gradient weighting and global parameter update operations independently; the numerical output of each module requires the collaboration of the other two modules to complete the entire aggregation process.

[0076] Furthermore, the weight parameter acquisition unit establishes multiple sets of parallel data reading channels. Each channel is connected to the privacy gradient cache area uploaded by each vocational college client. One set of channels reads the total number of local training samples of the corresponding client. After summing all client sample values, the total number of samples in the entire domain is obtained. Another set of channels reads the F1 index value of the local model of the corresponding client after testing on the validation dataset. All client index values ​​are summed synchronously to obtain the total index of the entire domain. The unit sets up independent storage space to store four types of values: single node sample count, total number of samples in the entire domain, single node F1 score, and total F1 score of the entire domain. The unit synchronously reads the local preset tradeoff coefficient constant. The constant value range is set from 0 to 1. The constant can be configured with a fixed value of 0.5 to balance the weight ratio of the two types of data. All values ​​collected by the unit are packaged and transmitted to the client weight solving unit. The entire acquisition operation synchronously collects two types of basic operation data: sample size dimension and model fitting effect dimension. General acquisition units that only collect sample count or only collect model index cannot provide dual-dimensional values ​​to support weight fusion calculation. Storing the two types of dimensional values ​​separately will interrupt the subsequent merging calculation process of weighting coefficients.

[0077] Furthermore, the client-side weight calculation unit receives all the basic values ​​output by the weight parameter acquisition unit. The unit is divided into two parallel numerical computation branches. The first branch performs a division conversion between the number of samples from a single client and the total number of samples in the entire domain. The conversion result is multiplied by the constant weighting coefficient. The second branch performs a division conversion between the F1 score of a single client and the sum of the F1 scores in the entire domain. The conversion result is multiplied by the difference between the constant and 1. The product of the two branches is added to obtain the corresponding client-specific aggregated weight. All clients repeat the same computation process to generate a one-to-one set of weight values. The set of weight values ​​is bound to the corresponding client's privacy-preserving gradient vector. The mapping entries are synchronously sent to the global model iteration unit. The two parallel computation branches achieve numerical balance based on the same weighting coefficient. The coefficient adjustment operation synchronously changes the proportion of sample size and model indicators within the weights. The converted values ​​of the two dimensions are synchronously merged to generate a single weight value. The general single-dimensional weight calculation branch only uses the calculation logic related to the number of samples.

[0078] Furthermore, the global model iteration unit receives the set of weight values ​​and the bound privacy gradient vectors output by the client weight solving unit. The unit retrieves all parameters of the previous round of the global model stored locally on the server. Internally, the unit traverses all client-mapped entries, performing element-wise multiplication between the weight value and the corresponding privacy gradient vector within a single entry. The vectors produced by the multiplication operations of all entries are accumulated to obtain the gradient weighted sum vector. The unit retrieves the preset global learning rate constant, multiplies the weighted sum vector by the learning rate constant, and performs element-wise addition with the parameters of the previous round of the global model. The vector produced by the addition operation is stored in the server parameter cache area as a new round of global model parameters. The new parameters are distributed to all vocational college client nodes according to the communication link. After receiving the parameters, each node replaces all encoder, attention projection matrix, and multilayer perceptron parameters of the local cross-modal fusion network. The entire iterative operation uses privacy gradient vectors that have been pruned and superimposed with noise, without accessing the original, perturbation-free gradients on the client. The parameter update logic is adapted to the gradient value distribution after differential privacy processing. After the parameter distribution is completed, this round of aggregation process ends, and the new round of local training and gradient upload operations are awaited by each client.

[0079] In one specific implementation, the dynamic demand profile update mechanism in step S6 includes: ① Set the profile update trigger condition: When the cumulative number of interactions between students and the art therapy recommendation system reaches a threshold. Or the time interval since the last update exceeds the threshold. At that time, the federated model is triggered to update incrementally; ② Each client collects newly added interaction behavior data, including art content click records, dwell time, rating feedback and sharing behavior, to build an incremental training dataset; ③ Employing an elastic weight fixation mechanism for incremental learning, while updating the model to adapt to new data, protects previously learned historical knowledge from catastrophic forgetting:

[0080] in, Let the loss function be the current task. The diagonal elements of the Fisher information matrix (measurement parameters) (Importance of historical mission) These are the historically optimal parameters. This is the curing strength coefficient.

[0081] In practical applications, the dynamic demand profile update process of this invention is divided into three consecutive execution modules: an update trigger judgment unit, an incremental dataset construction unit, and an EWC incremental loss calculation unit. These three modules uniformly accept the interactive behavior records generated in real-time by the art therapy recommendation system as basic input materials. The three modules complete their calculations in a fixed order, with the output of the first module being sent to the next module for computation. All three modules are deployed in the local computing environment of the vocational college client. The computation process does not retrieve the original copy of the historical model parameters from the global server. The three modules collaboratively produce a locally fine-tuned model adapted to the newly added interactive data. After backpropagation to generate gradient vectors, the fine-tuned model is fully integrated into the gradient privacy processing link in step S4. A single module cannot complete the entire incremental training process independently; the data produced by each module requires the cooperation of the other two modules to complete the entire dynamic profile update process.

[0082] Furthermore, the update trigger judgment unit internally establishes two parallel judgment channels. The first channel continuously accumulates the interactive behavior items generated by students operating the art therapy platform. The channel has a built-in interaction number threshold constant. When the accumulated number of items reaches the threshold constant, a trigger signal is output. The second channel records the timestamp of the previous round of incremental update completion. The channel has a built-in time interval threshold constant. When the difference between the current time and the historical timestamp exceeds the threshold constant, a trigger signal is output. The trigger signals output by the two channels are connected to the same signal aggregation unit. A valid trigger signal generated by either channel can start the downstream incremental dataset construction unit. The unit synchronously retains two types of threshold constants for maintenance personnel to adjust the values. The two sets of judgment logics run in parallel to synchronously manage the two types of update constraints: the volume of accumulated interactive data and the time span. The timing of the update start is somewhat one-sided.

[0083] Furthermore, after receiving the start signal from the trigger judgment unit, the incremental dataset construction unit retrieves all interactive behavior records that have not participated in previous training from the local storage medium. The records are split into four independent data categories: art content click items, page dwell time values, student rating values, and content sharing operation tags. The four types of data are simultaneously sent to the sliding window statistics and word embedding encoding unit built into step S1 to complete standardized feature transformation. The transformed behavioral feature vectors are bound and mapped to the corresponding student historical psychological scale and art preference features. All mapped items are integrated to form an incremental training dataset and stored in the local cache area. The feature vector dimension and value range within the dataset maintain the same specifications as the initial training dataset. The entire dataset construction process reuses the pre-processing multimodal preprocessing link. The four types of interactive records are simultaneously included in the incremental sample set. The general solution of selecting only a single type of interactive behavior to construct the dataset cannot fully capture the dynamic changes in students' aesthetic and psychological states.

[0084] Furthermore, the EWC incremental loss computation unit reads the standardized incremental sample set output by the incremental dataset construction unit, retrieves the locally stored historical best model parameter set, the diagonal element set of the Fisher information matrix, and the fixed strength coefficient constant, and splits the overall loss function into basic task loss components and EWC constraint loss components. The basic task loss components are generated by forward propagation based on the incremental dataset, while the EWC constraint loss components read the current value of each set of network parameters and the corresponding historical best parameter value, calculate the difference between the two sets of values, square the result, and multiply it by the corresponding diagonal element of the Fisher information matrix. The calculation involves summing the multiplication results of all parameters and multiplying the sum by the solidification strength coefficient constant to obtain the complete constraint loss value. The basic task loss component and the constraint loss component are added to generate the incremental training total loss function. The total loss function is used to drive the back-side fine-tuning of network parameters. The diagonal elements of the Fisher information matrix correspond to the contribution of the parameters to the historical multimodal healing modeling task. Parameters with higher diagonal element values ​​will have a larger penalty within the constraint loss component, and the parameter modification range is correspondingly limited. The solidification strength coefficient constant uniformly adjusts the overall penalty intensity of all constraint items, and the feature adaptation space corresponding to the new interaction data is excessively compressed.

[0085] In one specific implementation, the healing content matching method in step S7 is as follows: Based on demand profile vector The matching degree between it and various types of content in the art therapy content library is calculated using cosine similarity measurement:

[0086] in, For the first The feature vector of art therapy content (including color features, composition features, and style features). A score for the degree of matching; Sort by matching score from high to low, select the top-N art therapy content to recommend to students, where N typically ranges from 5 to 10.

[0087] In practical applications, the therapeutic content matching process of this invention is divided into three consecutively running modules: a vector storage and caching unit, a cosine similarity calculation unit, and a material screening and grouping unit. These three modules uniformly receive the student art therapeutic needs profile vectors output from the incremental update process in step S6 as the basic calculation material. The three modules complete the calculations in a fixed order, with the vector data produced by the previous module being completely transmitted to the next module for calculation. All three modules are deployed in the local computing environment of the vocational college client. During the calculation process, the student profile vectors are not transmitted to third-party material service nodes. The three modules collaboratively output the grouped art material items, which are directly pushed to the platform's front-end interactive interface. A single module cannot complete the complete material matching and grouping push operation independently; the data produced by each module requires the cooperation of the other two modules to complete the entire material matching process.

[0088] Furthermore, the vector storage cache unit is divided into two independent storage partitions. The first partition continuously receives the required profile vectors output by the incremental update process, and all dimensional values ​​within the vector are completely retained without deletion. The second partition batch inputs the feature vectors corresponding to all materials in the art therapy content library. Each material vector is simultaneously split into three sub-feature components: color, composition, and style. The vectors stored in the two partitions maintain the same dimensional specifications. A bidirectional reading channel is built inside the unit, which can simultaneously retrieve a single student profile vector and all material vectors in the library. The unit reuses the pre-processed multimodal feature extraction logic to generate material feature vectors. The dimensional and numerical distribution rules of the material vectors are consistent with those of the profile vectors. After the two sets of vector data are stored in the local cache, they are directly sent to the downstream similarity calculation unit. The general cache unit that stores a single type of vector cannot simultaneously retrieve both profile and material vectors. Inconsistent vector specifications will interrupt the similarity value calculation process.

[0089] Furthermore, the cosine similarity calculation unit reads the image vector and the single material vector output by the vector storage cache unit. The unit then performs three consecutive operations using a fixed numerical conversion formula: vector inner product, L2 norm calculation for both types of vectors, and ratio conversion. The inner product operation simultaneously reads all dimension values ​​of the image vector and all dimension values ​​of the material vector, performs corresponding multiplication and summation, and performs L2 norm calculations on each type of vector to obtain the modulus value. The two sets of modulus values ​​are multiplied together as the denominator, and the inner product result is used as the numerator. The numerator and denominator are then divided to obtain the matching degree corresponding to the single material. The score range covers a value from -1 to 1. The unit traverses all vectors in the material library to complete batch score conversion. The score set corresponding to all materials and portrait vectors is uniformly sent to the downstream material filtering and grouping unit. The entire score conversion logic relies on the corresponding relationship of the vector angle to complete the matching degree quantification. It will not be affected by the calculation offset caused by the difference in the numerical scale of the vector dimension. The general matching logic that only relies on the difference of the numerical value of a single dimension cannot simultaneously integrate the correlation information of the three healing dimensions. The score conversion result cannot fully reflect the correspondence between the students' multi-level needs and the characteristics of the materials.

[0090] Furthermore, the material filtering and grouping unit receives the material and score binding set output by the cosine similarity calculation unit. Within the unit, the set items are rearranged in descending order according to the score values. The unit has an internal entry truncation constant, with the constant value ranged from five to ten. The front end of the rearranged entry sequence truncates the corresponding constant number of material items. The truncated material items are grouped into three categories based on their own vector's internal color, composition, and style sub-feature components. The three groups of material items generate independent output queues, and the queue data is directly pushed to the platform's front-end display module. The unit synchronously retains the complete score sorting sequence and stores it in the local cache for secondary matching calculations after the next round of portrait vector updates. The combination of fixed-quantity truncation and three-dimensional grouping synchronously controls the total number of items displayed on the front end. At the same time, the output materials are split according to the healing type. A general filtering logic without quantity constraints and dimensional grouping would cause the number of items displayed on the front end to fluctuate, and the mixed arrangement of materials of different healing types would not meet the needs of students' stratification.

[0091] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0092] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A multimodal art therapy modeling and privacy protection method based on federated learning, characterized in that, The following steps are included: Step S1: Construct a federated learning framework, including a global aggregation server and N vocational college client nodes. Each client collects local multimodal data, including psychological scale assessment data, online behavior log data, and art image preference data. The three types of data are then standardized, preprocessed, and feature-encoded. Step S2: Deploy the active learning module on each client to perform semi-automatic annotation of art image preference data based on sample uncertainty, generate an annotated art preference training set, and control the annotation cost within a preset threshold; Step S3: Construct a cross-modal collaborative attention fusion network locally on each client, and perform cross-modal interactive modeling of psychological scale features, behavioral log features and visual features of art preferences to output a personalized art therapy needs profile vector for students. Step S4: In the local model gradient calculation stage of federated training, the gradient vector is pruned by norm and Laplacian noise that satisfies differential privacy constraints is added to generate privacy-preserving gradients, and the privacy-preserving gradients are uploaded to the global aggregation server. Step S5: The global aggregation server calculates adaptive aggregation weights based on the data volume ratio of each client and the performance of the local model on the validation set, generates a global demand profile model through weighted averaging, and distributes the global model to each client node; Step S6: Based on the continuous interaction between students and the art therapy recommendation system, periodically trigger the incremental update process of the federated model, fine-tune the global model based on the newly added interaction data, and realize the dynamic evolution of the demand profile. Step S7: Based on the updated demand profile vector, recommend personalized art therapy solutions to students through the healing content matching module, including three types of recommendation results: color therapy, composition therapy, and style therapy.

2. The method for multimodal art therapy modeling and privacy protection based on federated learning according to claim 1, characterized in that, Step S1 specifically includes: Each vocational college client collects psychological scale assessment data through the campus mental health management system and collects students' explicit ratings or implicit feedback data on art images through the art image preference annotation system. The art image annotation data covers three dimensions: color preference, composition preference, and style category preference. The psychological scale assessment data were normalized using Min-Max, mapping the scores of each scale to the [0,1] interval: ； in, The original scale scores, The minimum possible score for this scale. This represents the maximum possible score for the scale. These are the normalized characteristic values ​​of the scale; Sliding window statistics and word embedding encoding are performed on online behavior log data to transform continuous behavior sequences into fixed-dimensional behavior feature vectors. ； in, For behavior sequences, As a behavioral feature dimension, For learnable embedding matrices, This is a behavioral feature vector; Visual features are extracted from artistic images using a pre-trained convolutional neural network to extract color histogram features, compositional texture features, and style category features, generating an art preference visual feature vector. ； in, Input for artistic images, For pre-trained CNN parameters, As a dimension of art preference characteristics, This is a visual feature vector representing artistic preferences.

3. The method for multimodal art therapy modeling and privacy protection based on federated learning according to claim 1, characterized in that, The active learning module in step S2 is implemented as follows: Calculate the prediction uncertainty of unlabeled artistic image samples, and use the entropy maximization criterion to select high-value labeled samples: ； in, For the sample Uncertainty, This represents the total number of art preference categories (color, composition, style). For the current model to samples Category The predicted probability, These are the parameters of the current client's local model. Sort the samples by uncertainty from high to low, select the top-K samples for manual labeling, and use the prediction results of the current model as pseudo-labels for the remaining samples to achieve semi-automatic labeling; The cost constraint for labeling is: ； in, This represents the number of samples that require manual labeling. The total number of art image samples, The cost threshold is typically set to 0.1-0.

3.

4. The method for multimodal art therapy modeling and privacy protection based on federated learning according to claim 1, characterized in that, The cross-modal collaborative attention fusion network in step S3 is implemented as follows: Characteristics of psychological scales Behavioral log characteristics Visual characteristics of artistic preferences Input the encoders for each modality separately to obtain the modal implicit representations: ； in, These are the learnable parameters for the three modal encoders; Calculate cross-modal collaborative attention weights, using psychological scale features as queries and behavioral log features and art preference features as keys and values, to perform cross-modal attention computation: ； in, The projection matrix of the attention mechanism, The dimension of the key vector. The feature vector for the demand profile after cross-modal fusion; The fused feature vectors are input into a multilayer perceptron, which outputs the final art therapy needs profile vector: ； in, This is the dimension of the demand profile vector, typically ranging from 64 to 256. We create personalized art therapy needs profile vectors for students, with each dimension corresponding to a different intensity of therapy needs.

5. The multimodal art therapy modeling and privacy protection method based on federated learning according to claim 1, characterized in that, The differential privacy-preserving gradient processing in step S4 includes: Before uploading the local model gradients to the global server, the gradient information is made privacy-preserving, specifically including: Gradient norm clipping: Calculates the gradient vector of the current training batch. And based on a preset cropping threshold Perform L2 norm clipping on it, that is: when When the gradient is scaled to To limit the sensitivity of a single sample to the influence of the gradient and prevent gradient leakage from leading to the reverse inference of student's original data features, the specific formula is as follows: ； in, This is the original gradient vector. This is the gradient clipping threshold. Represents the L2 norm. The clipped gradient vector; Laplace noise injection: based on a preset privacy budget Sensitivity to gradient after clipping Random noise that satisfies a Laplace distribution is generated and then superimposed onto the clipped gradient vector to form a result that satisfies... -Perturbation gradient for differential privacy: ； in, This is the L2 sensitivity of the gradient (the upper limit of the gradient difference between two adjacent datasets). For privacy budget ( The smaller the value, the stronger the privacy protection (typically 0.5-5). Position parameters Scale parameters The Laplace distribution, For privacy protection gradients; Privacy protection gradient Uploaded to the global aggregation server, the local raw data and local model parameters do not leave the client node.

6. The method for multimodal art therapy modeling and privacy protection based on federated learning according to claim 1, characterized in that, The adaptive aggregation weight calculation method in step S5 is as follows: After receiving the privacy-preserving gradients uploaded by each client, the global aggregation server calculates the aggregation weights for each client based on the proportion of data volume and model performance. ； in, For the first The aggregate weight of each client, This represents the number of local training samples for this client. The total number of samples from all clients. For the first The F1 score of the client-side local model on the validation set. This is a tradeoff factor between data volume and model performance, typically set to 0.5; The formula for updating global model parameters is: ； in, For the first Round global model parameters, The global learning rate, For the first Privacy protection gradient for each client upload.

7. The method for multimodal art therapy modeling and privacy protection based on federated learning according to claim 1, characterized in that, The dynamic demand profile update mechanism in step S6 includes: ① Set the profile update trigger condition: When the cumulative number of interactions between students and the art therapy recommendation system reaches a threshold. Or the time interval since the last update exceeds the threshold. At that time, the federated model is triggered to update incrementally; ② Each client collects newly added interaction behavior data, including art content click records, dwell time, rating feedback and sharing behavior, to build an incremental training dataset; ③ Employing an elastic weight fixation mechanism for incremental learning, while updating the model to adapt to new data, protects previously learned historical knowledge from catastrophic forgetting: ； in, Let the loss function be the current task. The diagonal elements of the Fisher information matrix (measurement parameters) (Importance of historical mission) is the historical best parameter, and is the curing strength coefficient.

8. The method for multimodal art therapy modeling and privacy protection based on federated learning according to claim 1, characterized in that, The method for matching healing content in step S7 is as follows: Based on the demand profile vector, the matching degree between it and various types of content in the art therapy content library is calculated, using cosine similarity as a metric: ； Among them, is the first The feature vector of art therapy content (including color features, composition features, and style features). A score for the degree of matching; Sort by matching score from high to low, select the top-N art therapy content to recommend to students, where N typically ranges from 5 to 10.

Images