2608 results about "Learning methods" patented technology

The invention discloses a knowledge graph reasoning method based on dynamic rule perception memory. The method aims at solving the problems that an existing neural combination rule learning method is insufficient in expression ability and prone to splitting global semantics and local relation modes. The method comprises the following steps: introducing a lightweight dynamic relationship memory module, and executing self-attention on all relationships in a knowledge graph to capture global semantics; meanwhile, local relation interaction features are extracted through convolution, semantic fusion is carried out through a Transform encoder with multi-head attention and relative position coding, and unified and parallel-computing high-quality combination representation is constructed for each pair of relations in parallel. And meanwhile, a closed high-quality reasoning path is generated in combination with bidirectional breadth-first search. Through global semantic and local interactive collaborative modeling, the extendibility is ensured, the understanding of global semantics is enhanced, and the reasoning accuracy is improved. Global and local relation dependence is fused, an interpretable reasoning path is generated, and reasoning accuracy, expandability and robustness are improved.

An efficient heterogeneous federated learning method based on hybrid distillation includes: initializing, by a server, global model parameters, and setting a preset total number of training rounds and a number of clients participating in each of the training rounds; loading local datasets in the clients respectively, performing random transformations on the local datasets to generate client distillation data for the clients, sampling multiple sub-networks from an original network of each client, training each sub-network on the client distillation data to obtain updated local model parameters of each client, and uploading the updated local model parameters to the server; and receiving, by the server, the updated local model parameters, performing, by the server, server distillation based on the updated local model parameters and a preset auxiliary dataset to obtain updated global model parameters and an updated global model, and sending, by the server, the updated global model to the clients.

The invention relates to the technical field of data fusion, and discloses an animal scene-oriented adaptive multi-modal data fusion method, which comprises the following steps of: extracting spatio-temporal characteristics from multi-source heterogeneous data such as visual sense, auditory sense and physiological sensing, constructing an animal-environment-group ternary spatio-temporal relation graph, and constructing an animal-environment-group ternary spatio-temporal relation graph; a pilot frequency sampling problem is solved through an adaptive interpolation algorithm, cross-modal projection alignment is completed in a public semantic space, unified space-time representation is output, and confidence coefficient weight is dynamically calculated based on uncertainty measurement of each modal feature. According to the method, accurate alignment of multi-modal data is realized through the cross-modal space-time attention network, the multi-modal feature alignment error is reduced compared with that of a traditional LSTM method, the training data volume of a federated element migration reinforcement learning framework is reduced compared with that of a traditional migration learning method, and the cross-species generalization performance of the model is improved. A multi-level causal inference engine quantitatively reveals causal association between environmental factors and animal diseases, and in combination with a dynamic decision tree visualization technology, the decision recognition degree is improved.

The invention relates to the technical field of die-casting optimization, and discloses a die-casting process parameter optimization method and system based on digital twinning, and the method comprises the steps: arranging a sensor to collect the operation parameters of die-casting equipment and the quality data of a die casting in real time, and forming multi-source die-casting production data; according to multi-source die-casting production data, a multi-physical field simulation model is established, and a digital twinborn model is constructed. And comparing the virtual prediction result with the actually measured quality data, and constructing a virtual-real difference compensation network to correct the parameters of the digital twin model. And performing a multi-target reinforcement learning method based on the compensated digital twin model to generate optimal die-casting process parameters. And applying the optimal die-casting process parameters to die-casting equipment for verification, and updating the virtual-real difference compensation network according to a verification result. Intelligent optimization and continuous self-evolution of the die-casting process parameters are achieved, and the casting forming precision, the energy efficiency utilization rate and the production stability are improved.

The invention relates to the technical field of federated learning, in particular to a dynamic federated mutual learning method and system for balancing individuation and generalization, and the method specifically comprises the following steps: each client carries out the preprocessing of data to be processed of a model, and carries out the strong enhancement and weak enhancement processing; inputting the data subjected to strong enhancement processing into a shared model, inputting the data subjected to weak enhancement processing into a private model, and performing iterative training on the two models; related parameters of the shared model after each round of iterative training and a difference item between two model parameters are uploaded to a federation server; the federated server adopts a multi-dimensional adaptive aggregation strategy to obtain an updated global model, and returns the updated global model to each client to replace the shared model in the next round of training; and finally generating a generalization result and a personalized result. According to the method, the private-shared model architecture is constructed, and dynamic federated mutual learning is carried out in combination with the federated server, so that balance and collaborative improvement of individuation and generalization performance can be realized.

The invention relates to the technical field of learning methods, in particular to an OPGW heterogeneous detection method based on AI interference elimination, which comprises the following steps: acquiring a vibration signal segment, dividing mutation fragments, arranging channel sequence, comparing direction extension, extracting a direction change homing path segment, analyzing an offset trend positioning response area, and extracting fluctuation trend verification interruption. And combining state feature merging numbers, identifying a jump sequence and analyzing a switching relation, and obtaining disturbance boundary features. According to the method, a path chain is constructed through sudden change signal position division and a channel response sequence, a disturbance propagation starting point and an extension direction can be subjected to structured expression, a path segment corresponding to continuous direction change supports limitation of a response interval, a fluctuation trend and direction offset are combined to form segmented reference, and identification of characteristics of boundary evolution along with time is assisted; the hopping sequence in the tail end signal is used for restoring the disturbance boundary and enhancing the trajectory separation and feature retention capability of the multipath interference behavior in the differentiated environment.

The invention relates to an intelligent customer service self-learning method and system, and the method comprises the steps: collecting the interaction data of a user and an intelligent customer service, and forming a multi-dimensional data set; based on a PID (Proportion Integration Differentiation) controller, processing the performance indexes in the multi-dimensional data set, calculating a current error signal, and generating a corresponding control instruction according to the current error signal so as to adjust the response behavior of the intelligent customer service system in real time; evaluating the system performance data adjusted by the control instruction to obtain evaluation feedback; and according to the evaluation feedback, dynamically adjusting the parameters of the PID controller through a self-adaptive control strategy, and feeding back the adjusted parameters to the PID controller in the step S2. According to the invention, by introducing a closed-loop feedback mechanism based on the PID controller and a parameter adaptive optimization strategy, real-time regulation and control of the response behavior of the intelligent customer service system are realized, and the stability, the response speed and the user satisfaction of the system are remarkably improved.

A machine learning method where, in a first step, a first (general) machine learning model is trained using a first training dataset including unlabelled optical fibre sensing data. Then, in a second step, a transfer learning process is applied to adapt or fine-tune the first machine learning model to a more specific application (e.g. to perform a specific type of detection or classification). Due to the large volumes of optical fibre sensing data available, the first machine learning model may provide a general machine learning model which has a high level of generality and is highly adaptable.

The invention belongs to the technical field of asynchronous federated learning, and discloses a data isomerism-oriented knowledge alignment asynchronous federated learning method. According to the method, a data quality perception aggregation strategy is introduced, and a knowledge distillation mechanism based on the old degree is combined, so that a global model is subjected to balanced training on heterogeneous data of different devices, and the generalization ability of the model is improved. Meanwhile, a self-adaptive learning rate adjustment mechanism based on aggregation frequency and weight is designed, and it is ensured that contribution of different devices to the global model is more fair. According to the method, the training deviation in asynchronous federated learning is effectively relieved, the accuracy and stability of a global model are improved, and the method has a considerable application value for a real federated environment.

The invention discloses a protocol conversion and protocol self-learning method and system for transformer area optical storage and charging cooperation, and the method comprises the steps: constructing a digital twinborn body of a transformer area optical storage and charging system, carrying out the parallel operation of a protocol agent and a protocol agent in a virtual environment, and achieving the cooperative training of the protocol agent and the protocol agent through a hierarchical reinforcement learning architecture, the protocol agent is responsible for learning a dynamic priority scheduling and compression strategy for heterogeneous protocol messages such as Modbus, CAN and IEC 104, the protocol agent is responsible for learning a power balance and voltage stability control strategy based on photovoltaic output, energy storage SOC and charging load, and the two agents realize cross-domain collaborative optimization through a reward function mutual coupling mechanism. And finally, safely deploying the collaborative strategy obtained by training to the edge control equipment of the physical transformer area. According to the method, the problems of disjunction of protocol conversion and cooperative control, protocol strategy solidification, insufficient cross-domain cooperation and the like in the prior art are solved, and the operation efficiency and the self-adaptive capability of the transformer area optical storage and charging system are improved.

The invention discloses a non-independent identically distributed data asynchronous federal learning method based on an improved aggregation algorithm. The method comprises the steps that a server initializes a global model and issues the global model to all clients; and the client performs local training on the received global model by using local data, and uploads the model and model parameters to the server after training is completed. Then, the server adjusts a model lag degree based on a client data volume proportion, calculates model difference consistency, client historical contribution stability, old degree penalty of the client model and cosine similarity of the client model and the global model based on parameters of the client model and the current global model, and generates an asynchronous federal aggregation factor accordingly; and updating the global model parameters to generate a new global model. And finally, testing the global model by the server, and judging whether the learning process is stopped or not. According to the method, fair and effective model aggregation can be realized, and the model convergence stability and the final model detection precision are improved.

The invention discloses a hypergraph feature representation method, system and device based on fuzzy logic and a medium, and the method comprises the steps: obtaining original data, and constructing a hypergraph structure containing vertexes and hyperedges according to the original data; the hypergraph structure is initialized, and an initialized hypergraph structure is obtained; and inputting the initialized hypergraph structure into a preset hypergraph convolutional fuzzy network model for feature representation processing to obtain a hypergraph structure after feature representation processing. According to the hypergraph convolutional fuzzy network model, fuzzy representation and the hypergraph technology are deeply fused, the membership core concept of fuzzy logic is introduced, the hard coding association limitation of an existing hypergraph representation learning method that no black is white is broken through, the technical problems that a traditional hypergraph model cannot capture association gradients and fuzzy semantics are lost are effectively solved, and the learning efficiency is improved. Therefore, the flexibility and accuracy of the model are effectively improved, and stable prediction is provided in an uncertain data environment.

The invention relates to the technical field of control, in particular to a double-arm robot operation skill learning method and system based on big language model reasoning. The method comprises the following steps: firstly, carrying out context semantic modeling on an input natural language task instruction based on a large language model to generate a task semantic graph; in combination with a semantic entity in the task semantic graph and visual perception data collected by a robot, determining a three-dimensional space position of a target object through a multi-modal matching model, and constructing an environment semantic graph containing object nodes and spatial relation edges; generating an action sequence by using a language model according to the task semantic map and the environment semantic map, and generating a collaborative operation strategy based on the two-arm tail end state and an obstacle map; and finally, collecting feedback data of the sensor in real time when the action sequence is executed. According to the method provided by the invention, the understanding and execution capability of the two-arm robot on the unstructured natural language instruction is remarkably improved.

The invention provides a cross-modal joint contrast learning method and device and electronic equipment, and the method comprises the steps: constructing a sample data set which covers a plurality of task types, such as a text retrieval image, an image retrieval text, a text retrieval text, an image retrieval image, an image-text joint retrieval image and an image-text joint retrieval text; and generating prompt words for identifying task types for each task sample, splicing the prompt words with sample data to form task input, and determining modal types of a retrieval object and a retrieval target. A retrieval object and a retrieval target are respectively input into encoders of corresponding modes to extract features, the features are mapped to the same semantic space through a unified projection layer to obtain an embedded vector, and the parameters of the encoders and the projection layer are optimized by utilizing a contrast learning loss function based on the similarity of the retrieval object and the retrieval target, so that multi-task unified training is realized. Various cross-modal retrieval tasks can be supported in a unified semantic space at the same time, and the overall retrieval effect is improved on the premise of ensuring multi-task performance balance.

The invention discloses a multi-mode federated learning method and system, computer equipment and a readable storage medium, and belongs to the technical field of federated learning. The multi-modal federated learning method comprises the following steps: on each client node, mapping local data of various modals into a plurality of vectors in a unified semantic space, determining an incidence matrix of the data of the various modals, and fusing the plurality of vectors according to the incidence matrix to obtain a local semantic vector; training a local model by using the local semantic vector to obtain local model parameters, and uploading the local model parameters to a server; on the server, identifying the difference degree between the data distribution condition of each client node and the global data distribution condition, and determining the node weight vector of each client node; and performing weighted aggregation on the corresponding local model parameters by using the node weight vector of each client node to generate global model parameters for next federated learning. Therefore, the performance of the training model can be improved.

The present application relates to the field of federated learning and privacy protection, and particularly relates to a direction-aware local differential privacy federated learning method. The present application effectively solves the problem of serious decline in model performance caused by the destruction of gradient direction information by traditional differential privacy methods by adopting a direction-aware noise selection strategy based on an exponential mechanism locally on the client side. Specifically, the client first performs L2 norm clipping on the gradient to control the sensitivity, then generates K candidate noise vectors from a Gaussian distribution, evaluates the direction consistency of each candidate noise and the true gradient through cosine similarity, and selects the noise closest to the gradient direction in a probabilistic manner using the exponential mechanism, and finally adds the selected noise to the gradient to complete the perturbation. The method provides a strict epsilon-local differential privacy mathematical proof for the entire noise selection process, reduces the calculation time by about 80% compared to existing direction-aware methods, and achieves a triple balance of privacy protection, model utility and computational efficiency.

The invention discloses a causal perception sentiment analysis method and system based on thinking chain reasoning, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring and reading a multi-modal sentiment analysis data set; extracting video features from the video data in the multi-modal sentiment analysis data set, including voice, text and visual modal features; using the training set and the test set to train and verify the causal perception emotion polarity alignment model; inputting the test set into the trained causal perception emotion polarity alignment model to obtain an emotion state prediction result; video features are input into a causal perception emotion polarity alignment model, and emotion clues are extracted through thinking chain prompt and a self-supervision verification mechanism; then performing causal intervention and anti-factual reasoning on each modal feature by using an emotion clue to obtain a causal-related single-modal feature; and finally, obtaining joint feature representation from the causal-related single-mode features through cross-mode interaction by using a multi-mode representation learning method, and predicting an emotional state.

The invention provides a prostate cancer multi-modal data federal learning method and system, and relates to the technical field of intelligent medical treatment and distributed artificial intelligence cross, and the method comprises the steps: selecting a plurality of key feature regions in a feature space as a reference basis based on updated global feature parameters, and constructing a feature distribution topological structure; performing partition division on the feature distribution topological structure to generate a plurality of feature subspaces; calculating a spatial characteristic index according to the characteristic density and the distribution characteristic of each characteristic subspace, and generating a weight adjustment coefficient based on the spatial characteristic index; and dynamically optimizing the updated global feature parameters by using the weight adjustment coefficient to obtain optimized federal learning model parameters. Through multi-modal data feature alignment, gradient compression transmission, global gradient aggregation and parameter dynamic optimization, a complete prostate cancer multi-modal data federated learning framework is constructed, and model performance and cross-mechanism cooperation efficiency are improved.

The invention discloses a verifiable privacy protection federated learning method based on sensitive samples, and relates to the field of fault diagnosis. According to the method, the Poisson sampling process is introduced, the sampling probability is generated based on the privacy budget, it is ensured that all recorded privacy budgets are synchronously exhausted, data disastrous forgetting is effectively prevented, and the model effectiveness is improved. In the model verification module, a sensitive sample set is generated by means of a gradient maximization algorithm, a model integrity attack is detected, a user side only needs to submit a small number of sensitive samples for prediction through an inference service API provided by a private cloud client side, and if a returned result and a real result have significant deviation, it can be judged that the model is possibly tampered. According to the method, diversified privacy requirements of users can be met, black box verification on the integrity of the model is realized, and the precision of the model is improved.

The invention discloses a hierarchical semantic enhanced large model zero sample learning method, which is characterized by comprising the following steps: firstly, performing semantic enhancement on category names by using a large language model to generate rich text description, and constructing a dynamic and hierarchical semantic prototype by combining original semantic information, the prototype comprising global concepts, local attributes and relation representations; secondly, extracting global semantic features and local detail features of the image by adopting a vision-language large model and convolutional neural network double-branch structure, and fusing the global semantic features and the local detail features through an attention mechanism to obtain enhanced visual representation; finally, multi-level global alignment, local alignment and relation alignment are designed and jointly optimized, accurate mapping of enhanced visual features and hierarchical semantic prototypes is achieved on multiple granularities, and classification of invisible categories is finally completed. The objective of the invention is to solve the problem of limited model generalization ability caused by insufficient semantic representation and single vision-semantic granularity in the existing zero sample learning method, and to enhance semantic representation by introducing knowledge of a large language model and innovatively implement hierarchical alignment, so that the robustness of the zero sample learning method is improved. And the recognition precision and robustness of the model in traditional and generalized zero sample learning scenes are remarkably improved.

The invention belongs to the technical field of artificial intelligence, and discloses a method for generating a migratable adversarial sample for a vision-language pre-training model based on comparative learning, and the method comprises the steps: carrying out the discrimination of an image-text pair set obtained through the data enhancement of the vision-language pre-training model, and obtaining a positive sample set and a negative sample set; in the positive sample guided adversarial sample generation process, negative samples are introduced to further enrich the diversity of the adversarial samples, so that cross-modal interaction is realized to the greatest extent; in the adversarial sample iteration generation process, through combination of three different types of learning modes of positive sample learning, negative sample learning and contrast learning, a transferable vision-language pre-training model adversarial sample is guided to be generated. Wherein the adversarial samples generated by fusing three learning methods show better mobility in different vision-language pre-training models and downstream tasks of the different vision-language pre-training models.

The invention provides an unbiased missing modal learning method based on a multi-stage double diffusion network, and belongs to the field of computer vision, natural language processing and multi-modal information fusion. The method comprises the following steps: a multi-modal feature extraction module maps text modal data, image modal data and audio modal data into a unified potential representation space; the potential space representation of the three modals is used as the original feature of the missing modality and the original feature of the available modality; based on the missing modal original features and the available modal original features, training a multi-stage double-diffusion module through forward diffusion and reverse diffusion to obtain a trained multi-stage double-diffusion module, and based on the trained multi-stage double-diffusion module, performing global structure generation and modal conversion through available modal data to obtain a multi-stage double-diffusion model; and performing local detail optimization reasoning to generate missing modal data. According to the method, the problem of modal generation deviation in the existing multi-modal learning is solved, and the effectiveness of the multi-modal learning effect in the missing scene is enhanced.

The invention discloses a privacy protection federated learning method and system based on Transform and knowledge distillation, and belongs to the technical field of artificial intelligence and network security, and the method comprises the steps: taking an attention mechanism of a Transform model as a core component of local feature extraction, so as to capture a data long-distance dependency relationship and improve the feature representation quality; a Paillier encryption protocol is introduced to realize homomorphic encryption transmission of model weights; the knowledge distillation technology is adopted at the central server side, the aggregation global model serves as a teacher model to extract soft knowledge, and the feedback client side compresses the model and optimizes the model; according to the invention, the Transform is used as a local feature extractor, the Paillier encryption protocol is combined, and the knowledge distillation technology is adopted after the central server is aggregated, so that the detection performance under Non-IID data is optimized, and both data security and detection efficiency are realized.

The invention discloses an asynchronous hierarchical federated learning method, system, device and medium, and relates to the technical field of federated learning, and the method comprises the steps: clustering devices with similar training time in the same cluster at a client-edge end by using a mode based on a dynamic time window, and carrying out the clustering of the devices in the same cluster; dynamically grouping the equipment and asynchronously receiving local model update uploaded by the client; and at the edge end-central server, the performance evaluation and the historical participation degree of the edge server are dynamically scored, and the weight of the edge server participating in the global model aggregation is dynamically adjusted according to the score, so that a plurality of edge aggregation models are subjected to new global aggregation to obtain the global model; according to the method, the equipment is dynamically grouped, the local model is updated, the problem of equipment heterogeneity in the equipment is solved, the weight of the edge server participating in global model aggregation is dynamically adjusted according to the score, the heterogeneity condition existing in data is counteracted, and the federal learning aggregation efficiency is greatly improved.

The invention discloses a federal learning method and system based on global prototype guidance. The method comprises the following steps: a server sends a global model and a global prototype to a local client; the local client performs rebalance comparison learning by using the global prototype, and updates the local model; the local client generates a balanced confrontation positive prototype instance set and trains a classifier; the local client side calculates a local prototype and uploads the local prototype and the updated local model to the server side; and the service aggregates the updated local model and local prototype, and updates the global model and global prototype. The system comprises a server and a local client. According to the method, the Non-IID problem in federated learning is solved, and the accuracy and generalization of the model are improved. The method can be widely applied to the technical field of distributed machine learning.

The invention relates to the technical field of computers, and discloses a fragmentation block chain federal learning method based on a large language model multi-agent, and the method comprises the steps: S1, initializing a client; s2, dynamic fragmentation scheduling and distribution; s3, generating and uploading local knowledge; s4, intelligent agent collaborative routing and knowledge acquisition; s5, knowledge fusion and model updating; and S6, repeatedly executing the steps S3 to S5 until the model converges or reaches a preset number of iterations. According to the invention, under a decentralized and fragmented federated learning architecture, the complex reasoning ability of a large language model and the autonomous cooperation mechanism of three multi-agent systems, namely a fragmented scheduling agent, a fragmented knowledge state agent and a global knowledge routing agent, are deeply fused; according to the mechanism, dynamic optimization of a bottom layer fragment structure and intelligent routing of high-value knowledge are achieved in an intelligent mode, and therefore the overall efficiency and model performance of a system under the condition of heterogeneous data and heterogeneous equipment are remarkably improved.

The invention discloses an industrial Internet of Things federated learning method based on a federated increment decision tree, and the method comprises the steps: a cloud server deploys and initializes a federated learning global model and the federated increment decision tree, and sets a statistical histogram bucket boundary set of each feature value; in each federated learning iteration, the cloud server broadcasts a federated learning global model parameter, each industrial device adopts a local data set to train and count to obtain a local gradient histogram parameter, the edge server performs local aggregation on the local gradient histogram parameter and the federated learning model parameter, and the edge server performs local aggregation on the local gradient histogram parameter and the federated learning global model parameter; and the cloud server globally aggregates the local aggregation parameters of the gradient histogram and then incrementally trains the federated increment decision tree, simultaneously aggregates the parameters of a federated learning global model, and adaptively calculates an aggregation weight based on a second-order gradient value during local aggregation and global aggregation of the parameters of the federated learning model. The federal learning overhead can be effectively reduced, and the performance of the federal learning model is improved.

The invention discloses a water quality probability forecasting method based on Bayesian multi-time-sequence deep learning. The method comprises the following steps: S1, determining a forecasted water environment water ecological index, a driving index and a forecasting day number; s2, collecting time sequence data monitored by the forecasting indexes and the driving indexes, and after data preprocessing, constructing a data set required by model construction; s3, carrying out data division on the time sequence data, constructing a driving index forecasting model by adopting a multi-time sequence deep learning method, and carrying out parameter learning by selecting a Bayesian random discarding method; s4, performing effect evaluation on the accuracy and precision of the model, and adopting a hyper-parameter optimization method to improve the simulation forecast effect; s5, carrying out model training by adopting all data without segmenting the training set and the test set, carrying out water quality probability forecasting by utilizing the trained model, and outputting a forecasting mean value and a confidence interval; according to the method, the confidence interval is output while high-precision prediction is provided, and the scientificity and stability of prediction are improved.

The invention discloses a data center uninterruptible power supply intelligent control method and device based on reinforcement learning, and relates to the technical field of energy management. The method comprises the steps that a health perception state space is constructed, and the state space comprises operation state features and standby battery multi-dimensional health vectors generated through training on a plurality of distributed uninterruptible power supply stations through a federated learning method; defining a multi-target reward function for comprehensively quantifying the economic cost based on the time-of-use electricity price, the standby duration satisfaction degree of the service level agreement and the battery life loss cost; through a reinforcement learning algorithm with a security constraint layer, based on the state space and the reward function, training an intelligent agent to learn an optimal power supply control strategy, the security constraint layer forcibly executing a preset operation security boundary in training and execution; and according to the learned optimal strategy, a real-time control instruction for the uninterruptible power supply is generated and executed.

The invention relates to a multi-modal personalized federated learning method based on a hybrid expert adapter, and belongs to the technical field of multi-modal large model federated learning. The method comprises the following steps: loading a pre-training vision-language model as a frozen backbone network, constructing an adapter module, and constructing a multi-modal personalized federated learning model; initializing shared adapter parameters and private adapter parameters to complete local model construction; receiving the latest shared adapter parameter, covering and updating, and inheriting the private adapter parameter of the previous round; a model is trained through a local multi-modal data set, after image and text data are subjected to feature extraction through a backbone network, comparison loss is calculated and adapter parameters are updated through a private dimension reduction layer, a shared hybrid expert layer and a private dimension raising layer, and expert network activation cumulative frequency is counted to generate a contribution statistical vector; after relevant parameters are uploaded, the gating network performs weighted averaging according to the data volume, and the expert network performs weighted aggregation in combination with the data volume and the utilization rate to generate a global sharing model. According to the invention, personalized federal learning in a multi-modal scene is realized.