A method, apparatus, device and medium for updating a fundus image recognition model

Abstract

This invention discloses a method, apparatus, device, and medium for updating a fundus image recognition model, relating to the field of ophthalmology. The method includes: determining a set of target features for the fundus image based on the acquired knowledge of fundus image recognition model samples corresponding to the user terminal to be updated; acquiring the activation values ​​of neurons in the recording network corresponding to each user terminal to be updated; constructing a local fundus image recognition model subgraph based on the fundus image recognition model sample knowledge, activation values, and a set of common features of the fundus image; transmitting all local fundus image recognition model subgraphs to a server; performing subgraph fusion on all the local fundus image recognition model subgraphs to obtain a fused subgraph; and updating the local fundus image recognition model using the fused subgraph. This invention transforms the fusion and updating of the user terminal model into an operation of subgraph fusion to generate a full image, ensuring privacy and security throughout the model fusion cycle based on multiple privacy computing technologies, and achieving secure model transmission and incremental fusion.

Description

Technical Field

[0001] This invention relates to the field of ophthalmology, and in particular to a method, apparatus, device, and medium for updating a fundus image recognition model. Background Technology

[0002] In ophthalmology applications and medical auxiliary diagnostic models, the dynamic changes in the data feature space have become a significant issue due to the expansion of ophthalmology business scenarios and the increase in medical data characteristics. This is particularly true in the field of medical image diagnosis, where the increasing volume of medical image data and the deepening of clinical research have led to a continuous increase in the feature dimensions and diversity required by diagnostic models. However, these changes in the data feature space can negatively impact the performance of deployed models, even rendering them unable to adapt to new ophthalmology needs, posing a significant challenge to the continued application of medical auxiliary diagnostic models. In ophthalmology applications and medical auxiliary diagnostic models, re-integrating multiple users for data-level federated learning faces resource and time overhead issues. Traditional federated learning methods require the aggregation and processing of data from various parties on a central server, which consumes substantial computing resources and communication overhead. Furthermore, "model federated optimization" using pre-trained models deployed on terminals also carries the risk of model privacy leakage.

[0003] Existing methods for addressing feature data changes primarily focus on incremental learning at the data level, with limited research on secure model transfer and incremental updates under privacy and security constraints. In practical applications, secure model transfer and incremental fusion are crucial real-world problems. Existing research is inadequate when models experience precipitous performance drops or become unusable due to changes in the data feature space. For example, incremental learning methods at the data level are difficult to implement when training samples are unavailable; the aforementioned model fusion methods still cannot solve the model fusion problem caused by input layer misalignment due to feature space changes. Furthermore, the processes involved also raise security concerns regarding sensitive data features and model transfer. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for updating a fundus image recognition model, capable of secure model transfer and incremental fusion. The specific solution is as follows:

[0005] In a first aspect, the present invention discloses a method for updating a fundus image recognition model, applied to a user terminal, comprising:

[0006] Obtain the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes the fundus image common feature set and the fundus image differential feature set;

[0007] Based on the preset training set and preset inference model, the activation value of the recording network neuron corresponding to each user terminal to be updated is obtained, and a local fundus image recognition model subgraph of each user terminal to be updated is constructed based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus image.

[0008] All the local fundus image recognition model sub-images are transmitted to the server so that a preset sub-image fusion operation is performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image.

[0009] The system receives the fused sub-image sent by the server and uses the fused sub-image to update the local fundus image recognition model.

[0010] Optionally, obtaining the fundus image recognition model sample knowledge corresponding to the user terminal to be updated includes:

[0011] Obtain the feature set contained in each of the local data of the user terminal to be updated, and generate a corresponding secret share based on the feature set; wherein the number of secret shares is less than or equal to the number of features in the feature set;

[0012] Determine the receiving user terminal and the sending user terminal from all the user terminals to be updated;

[0013] Sequentially determine whether each feature in the receiver feature set has a corresponding target feature in the sender feature set; the receiver feature set is the set corresponding to the receiver user terminal, and the sender feature set is the set corresponding to the sender user terminal;

[0014] If the target feature exists in the sender feature set, the target secret share corresponding to the target feature in the sender secret share is sent to the receiving user terminal; the sender secret share is the secret share corresponding to the sender feature set of the sender terminal.

[0015] Optionally, determining the target feature set of the fundus image based on the sample knowledge of the fundus image recognition model includes:

[0016] Perform an XOR operation on all the target secret shares received by the receiving user terminal to obtain the calculation result;

[0017] If the calculation result is 0, then the target feature corresponding to the currently calculated target secret sharing is determined as a shared feature;

[0018] If the calculation result is not 0, then the target feature corresponding to the currently calculated target secret sharing is determined as the difference feature;

[0019] The fundus image common feature set is constructed based on all the common features, the fundus image differential feature set is constructed based on all the differential features, and the fundus image common feature set and the fundus image differential feature set are determined as the fundus image target feature set.

[0020] Optionally, after determining the target feature set of the fundus image based on the sample knowledge of the fundus image recognition model, the method further includes:

[0021] Obtain the local fundus image recognition model of the sending user terminal;

[0022] At the sending user end, a pre-defined verification operation is performed on the local fundus image recognition model of the sending end using a pairing verification method on an elliptic curve.

[0023] When the verification result is successful, the step of obtaining the activation value of the network neuron corresponding to each user terminal to be updated based on the preset training set and preset inference model is executed.

[0024] Optionally, the step of obtaining the activation values ​​of the network neurons corresponding to each user terminal to be updated based on a preset training set and a preset inference model, and constructing a local fundus image recognition model subgraph for each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation values, and the common feature set of the fundus images, includes:

[0025] Based on a preset training set and a preset inference model, the receiving end activation value corresponding to the receiving user end and the sending end activation value corresponding to the sending user end are obtained respectively.

[0026] The activation values ​​of the receiving end and the sending end are arranged according to a preset category order to obtain the arranged activation values, and the arranged activation values ​​are encrypted using a preset encryption algorithm.

[0027] Based on the sorted activation value corresponding to the sending user terminal, the common feature set of the fundus image, the model network structure of the sending terminal's local fundus image recognition model, and the edge features, a subgraph of the sending terminal's local fundus image recognition model is constructed.

[0028] Optionally, after constructing the local fundus image recognition model subgraph for each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus images, the method further includes:

[0029] Based on the localized differential privacy method, random sampling is performed on the common node set corresponding to the common feature set of the fundus image to obtain the sampled nodes;

[0030] A preset edge recombination operation is performed on all edges corresponding to the sampled nodes to obtain a reconstructed local fundus image recognition model subgraph.

[0031] Optionally, the step of transmitting all the local fundus image recognition model sub-images to the server, so as to perform a preset sub-image fusion operation on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image, includes:

[0032] All the reconstructed local fundus image recognition model subgraphs are transmitted to the server. Based on the maximum number of nodes in all the reconstructed local fundus image recognition model subgraphs, corresponding nodes to be supplemented are determined according to the maximum number of nodes and the fundus image difference feature set. The nodes to be supplemented are filled into the fundus image difference feature set, and an initialization operation is performed according to the connection weights between the nodes to be supplemented and the target node to obtain the processed subgraph. The embedding representation of the nodes with a pre-set dimension and which are not masked is updated using a graph neural network. Based on the updated first embedding representation, the updated second embedding representation, the preset graph encoder, and the preset graph decoder, a preset subgraph fusion operation is performed to obtain the fused subgraph. The target node is a node in the network located in the adjacent layer of the layer where the node to be supplemented is located.

[0033] Secondly, this invention discloses a fundus image recognition model update device, applied to a user terminal, comprising:

[0034] The feature set determination module is used to acquire the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes the fundus image common feature set and the fundus image difference feature set;

[0035] The activation value acquisition module is used to acquire the activation value of the network neuron corresponding to each user terminal to be updated based on the preset training set and the preset inference model.

[0036] The subgraph construction module is used to construct a local fundus image recognition model subgraph for each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus image.

[0037] The sub-image fusion module is used to transmit all the local fundus image recognition model sub-images to the server so that a preset sub-image fusion operation can be performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image.

[0038] The model update module is used to receive the fused sub-image sent by the server and update the local fundus image recognition model using the fused sub-image.

[0039] Thirdly, the present invention discloses an electronic device, comprising:

[0040] Memory, used to store computer programs;

[0041] A processor is configured to execute the computer program to implement the steps of the fundus image recognition model update method disclosed above.

[0042] Fourthly, the present invention discloses a computer-readable medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the fundus image recognition model update method disclosed above.

[0043] As can be seen, the present invention provides a method for updating a fundus image recognition model, comprising: acquiring fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determining a fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes a fundus image common feature set and a fundus image difference feature set; acquiring activation values ​​for recording network neurons corresponding to each user terminal to be updated based on a preset training set and a preset inference model, and constructing a local fundus image recognition model subgraph for each user terminal to be updated based on the fundus image recognition model sample knowledge, the activation values, and the fundus image common feature set; transmitting all the local fundus image recognition model subgraphs to the server so as to perform a preset subgraph fusion operation on all the local fundus image recognition model subgraphs based on the fundus image difference feature set to obtain a fused subgraph; receiving the fused subgraph sent by the server and updating the local fundus image recognition model using the fused subgraph. Therefore, this invention sends each user-end-generated subgraph to the server for subgraph fusion, so as to transform the fusion update of the user-end model into the operation of subgraph fusion to generate the full graph. Based on multiple privacy computing technologies, it ensures the privacy and security of the model fusion throughout the entire lifecycle, realizes the secure transmission and incremental fusion of the user-end model to effectively adapt to changes in data features, reduces the overhead of data transmission and centralized computing, and protects the privacy of the model. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0045] Figure 1 This is a flowchart of a fundus image recognition model update method disclosed in this invention;

[0046] Figure 2 This is a schematic diagram of a specific fundus image recognition model update disclosed in this invention;

[0047] Figure 3 This is a flowchart of a specific fundus image recognition model update method disclosed in this invention;

[0048] Figure 4 This is a flowchart of a specific fundus image recognition model update method disclosed in this invention;

[0049] Figure 5 This is a schematic diagram of the structure of the fundus image recognition model updating device provided by the present invention;

[0050] Figure 6 This invention provides a structural diagram of an electronic device. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Currently, existing methods for addressing feature data changes primarily focus on incremental learning, with limited research on secure model transfer and incremental updates under privacy and security constraints. In practical applications, secure model transfer and incremental fusion are crucial issues. Existing research is inadequate when models experience precipitous performance drops or become unusable due to changes in the data feature space. For example, incremental learning methods at the data level are difficult to implement when training samples are unavailable; the aforementioned model fusion methods still cannot solve the model fusion problem caused by input layer misalignment due to feature space changes, and there are also security concerns regarding sensitive data features and model transfer. Therefore, this invention provides a fundus image recognition model update method that enables secure model transfer and incremental fusion for user-side models to effectively adapt to changes in data features, reducing data transmission and centralized computation overhead while protecting model privacy.

[0053] This invention discloses a method for updating a fundus image recognition model, see [link to relevant documentation]. Figure 1 As shown, when applied to the user end, the method includes:

[0054] Step S11: Obtain the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes the fundus image common feature set and the fundus image difference feature set.

[0055] In this embodiment, the fundus image recognition model sample knowledge corresponding to the user terminal to be updated is obtained, and the fundus image target feature set is determined based on the fundus image recognition model sample knowledge; the fundus image target feature set includes a fundus image common feature set and a fundus image difference feature set. Specifically, the feature set contained in the local data of each user terminal to be updated is obtained, and a corresponding secret share is generated according to the feature set; wherein, the number of secret shares is less than or equal to the number of features in the feature set; the receiving user terminal and the sending user terminal are determined from all the user terminals to be updated; it is sequentially determined whether each feature in the receiving end feature set has a corresponding target feature in the sending end feature set; the receiving end feature set is the set corresponding to the receiving user terminal, and the sending end feature set is the set corresponding to the sending user terminal; if the sending end feature set has a corresponding target feature, the target secret share corresponding to the target feature in the sending end secret share is sent to the receiving user terminal; the sending end secret share is the secret share corresponding to the sending feature set of the sending user terminal. An XOR operation is performed on all the target secret shares received by the receiving user terminal to obtain a calculation result; if the calculation result is 0, the target feature corresponding to the currently calculated target secret share is determined as a common feature; if the calculation result is not 0, the target feature corresponding to the currently calculated target secret share is determined as a difference feature; a set of common features of fundus image is constructed based on all the common features, a set of difference features of fundus image is constructed based on all the difference features, and the set of common features of fundus image and the set of difference features of fundus image are determined as the set of target features of fundus image.

[0056] It is understandable that secure knowledge transfer based on multi-party privacy computation is necessary, such as the transfer of sample knowledge in fundus image recognition models based on decentralized privacy set intersection (PSI). Private Set Intersection (PSI), as a crucial preliminary step in privacy computation, identifies common samples among multiple parties before privacy computation, ensuring that samples unique to each party are not leaked. For privacy-sensitive data features, a decentralized PSI computation method is proposed for secure knowledge transfer of multi-party features. Assume each user terminal U... i Both refer to the feature set X = {x1, x2, ..., x...} contained in the local data. k Generate n secret sharing (k is greater than or equal to n) such that In each pair of user terminals U i (Sender) and U j End-to-end secret sharing is achieved between receivers using an Oblivious Pseudo-Random Function (OPRF). For example, receiver U... j For each characteristic it possesses Determine if a corresponding feature exists in the feature set corresponding to the sender. If a corresponding feature exists, then start from the sender U. i The secret share corresponding to this feature is obtained from the corresponding secret share. OPRF guarantees that when the sender U... i It also possesses the current characteristics, namely At that time, U j Able to obtain the corresponding If the sender is U i If the current feature is not available, the receiver will receive a random return value. It should be noted that each user terminal must perform the above calculations as a receiver to obtain its own secret share. The secret share is the secret share obtained from other senders after performing the above steps, if the data from each user terminal contains feature x. k ,So This achieves secret sharing without the involvement of a trusted third party. Then, the user terminal with the most data features is selected as the decryption party. Based on OPRF, the decryption party performs an XOR operation on the secret sharing values ​​obtained from all user terminals (including its own secret sharing value). This process yields the XOR result for each feature. If the XOR result is 0, it indicates that the feature is a common feature across all user terminal data. The set of common features in fundus images is denoted as... If the result of the XOR calculation is not 0, it means that the feature is a difference feature among all user terminal data.

[0057] Step S12: Based on the preset training set and preset inference model, obtain the activation value of the recording network neuron corresponding to each user terminal to be updated, and construct the local fundus image recognition model subgraph of each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus image.

[0058] In this embodiment, after obtaining the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determining the fundus image target feature set based on the fundus image recognition model sample knowledge, the activation values ​​of the recording network neurons corresponding to each user terminal to be updated are obtained based on a preset training set and a preset inference model. A local fundus image recognition model subgraph for each user terminal to be updated is then constructed based on the fundus image recognition model sample knowledge, the activation values, and the common feature set of fundus images. Specifically, the receiving end activation value corresponding to the receiving user terminal and the sending end activation value corresponding to the sending user terminal are obtained based on the preset training set and the preset inference model, respectively. The receiving end activation value and the sending end activation value are arranged according to a preset category order to obtain the arranged activation value, and the arranged activation value is encrypted using a preset encryption algorithm. A local fundus image recognition model subgraph for the sending end is constructed based on the arranged activation value corresponding to the sending user terminal, the common feature set of fundus images, the model network structure of the local fundus image recognition model of the sending end, and the edge features.

[0059] Understandably, this involves the transmission of model inference information based on Paillier homomorphic encryption. After the user selects the update target of the local fundus image recognition model based on the sample knowledge and inference capability of the fundus image recognition model, a model inference activation value transmission method based on Paillier homomorphic encryption is proposed to prepare for secure model fusion. Assume that the user-side U... i and U j The model is fused and updated. Each user terminal inputs a small amount of data (training samples) of different categories into the model for inference, records the activation values ​​of network neurons, arranges the activation values ​​according to the pre-agreed category order carried by the activation values, and encrypts the arranged activation values. Privacy calculation is performed using the additive homomorphic property of the Paillier algorithm. Without disclosing data and the model between user terminals, the model inference information such as differences in activation values ​​and parameter distributions for samples of the same layer and category in the network is realized. The privacy calculation formula is as follows:

[0060] D(E(m1, r1)E(m2, r2)mod n 2 ) = (m1, m2) mod n;

[0061] In the formula, D represents the decryption operation, E represents the encryption operation, m1 and m2 are plaintext information, n is the product of two random large prime numbers, and r1 and r2 These are random numbers (all greater than zero and less than n), and mod is the modulo operation.

[0062] Step S13: Transmit all the local fundus image recognition model sub-images to the server so that a preset sub-image fusion operation can be performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image.

[0063] In this embodiment, after constructing a local fundus image recognition model sub-graph for each user terminal to be updated based on the fundus image recognition model sample knowledge, the activation value, and the common feature set of fundus images, all the local fundus image recognition model sub-graphs are transmitted to the server so that a preset sub-graph fusion operation is performed on all the local fundus image recognition model sub-graphs based on the fundus image difference feature set to obtain the fused sub-graph. Specifically, all the reconstructed local fundus image recognition model subgraphs are transmitted to the server. Based on the maximum number of nodes in all the reconstructed local fundus image recognition model subgraphs, corresponding nodes to be supplemented are determined according to the maximum number of nodes and the fundus image difference feature set. The nodes to be supplemented are then filled into the fundus image difference feature set, and an initialization operation is performed based on the connection weights between the nodes to be supplemented and the target node to obtain a processed subgraph. A graph neural network is then used to update the embedding representations of nodes of a pre-set dimension that are not masked. Based on the updated first embedding representation, the updated second embedding representation, a preset graph encoder, and a preset graph decoder, a preset subgraph fusion operation is performed to obtain a fused subgraph. The target node is a node in the network located in an adjacent layer to the layer containing the node to be supplemented.

[0064] Step S14: Receive the fused sub-image sent by the server and update the local fundus image recognition model using the fused sub-image.

[0065] In this embodiment, all the local fundus image recognition model sub-images are transmitted to the server so that a preset sub-image fusion operation is performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image. Then, the fused sub-image sent by the server is received and the local fundus image recognition model is updated using the fused sub-image.

[0066] like Figure 2As shown, this invention securely transmits cross-user-end model knowledge, such as feature space changes of local data, network structure of local fundus image recognition models, and model parameter distribution, between user terminals using multi-party privacy computation technology. It constructs a weighted, fully connected local fundus image recognition model subgraph, and partitions and rearranges the feature parts of the subgraph by dividing and combining common nodes (common feature parts) and differential nodes (differential feature parts), achieving effective adaptation of data features. The rearranged subgraph is then sent to the server instead of the pre-trained model itself. Upon receiving the local fundus image recognition model subgraph uploaded by the user terminal, the server performs node completion within the subgraph and soft alignment between subgraphs. Based on the mutual representation of all nodes in the subgraph, the fusion and update of multiple local fundus image recognition models is transformed into the problem of fusing multiple subgraphs to generate a full graph. The fused subgraph is then sent to the user terminal for model updates, ultimately achieving secure model transmission and incremental fusion with effective adaptation to data feature changes on the user terminal. The server, upon receiving the local fundus image recognition model subgraph uploaded by the user terminal, performs node completion within the subgraph and soft alignment between subgraphs. By establishing the mutual representation relationships between nodes in the subgraph, the fusion and updating of multiple local fundus image recognition models is transformed into the problem of generating the entire image. Ultimately, secure model transfer and incremental fusion are achieved to ensure that the model can effectively adapt to changes in data features.

[0067] In one specific embodiment, for example, a doctor's client acquires the patient's fundus images into the system. The system then initializes a local fundus image recognition model based on existing fundus image recognition model knowledge. The system rapidly extracts the target feature set from the fundus images and simultaneously obtains the activation values ​​of neurons in the local model. During this process, the client continuously inputs hyperparameters into the system. Through various privacy-preserving computation techniques, the system provides the client with secure and accurate cross-institutional, cross-user, and cross-model interaction results (sample intersection, model capabilities, task requirements, etc.). Subsequently, the system constructs a local fundus image recognition model subgraph based on the fundus image recognition model sample knowledge, activation values, and the common feature set of fundus images. This subgraph contains important features of the patient's fundus images and the knowledge of the deployed model. The client then transmits the constructed local subgraph to the server. The server, supported by a graph neural network, merges the received subgraphs into a new subgraph. This process effectively integrates the information from each subgraph, further improving the model's generalization and accuracy. Finally, the fused sub-image is sent back to the user's device, where it is transformed and applied to update the local fundus image recognition model, thus achieving local model updates and optimizations. Throughout the process, the privacy and security of medical data and models are fully protected, allowing doctors to use this data with confidence for diagnosis, while also respecting and protecting patient privacy.

[0068] Understandably, by performing "model federated optimization" on pre-trained models deployed on terminals, this invention can distribute computational tasks across terminal devices, reducing the overhead of data transmission and centralized computation. During federated learning, participants need to share model updates and gradient information, potentially posing a risk of privacy leakage. By researching secure model knowledge transfer mechanisms and incremental fusion algorithms, this invention ensures the secure transfer of knowledge from locally pre-trained models while protecting model privacy during federated learning, thus resolving the potential model privacy leakage risk in "model federated optimization."

[0069] To address the issue of performance degradation or obsolescence of deployed models due to changes in data features, this invention proposes a model transfer and fusion method for intelligent ophthalmology medical auxiliary diagnosis, adapting to changes in data features. It features adaptive model deployment for ophthalmology scenarios to handle dynamic changes in the data feature space. While ensuring data security and model privacy, it considers feature space changes and data privacy security, proposing four secure model knowledge transfer methods. The fusion and update of the user-side model is transformed into the problem of subgraph fusion to generate the full graph. Multiple privacy-preserving computation techniques ensure privacy and security throughout the model fusion lifecycle. Ultimately, it achieves secure model transfer and incremental fusion that effectively adapts to changes in data features, enabling the continuous and effective application of medical auxiliary diagnosis models, significantly improving the accuracy of medical judgments, and promoting the intelligent development of the ophthalmology field. This invention does not directly use the pre-trained model itself, but instead utilizes a rearranged subgraph to replace the pre-trained model.

[0070] As can be seen, the present invention provides a method for updating a fundus image recognition model, comprising: acquiring fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determining a fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes a fundus image common feature set and a fundus image difference feature set; acquiring activation values ​​for recording network neurons corresponding to each user terminal to be updated based on a preset training set and a preset inference model, and constructing a local fundus image recognition model subgraph for each user terminal to be updated based on the fundus image recognition model sample knowledge, the activation values, and the fundus image common feature set; transmitting all the local fundus image recognition model subgraphs to the server so as to perform a preset subgraph fusion operation on all the local fundus image recognition model subgraphs based on the fundus image difference feature set to obtain a fused subgraph; receiving the fused subgraph sent by the server and updating the local fundus image recognition model using the fused subgraph. Therefore, this invention sends each user-end-generated subgraph to the server for subgraph fusion, so as to transform the fusion update of the user-end model into the operation of subgraph fusion to generate the full graph. Based on multiple privacy computing technologies, it ensures the privacy and security of the model fusion throughout the entire lifecycle, realizes the secure transmission and incremental fusion of the user-end model to effectively adapt to changes in data features, reduces the overhead of data transmission and centralized computing, and protects the privacy of the model.

[0071] See Figure 3 As shown, this embodiment of the invention discloses a method for updating a fundus image recognition model. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution.

[0072] Step S21: Obtain the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge.

[0073] Step S22: Obtain the local fundus image recognition model of the sending user terminal, and perform a preset verification operation on the local fundus image recognition model of the sending user terminal using the pairing verification method on the elliptic curve.

[0074] In this embodiment, the step of obtaining the local fundus image recognition model of the sending user terminal and performing a preset verification operation on the local fundus image recognition model of the sending user terminal using the pairing verification method on elliptic curves can be executed synchronously with step S21, or it can be executed after step S21 is completed. It is understood that the verification is performed based on the transfer of model reasoning ability using bilinear mapping zero-knowledge proof. Zero-knowledge proof is a method in which one party (the prover) proves a proposition to another party (the verifier), without revealing any information other than the fact that "the proposition is true". For each user terminal U... i Having a pre-trained model, in order to facilitate its fusion and updates with other user-end models, each user end might exaggerate or even fabricate the reasoning capabilities of the model it possesses. To address the trust issue of user-end models, a proposed approach is to employ bilinear paired zero-knowledge proofs to transfer the model's reasoning capabilities among multiple user ends. This achieves a reliable consensus on model capability assessment without sharing model information, allowing user ends to flexibly choose which models to merge and update, thus ensuring the quality of model fusion.

[0075] Assume F i ( xi =wx i +b represents the user terminal U. i The local fundus image recognition model, in order to enable U j Believe in U i The model for x j This is also effective, using pairing on an elliptic curve for verification (verification at the sending end), where w is the neural network weight and b is the bias. If U i If the model does indeed possess the aforementioned reasoning ability, then it satisfies the following formula:

[0076]

[0077] In the formula, e is an elliptic curve, and G is a base point on the curve.

[0078] Using bilinear modeling, it is sufficient to verify that the following paired equation holds, without needing to pass the model and the specific value of x.

[0079] e(wG,xG)·e(G,bG)=1;

[0080] In the formula, wG, xG, and bG are points that map model information onto the elliptic curve group, w is the neural network weight, x is the feature value, and b is the bias.

[0081] Step S23: When the verification result is successful, obtain the activation value of the recording network neuron corresponding to each user terminal to be updated based on the preset training set and preset inference model.

[0082] In this embodiment, when the verification equation in step S22 is true, it indicates that the verification is successful. At this time, the activation value of the recording network neuron corresponding to each user terminal to be updated is obtained based on the preset training set and the preset inference model.

[0083] Step S24: Construct a subgraph of the local fundus image recognition model of the sending end based on the arranged activation value corresponding to the sending user end, the common feature set of the fundus image, the model network structure of the local fundus image recognition model of the sending end, and the edge features.

[0084] In this embodiment, the user terminal U i A local fundus image recognition model subgraph G is constructed using the locally deployed model network structure, parameters, and a set of shared features from fundus images. i = <V i E i >,V i For user-side U i The local fundus image recognition model network is a set of all neurons (i.e., a set of points). The features (i.e., activation values) at each point represent the model inference information corresponding to that neuron. E i Let be the set of edges, and the features on the edges are the weights connecting the neurons. Specifically, the set of neurons, the features at the nodes, the set of edges, and the features on the edges are obtained based on the model's network structure; the weights connecting the neurons are the parameters.

[0085] It should be noted that after constructing the local fundus image recognition model subgraph, the common feature set F of fundus images is used to compare G. i Node partitioning and rearrangement are performed to obtain the reconstructed local fundus image recognition model subgraph G′. i =<(V′) iG , V′ iL ), E′ i ). Where V′ iG V′ is the set of neurons (nodes) corresponding to common features. iL E′ is the set of neurons (nodes) corresponding to the differential features. i This is the set of edges that are rearranged according to the node connection weights.

[0086] Step S25: Randomly sample the common node set corresponding to the common feature set of the fundus image based on the localized differential privacy method to obtain the sampled nodes.

[0087] In this embodiment, after constructing a subgraph of the local fundus image recognition model based on the arranged activation values ​​corresponding to the sending user terminal, the common feature set of the fundus images, the model network structure of the local fundus image recognition model of the sending terminal, and edge features, random sampling is performed from the common node set corresponding to the common feature set of the fundus images based on the local differential privacy method to obtain the sampled nodes. It can be understood that the node set V′ corresponding to the common features between user terminals is obtained based on Local Differential Privacy (LDP). iG Perform random sampling that satisfies LDP to obtain the sampled nodes.

[0088] Step S26: Perform a preset edge recombination operation on all edges corresponding to the sampled nodes to obtain a reconstructed local fundus image recognition model subgraph.

[0089] In this embodiment, a localized differential privacy method is used to randomly sample the common node set corresponding to the common feature set of the fundus image. After obtaining the sampled nodes, a preset edge recombination operation is performed on all edges corresponding to the sampled nodes to obtain a reconstructed local fundus image recognition model subgraph. It can be understood that the privacy budgets ∈ and V′ in the localized differential privacy method are adjusted according to different needs during the edge recombination of the sampled nodes. iG Number of nodes sampled m iG This approach balances the privacy and security of the local fundus image recognition model with the usability of its sub-images. Finally, each user client uploads its constructed and privacy-protected local fundus image recognition model sub-images to the server for secure model fusion.

[0090] Step S27: Transmit all the local fundus image recognition model sub-images to the server so that a preset sub-image fusion operation can be performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image.

[0091] In this embodiment, a preset edge recombination operation is performed on all edges corresponding to the sampled nodes to obtain a recombined local fundus image recognition model subgraph. Then, all the local fundus image recognition model subgraphs are transmitted to the server so that a preset subgraph fusion operation is performed on all the local fundus image recognition model subgraphs based on the fundus image difference feature set to obtain a fused subgraph. The model generated by the subgraph fusion is then safely transferred and incrementally updated.

[0092] Understandably, after receiving the local fundus image recognition model sub-image uploaded by the user, the server completes the sub-image by filling in the nodes according to the criterion of maximizing the number of nodes, and places the supplemented nodes into the node set V′ corresponding to the differential features.iL In this approach, the connection weights between supplementary nodes and other nodes (located in adjacent layers of the network) are initialized according to the average weights of that layer, thereby achieving soft alignment between subgraphs. For nodes within a subgraph, an initial embedding representation of dimension d is set. Random masking of some nodes is performed through self-supervised methods. A Graph Neural Network (GNN) is then used to update the neighbor embedding representations and embedding representations of the unmasked nodes. The mutual representations and embedding representation updates of nodes in the l-th layer of the GNN are as follows:

[0093]

[0094]

[0095] Where r is the embedding representation of each node in each layer, L is local, and G is global, representing the mutual representation between nodes L and G. The node representation of layer l is updated by passing it from layer l-1. In the formula, E indicates that the k-th dimension feature exists (is not 0), COMB is the merge operation, AGG is the aggregation operation, and P... (l) It is a weight matrix. For the masked nodes, the initial state of their embedding representation is set to a zero vector of dimension d. The updated subgraph embedding representation can adequately represent neurons corresponding to common and dissimilar features as neighbors. Let the GNN used to generate node representations be the graph encoder q. φ (r′ i |G′ i ), jointly train a graph decoder p θ (G|r′ i ,λAGG(r′) iG ), μAGG(r′ iL The entire image is then fused and generated. The overall optimization loss function is:

[0096]

[0097] Where L is the loss function, and the information in parentheses represents adjustable or learnable parameters. For the desired outcome, KL represents the KL (Kullback-Leibler divergence), λ and μ are scale parameters, and AGG is the aggregation method of the embedding representation, which can be selected according to different fusion requirements. Based on this, each user terminal can freely and securely share the graph embedding representation representing the local fundus image recognition model, and transfer and fuse it among multiple parties, realizing the secure transfer and incremental update of the model.

[0098] It is understood that the fusion parameters are updated based on the graph decoder, the graph encoder, and the aforementioned loss function in order to obtain the fused subgraph based on the mutual representation, the embedded representation, and the updated fusion parameters.

[0099] like Figure 4 As shown, the steps in this invention starting from the user end are as follows: Construct a local fundus image recognition model sub-graph, perform multi-party privacy calculations, and upload the constructed sub-graph to the server for sub-graph fusion. Then, receive the updated sub-graph (i.e., the fused sub-graph) sent by the server, convert the updated sub-graph into a model, and perform model performance testing to determine if the model performance meets the requirements. If it does, the process ends; otherwise, it re-enters the step of uploading the constructed sub-graph to the server for sub-graph fusion. The steps in this invention starting from the server end are as follows: Receive the sub-graph after feature space changes and privacy processing, complete the sub-graph nodes, perform soft alignment, rearrange and fuse the soft-aligned sub-graph, transmit the fused sub-graph to the corresponding user end for model updates, receive update information from the user end, and determine if the model performance meets the requirements. If it does, the process ends; otherwise, it re-enters the sub-graph soft alignment step.

[0100] To securely transfer knowledge from locally pre-trained models and effectively achieve incremental fusion updates, this invention studies secure model knowledge transfer mechanisms and incremental fusion algorithms to ensure model privacy protection and achieve continuous optimization and improvement of model performance. This provides an effective solution for federated learning applications of ophthalmic medical auxiliary diagnostic models, ensuring data security and privacy protection, and providing strong support for the intelligent development of the ophthalmic medical field.

[0101] Step S28: Receive the fused sub-image sent by the server and update the local fundus image recognition model using the fused sub-image.

[0102] For details regarding steps S21 and S28, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.

[0103] As can be seen, this application embodiment obtains the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determines the fundus image target feature set based on the fundus image recognition model sample knowledge; obtains the sending end local fundus image recognition model of the sending user terminal, and performs a preset verification operation on the sending end local fundus image recognition model using the pairing verification method on the elliptic curve; when the verification result is successful, obtains the activation value of the recording network neuron corresponding to each user terminal to be updated based on the preset training set and the preset inference model; constructs the sending end local fundus image recognition model subgraph based on the sorted activation value corresponding to the sending user terminal, the common feature set of the fundus image, the model network structure of the sending end local fundus image recognition model, and the edge features. The system randomly samples from the common node set corresponding to the common feature set of the fundus image based on the localized differential privacy method to obtain sampled nodes; it performs a preset edge recombination operation on all edges corresponding to the sampled nodes to obtain a recombined local fundus image recognition model subgraph; it transmits all the local fundus image recognition model subgraphs to the server so that a preset subgraph fusion operation is performed on all the local fundus image recognition model subgraphs based on the fundus image difference feature set to obtain a fused subgraph; it receives the fused subgraph sent by the server and uses the fused subgraph to update the local fundus image recognition model, realizing the secure transmission and incremental fusion of the model for effective adaptation to changes in data features on the user end, reducing the overhead of data transmission and centralized computing, and protecting the privacy of the model.

[0104] See Figure 5 As shown, this embodiment of the invention also discloses a fundus image recognition model update device, applied to a user terminal, comprising:

[0105] The feature set determination module 11 is used to acquire the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes the fundus image common feature set and the fundus image difference feature set;

[0106] The activation value acquisition module 12 is used to acquire the activation value of the network neuron corresponding to each user terminal to be updated based on the preset training set and the preset inference model.

[0107] Subgraph construction module 13 is used to construct a local fundus image recognition model subgraph for each user terminal to be updated based on the fundus image recognition model sample knowledge, the activation value, and the common feature set of fundus images.

[0108] The sub-image fusion module 14 is used to transmit all the local fundus image recognition model sub-images to the server so that a preset sub-image fusion operation is performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image.

[0109] The model update module 15 is used to receive the fused sub-image sent by the server and update the local fundus image recognition model using the fused sub-image.

[0110] As can be seen, the present invention includes: acquiring fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determining a fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes a fundus image common feature set and a fundus image difference feature set; acquiring activation values ​​for recording network neurons corresponding to each user terminal to be updated based on a preset training set and a preset inference model, and constructing a local fundus image recognition model subgraph for each user terminal to be updated based on the fundus image recognition model sample knowledge, the activation values, and the fundus image common feature set; transmitting all the local fundus image recognition model subgraphs to the server so as to perform a preset subgraph fusion operation on all the local fundus image recognition model subgraphs based on the fundus image difference feature set to obtain a fused subgraph; receiving the fused subgraph sent by the server and updating the local fundus image recognition model using the fused subgraph. Therefore, this invention sends each user-end-generated subgraph to the server for subgraph fusion, so as to transform the fusion update of the user-end model into the operation of subgraph fusion to generate the full graph. Based on multiple privacy computing technologies, it ensures the privacy and security of the model fusion throughout the entire lifecycle, realizes the secure transmission and incremental fusion of the user-end model to effectively adapt to changes in data features, reduces the overhead of data transmission and centralized computing, and protects the privacy of the model.

[0111] In some specific embodiments, the feature set determination module 11 specifically includes:

[0112] The feature set acquisition unit is used to acquire the feature set contained in each of the local data of the user terminal to be updated;

[0113] A secret sharing generation unit is used to generate corresponding secret shares based on the feature set; wherein the number of secret shares is less than or equal to the number of features in the feature set;

[0114] The user terminal determination unit is used to determine the receiving user terminal and the sending user terminal from all the user terminals to be updated.

[0115] The target feature determination unit is used to sequentially determine whether each feature in the receiver feature set has a corresponding target feature in the sender feature set; the receiver feature set is the set corresponding to the receiver user end, and the sender feature set is the set corresponding to the sender user end;

[0116] A target secret sharing sending unit is configured to send a target secret sharing corresponding to the target feature in the sending end feature set to the receiving user end if the sending end feature set contains a corresponding target feature; the sending end secret sharing is a secret sharing corresponding to the sending feature set of the sending user end.

[0117] The XOR calculation unit is used to perform an XOR calculation on all the target secret shares received by the receiving user terminal to obtain the calculation result;

[0118] A common feature determination unit is used to determine the target feature corresponding to the currently calculated target secret sharing as a common feature if the calculation result is 0;

[0119] A difference feature determination unit is used to determine the target feature corresponding to the currently calculated target secret sharing as a difference feature if the calculation result is not 0.

[0120] A fundus image common feature set construction unit is used to construct the fundus image common feature set based on all the common features;

[0121] A fundus image differential feature set construction unit is used to construct the fundus image differential feature set based on all the differential features;

[0122] A fundus image target feature set determination unit is used to determine the fundus image target feature set as the fundus image common feature set and the fundus image differential feature set;

[0123] The sending end local fundus image recognition model acquisition unit is used to acquire the sending end local fundus image recognition model of the sending user end;

[0124] The verification unit is used to perform a preset verification operation on the local fundus image recognition model of the sending end using a pairing verification method on an elliptic curve at the sending user end.

[0125] In some specific embodiments, the activation value acquisition module 12 specifically includes:

[0126] The activation value acquisition unit is used to acquire the receiving end activation value corresponding to the receiving user end and the sending end activation value corresponding to the sending user end based on the preset training set and the preset inference model when the verification result is verification passed.

[0127] An activation value sorting unit is used to sort the activation values ​​of the receiving end and the sending end according to a preset category order to obtain sorted activation values.

[0128] The arrangement activation value encryption unit is used to encrypt the arrangement activation value using a preset encryption algorithm.

[0129] In some specific embodiments, the subgraph construction module 13 specifically includes:

[0130] The local fundus image recognition model subgraph construction unit is used to construct the local fundus image recognition model subgraph of the sending end based on the arranged activation value corresponding to the sending user end, the common feature set of the fundus image, the model network structure of the local fundus image recognition model of the sending end, and the edge features.

[0131] The random sampling unit is used to randomly sample from the common node set corresponding to the common feature set of the fundus image based on the localized differential privacy method to obtain the sampled nodes;

[0132] The local fundus image recognition model subgraph reconstruction unit is used to perform a preset edge reconstruction operation on all edges corresponding to the sampled nodes to obtain a reconstructed local fundus image recognition model subgraph.

[0133] In some specific embodiments, the subgraph fusion module 14 specifically includes:

[0134] The subgraph fusion unit is used to transmit all the reconstructed local fundus image recognition model subgraphs to the server. Based on the maximum number of nodes in all the reconstructed local fundus image recognition model subgraphs, it determines the corresponding nodes to be supplemented according to the maximum number of nodes and the fundus image difference feature set. The nodes to be supplemented are then filled into the fundus image difference feature set, and an initialization operation is performed according to the connection weights between the nodes to be supplemented and the target node to obtain a processed subgraph. A graph neural network is used to update the embedding representation of nodes of a pre-set dimension that are not masked. Based on the updated first embedding representation, the updated second embedding representation, a preset graph encoder, and a preset graph decoder, a preset subgraph fusion operation is performed to obtain a fused subgraph. The target node is a node in the network located in an adjacent layer to the layer containing the node to be supplemented.

[0135] In some specific embodiments, the model update module 15 specifically includes:

[0136] The merged subgraph receiving unit is used to receive the merged subgraph sent by the server;

[0137] The model update unit is used to update the local fundus image recognition model using the fused sub-image.

[0138] Furthermore, embodiments of the present invention also provide an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of the invention.

[0139] Figure 6 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present invention. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the fundus image recognition model update method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0140] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this invention, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0141] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0142] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the fundus image recognition model update method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0143] Furthermore, this embodiment of the invention also discloses a medium storing a computer program, which, when loaded and executed by a processor, implements the steps of the fundus image recognition model update method disclosed in any of the foregoing embodiments.

[0144] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0145] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0146] The foregoing has provided a detailed description of the fundus image recognition model update method, apparatus, device, and medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for updating a fundus image recognition model, characterized in that, Applied to the user end, including: Obtain the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes the fundus image common feature set and the fundus image differential feature set; Based on the preset training set and preset inference model, the activation value of the recording network neuron corresponding to each user terminal to be updated is obtained, and a local fundus image recognition model subgraph of each user terminal to be updated is constructed based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus image. All the local fundus image recognition model sub-images are transmitted to the server so that a preset sub-image fusion operation is performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image. The process involves receiving the fused sub-image sent by the server and updating the local fundus image recognition model using the fused sub-image; obtaining the fundus image recognition model sample knowledge corresponding to the user terminal to be updated includes: Obtain the feature set contained in each of the local data of the user terminal to be updated, and generate a corresponding secret share based on the feature set; wherein the number of secret shares is less than or equal to the number of features in the feature set; Determine the receiving user terminal and the sending user terminal from all the user terminals to be updated; Sequentially determine whether each feature in the receiver feature set has a corresponding target feature in the sender feature set; the receiver feature set is the set corresponding to the receiver user terminal, and the sender feature set is the set corresponding to the sender user terminal; If the target feature exists in the sender feature set, the target secret share corresponding to the target feature in the sender secret share is sent to the receiving user terminal; the sender secret share is the secret share corresponding to the sender feature set of the sender terminal.

2. The fundus image recognition model update method according to claim 1, characterized in that, The determination of the target feature set of the fundus image based on the sample knowledge of the fundus image recognition model includes: Perform an XOR operation on all the target secret shares received by the receiving user terminal to obtain the calculation result; If the calculation result is 0, then the target feature corresponding to the currently calculated target secret sharing is determined as a shared feature; If the calculation result is not 0, then the target feature corresponding to the currently calculated target secret sharing is determined as the difference feature; The fundus image common feature set is constructed based on all the common features, the fundus image differential feature set is constructed based on all the differential features, and the fundus image common feature set and the fundus image differential feature set are determined as the fundus image target feature set.

3. The fundus image recognition model update method according to claim 1, characterized in that, After determining the target feature set of the fundus image based on the sample knowledge of the fundus image recognition model, the method further includes: Obtain the local fundus image recognition model of the sending user terminal; At the sending user end, a pre-defined verification operation is performed on the local fundus image recognition model of the sending end using a pairing verification method on an elliptic curve. When the verification result is successful, the step of obtaining the activation value of the network neuron corresponding to each user terminal to be updated based on the preset training set and preset inference model is executed.

4. The fundus image recognition model update method according to claim 3, characterized in that, The step of obtaining the activation values ​​of the network neurons corresponding to each user terminal to be updated based on a preset training set and a preset inference model, and constructing a local fundus image recognition model subgraph for each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation values, and the common feature set of the fundus images, includes: Based on a preset training set and a preset inference model, the receiving end activation value corresponding to the receiving user end and the sending end activation value corresponding to the sending user end are obtained respectively. The activation values ​​of the receiving end and the sending end are arranged according to a preset category order to obtain the arranged activation values, and the arranged activation values ​​are encrypted using a preset encryption algorithm. Based on the sorted activation value corresponding to the sending user terminal, the common feature set of the fundus image, the model network structure of the sending terminal's local fundus image recognition model, and the edge features, a subgraph of the sending terminal's local fundus image recognition model is constructed.

5. The fundus image recognition model update method according to any one of claims 1 to 4, characterized in that, After constructing the local fundus image recognition model subgraph for each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus image, the process further includes: Based on the localized differential privacy method, random sampling is performed on the common node set corresponding to the common feature set of the fundus image to obtain the sampled nodes; A preset edge recombination operation is performed on all edges corresponding to the sampled nodes to obtain a reconstructed local fundus image recognition model subgraph.

6. The fundus image recognition model update method according to claim 5, characterized in that, The step of transmitting all the local fundus image recognition model sub-images to the server, so as to perform a preset sub-image fusion operation on all the local fundus image recognition model sub-images based on the fundus image difference feature set, to obtain the fused sub-image, includes: All the reconstructed local fundus image recognition model subgraphs are transmitted to the server. Based on the maximum number of nodes in all the reconstructed local fundus image recognition model subgraphs, corresponding nodes to be supplemented are determined according to the maximum number of nodes and the fundus image difference feature set. The nodes to be supplemented are filled into the fundus image difference feature set, and an initialization operation is performed according to the connection weights between the nodes to be supplemented and the target node to obtain the processed subgraph. The embedding representation of the nodes with a pre-set dimension and which are not masked is updated using a graph neural network. Based on the updated first embedding representation, the updated second embedding representation, the preset graph encoder, and the preset graph decoder, a preset subgraph fusion operation is performed to obtain the fused subgraph. The target node is a node in the network located in the adjacent layer of the layer where the node to be supplemented is located.

7. A device for updating a fundus image recognition model, characterized in that, Applied to the user end, including: The feature set determination module is used to acquire the fundus image recognition model sample knowledge corresponding to the user terminal to be updated, and determine the fundus image target feature set based on the fundus image recognition model sample knowledge; the fundus image target feature set includes the fundus image common feature set and the fundus image difference feature set; The activation value acquisition module is used to acquire the activation value of the network neuron corresponding to each user terminal to be updated based on the preset training set and the preset inference model. The subgraph construction module is used to construct a local fundus image recognition model subgraph for each user terminal to be updated based on the sample knowledge of the fundus image recognition model, the activation value, and the common feature set of the fundus image. The sub-image fusion module is used to transmit all the local fundus image recognition model sub-images to the server so that a preset sub-image fusion operation can be performed on all the local fundus image recognition model sub-images based on the fundus image difference feature set to obtain the fused sub-image. The model update module is used to receive the fused sub-image sent by the server and update the local fundus image recognition model using the fused sub-image; The feature set determination module includes: The feature set acquisition unit is used to acquire the feature set contained in each of the local data of the user terminal to be updated; A secret sharing generation unit is used to generate corresponding secret shares based on the feature set; wherein the number of secret shares is less than or equal to the number of features in the feature set; The user terminal determination unit is used to determine the receiving user terminal and the sending user terminal from all the user terminals to be updated. The target feature determination unit is used to sequentially determine whether each feature in the receiver feature set has a corresponding target feature in the sender feature set; the receiver feature set is the set corresponding to the receiver user end, and the sender feature set is the set corresponding to the sender user end; A target secret sharing sending unit is configured to send a target secret sharing corresponding to the target feature in the sending end feature set to the receiving user end if the sending end feature set contains a corresponding target feature; the sending end secret sharing is a secret sharing corresponding to the sending feature set of the sending user end.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the fundus image recognition model update method as described in any one of claims 1 to 6.

9. A computer-readable medium, characterized in that, Used to store a computer program; wherein, when the computer program is executed by a processor, it implements the fundus image recognition model update method as described in any one of claims 1 to 6.

Images