A method, related apparatus, equipment, and storage medium for model training.

Abstract

This application discloses a model training method based on federated learning, relating to the field of artificial intelligence. The method includes: acquiring K molecular sample features and K molecular annotation features; determining the mixing cardinality of the K features according to a beta distribution; generating target molecular sample features and target molecular annotation features for the mixed molecular samples based on the mixing cardinality, the K molecular sample features, and the K molecular annotation features; obtaining a first local gradient value for the local model based on the target molecular sample features and target molecular annotation features; obtaining a second local gradient value when the model training conditions are met; and sending the second local gradient value to a central device. This application also provides related apparatus, equipment, and storage media. This application can eliminate the influence of noise, thereby improving the robustness of the model. Furthermore, under the federated learning framework, multiple institutions can jointly train a global model, which is beneficial for improving the accuracy of model predictions.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, related apparatus, device, and storage medium for model training. Background Technology

[0002] Artificial intelligence (AI) is an emerging science and technology that studies and develops methods to simulate, extend, and expand human intelligence. In recent years, AI has been applied to the medical field, and advancements in AI technology are impacting the drug development process and product lifecycles.

[0003] Currently, in the context of federal pharmaceutical manufacturing, a learning framework can be constructed using Federated Graph Neural Networks (FedGraphNN). FedGraphNN allows local models used by various institutions to be configured as deep graph neural networks, and then end-to-end model training can be achieved using the Federated Averaging (FedAVG) algorithm.

[0004] The inventors have discovered at least the following problems in existing solutions: However, in real-world scenarios, the training data generated by each institution may experience data noise due to external factors (e.g., the solvent used in the experiment and the experimental environment), leading to poor model performance on data from some institutions. Simultaneously, the training data generated by each institution is very limited, resulting in poor model prediction performance. Summary of the Invention

[0005] This application provides a method, related apparatus, device, and storage medium for model training. By mixing noise-free and noisy training data, this application can eliminate the influence of noise, thereby improving the robustness of the model. Furthermore, under a federated learning framework, multiple institutions can jointly train a global model, thus addressing the problem of insufficient environmental training data to some extent and improving the accuracy of model predictions.

[0006] In view of this, this application provides a method for model training, including:

[0007] Obtain K molecular sample features and K molecular annotation features, where the molecular sample features and molecular annotation features have a corresponding relationship, and K is an integer greater than 1;

[0008] K feature mixing cards are determined based on the beta distribution, where the sum of the K feature mixing cards is 1;

[0009] Based on the K feature mixing cardinality, K molecular sample features, and K molecular annotation features, generate the target molecular sample features and target molecular annotation features of the mixed molecular sample;

[0010] Based on the target molecule sample features and target molecule annotation features, the first local gradient value for the local model is obtained, and the second local gradient value is obtained when the model training conditions are met.

[0011] Send the second local gradient value to the central device.

[0012] This application also provides a method for model training, including:

[0013] The model parameters of the global model are sent to M local devices so that each local device uses the model parameters of the global model as the model parameters of its local model. Based on the target molecule sample features and the target molecule annotation features, the first local gradient value for the local model is obtained. When the model training conditions are met, the second local gradient value is obtained, where M is an integer greater than or equal to 1.

[0014] Receive the second local gradient value sent by each of the M local devices;

[0015] Update the model parameters of the global model based on the second local gradient values ​​sent by each of the M local devices;

[0016] Among them, the target molecule sample features and the target molecule annotation features are generated by the local device based on the K feature mixing cardinality, the K molecule sample features and the K molecule annotation features. The K feature mixing cardinality is determined according to the beta distribution. The molecule sample features and the molecule annotation features have a corresponding relationship. The sum of the K feature mixing cardinality is 1, and K is an integer greater than 1.

[0017] This application also provides a model training apparatus, comprising:

[0018] The acquisition module is used to acquire K molecular sample features and K molecular annotation features, where there is a correspondence between the molecular sample features and the molecular annotation features, and K is an integer greater than 1;

[0019] The determination module is used to determine the mixing cardinality of K features based on the beta distribution, wherein the sum of the K feature mixing cardinality is 1;

[0020] The generation module is used to generate target molecular sample features and target molecular annotation features of the mixed molecular sample based on K feature mixing cardinality, K molecular sample features and K molecular annotation features;

[0021] The acquisition module is also used to obtain the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features, and to obtain the second local gradient value when the model training conditions are met.

[0022] The sending module is used to send the second local gradient value to the central device.

[0023] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0024] The acquisition module is specifically used to acquire K molecular samples, where each molecular sample corresponds to at least one annotation result;

[0025] For each molecular sample, feature extraction is performed to obtain the molecular sample features corresponding to the molecular sample;

[0026] For each molecular sample, based on at least one annotation result corresponding to the molecular sample, a molecular annotation feature corresponding to the molecular sample is generated.

[0027] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0028] The acquisition module is specifically used to generate molecular graph samples based on molecular samples. The molecular graph samples include vertices and edges, where vertices represent atoms and edges represent chemical bonds.

[0029] Based on molecular graph samples, molecular sample features are obtained through graph neural networks.

[0030] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0031] The acquisition module is specifically used to send data access requests to the target local device;

[0032] If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1.

[0033] Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1;

[0034] P molecular samples and Q molecular samples are used as K molecular samples.

[0035] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0036] The determination module is specifically used to obtain random numbers that satisfy the beta distribution and use the random numbers as the first feature mixing base.

[0037] The second feature mixing basis is determined based on the first feature mixing basis.

[0038] In one possible design, in another implementation of another aspect of the embodiments of this application, the K molecular sample features include first molecular sample features and second molecular sample features, and the K molecular annotation features include first molecular annotation features and second molecular annotation features;

[0039] The generation module is specifically used to weight the features of the first molecular sample using the first feature mixing basis to obtain the first feature vector;

[0040] The second feature vector is obtained by weighting the features of the second molecular sample using the second feature mixing basis.

[0041] Based on the first feature vector and the second feature vector, generate the target molecular sample features of the mixed molecular sample;

[0042] The first feature vector is obtained by weighting the first feature mixing cardinality.

[0043] The second feature vector is obtained by weighting the second feature mixing basis of the second molecule annotation features;

[0044] Based on the third and fourth feature vectors, target molecular annotation features of the mixed molecular samples are generated.

[0045] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0046] The determination module is specifically used to obtain at least three random numbers that satisfy a beta distribution;

[0047] Based on the sum of at least three random numbers, normalize each random number to obtain the feature mixing cardinality corresponding to each random number.

[0048] In one possible design, in another implementation of another aspect of the embodiments of this application, the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features;

[0049] The generation module is specifically used to use the feature mixing base corresponding to each random number to perform a weighted summation of the features of at least three molecular samples to obtain the target molecular sample features of the mixed molecular samples;

[0050] By using the feature mixing cardinality corresponding to each random number, at least three molecular annotation features are weighted and summed to obtain the target molecular annotation features of the mixed molecular sample.

[0051] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0052] The acquisition module is also used to, before acquiring the first local gradient value for the local model based on the target molecule sample features and target molecule annotation features, respond to the model update instruction sent by the central device, acquire the model parameters of the global model from the central device, and use the model parameters of the global model as the model parameters of the local model.

[0053] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0054] The acquisition module is specifically used to obtain the molecular attribute prediction scores of the mixed molecular samples based on the characteristics of the target molecular samples through a local model. The molecular attribute prediction scores represent the predicted scores for the target molecular attributes.

[0055] Based on the predicted scores of molecular attributes and the labeled features of the target molecules, the first local gradient value for the local model is determined by the first loss function, where the labeled features of the target molecules represent the actual scores for the target molecular attributes.

[0056] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0057] The acquisition module is specifically used to obtain the molecular attribute prediction distribution of the mixed molecular samples based on the features of the target molecular samples through a local model. The molecular attribute prediction distribution represents the probability of predicting that the sample belongs to each molecular attribute.

[0058] Based on the molecular attribute prediction distribution and the target molecule annotation features, the first local gradient value for the local model is determined by the second loss function, where the target molecule annotation features represent the probability that the annotation belongs to each molecular attribute.

[0059] This application also provides a model training apparatus, comprising:

[0060] The sending module is used to send the model parameters of the global model to M local devices, so that each local device uses the model parameters of the global model as the model parameters of its local model, and obtains the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features. When the model training conditions are met, the second local gradient value is obtained, where M is an integer greater than or equal to 1.

[0061] The receiving module is used to receive the second local gradient value sent by each of the M local devices;

[0062] The update module is used to update the model parameters of the global model based on the second local gradient value sent by each of the M local devices;

[0063] Among them, the target molecule sample features and the target molecule annotation features are generated by the local device based on the K feature mixing cardinality, the K molecule sample features and the K molecule annotation features. The K feature mixing cardinality is determined according to the beta distribution. The molecule sample features and the molecule annotation features have a corresponding relationship. The sum of the K feature mixing cardinality is 1, and K is an integer greater than 1.

[0064] In one possible design, in another implementation of another aspect of the embodiments of this application, the model training device further includes an acquisition module and a sampling module;

[0065] The acquisition module is used to acquire the first weight of each of the T local devices before sending the model parameters of the global model to the M local devices, where T is an integer greater than or equal to M, and the sum of the first weights of the T local devices is 1.

[0066] The sampling module is used to sample M local devices from the T local devices based on the first weight corresponding to each local device in the T local devices and through the probability density function.

[0067] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0068] The receiving module is also used to receive the loss value sent by each of the M local devices;

[0069] The update module is also used to update the first weight corresponding to each of the M local devices based on the loss value sent by each of the M local devices, so as to obtain the second weight corresponding to each of the M local devices.

[0070] In one possible design, in another implementation of another aspect of the embodiments of this application,

[0071] The update module is specifically used to update the first weight corresponding to each of the M local devices based on the loss value sent by each of the M local devices, so as to obtain the weight to be processed corresponding to each of the M local devices.

[0072] Based on the sum of the unprocessed weights corresponding to each of the M local devices, normalize each unprocessed weight to obtain the normalized weights corresponding to each of the M local devices.

[0073] Based on the sum of the remaining weights and the normalized weights corresponding to each of the M local devices, determine the second weight corresponding to each of the M local devices.

[0074] This application also provides a computer device, including: a memory, a processor, and a bus system;

[0075] The memory is used to store programs;

[0076] The processor is used to execute programs in memory, and the processor is used to execute the methods mentioned above according to the instructions in the program code;

[0077] Bus systems are used to connect memory and processor to enable communication between them.

[0078] Another aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the methods described above.

[0079] Another aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the above aspects.

[0080] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0081] This application provides a federated learning-based model training method. First, K molecular sample features and K molecular annotation features are obtained. Furthermore, the mixing cardinality of the K features needs to be determined based on the beta distribution. Then, based on the K feature mixing cardinality, K molecular sample features, and K molecular annotation features, target molecular sample features and target molecular annotation features of the mixed molecular samples can be generated. Then, based on the target molecular sample features and target molecular annotation features, a first local gradient value for the local model is obtained. When the model training conditions are met, a second local gradient value is obtained. Finally, the second local gradient value is sent to the central device, which uses the second local gradient value to update the model parameters of the global model. Although the training data from various institutions may contain noise, some training data is actually noise-free. Therefore, this application introduces feature mixing technology. By mixing noise-free training data with noisy training data, the influence of noise can be eliminated, thereby improving the robustness of the model. At the same time, on the one hand, data augmentation is achieved by constructing new training data; on the other hand, under the federated learning framework, multiple institutions can jointly train a global model, which to some extent solves the problem of insufficient training data in the environment and helps to improve the accuracy of model prediction. Attached Figure Description

[0082] Figure 1 A diagram illustrating data relationships in horizontal federated learning;

[0083] Figure 2 This is a schematic diagram of the architecture of the federated learning system in the embodiments of this application;

[0084] Figure 3 This is a flowchart illustrating the model training method in an embodiment of this application;

[0085] Figure 4 This is a schematic diagram of molecular encoding based on graph neural networks in an embodiment of this application;

[0086] Figure 5 This is a schematic diagram of an architecture for sharing samples among multiple local devices in an embodiment of this application;

[0087] Figure 6 This is a schematic diagram of a training framework based on a regression task in an embodiment of this application;

[0088] Figure 7 This is a schematic diagram of a training framework based on a classification task in an embodiment of this application;

[0089] Figure 8 This is another flowchart illustrating the model training method in this application embodiment;

[0090] Figure 9This is a schematic diagram of a model training device in an embodiment of this application;

[0091] Figure 10 This is another schematic diagram of the model training device in the embodiments of this application;

[0092] Figure 11 This is a schematic diagram of the server structure in an embodiment of this application;

[0093] Figure 12 This is a schematic diagram of the structure of a terminal device in an embodiment of this application. Detailed Implementation

[0094] This application provides a method, related apparatus, device, and storage medium for model training. By mixing noise-free and noisy training data, this application can eliminate the influence of noise, thereby improving the robustness of the model. Furthermore, under a federated learning framework, multiple institutions can jointly train a global model, thus addressing the problem of insufficient environmental training data to some extent and improving the accuracy of model predictions.

[0095] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “corresponding to,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0096] In recent years, the pharmaceutical industry has seen significant growth in data digitization. The challenge brought by digitization is applying this data to solve complex problems. This has spurred the use of Artificial Intelligence (AI), and its application in the pharmaceutical field is continuously expanding. AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0097] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0098] The following will introduce several application scenarios of AI in the medical field.

[0099] (1) Discovering drug targets: AI can be used to search for potential target information from a large amount of literature and compare the potential of different targets.

[0100] (2) Drug screening and structure optimization: It is necessary to evaluate the interaction ability between different candidate molecules and the target, and screen out suitable molecules. AI can make comprehensive use of various existing information to evaluate, such as protein structure, chemical information and structure-activity relationship data, to predict interactions.

[0101] (3) Property Prediction: AI can be used to predict the physicochemical properties, biological activities, and toxicity of molecules. For example, machine learning (ML) can be used to train the system using datasets generated during the optimization process. ML is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. ML is the core of AI and the fundamental way to make computers intelligent; its applications span all areas of AI. ML and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0102] As described above, the goal of drug discovery is to find candidate molecules with desired chemical properties. Recently, deep neural networks have achieved breakthroughs in a series of tasks, such as molecular property prediction and structure-based drug design (SBDD). However, in real-world scenarios, training deep neural networks often faces problems such as insufficient data and data noise. This application proposes a model training method based on federated learning, which can be used to solve the above two problems. Federated learning (FL), also known as federated machine learning, joint learning, or consortium learning, defines a machine learning framework within which virtual models are designed to address the problem of collaboration among different data owners without exchanging data. The virtual model is the optimal model formed by aggregating data from all parties, and each region provides services to its local target based on the model.

[0103] Federated learning is categorized into horizontal federated learning, vertical federated learning, and federated transfer learning, depending on the dataset. This application uses horizontal federated learning as an example for introduction; however, this should not be construed as a limitation of this application. For easier understanding, please refer to [link to relevant documentation]. Figure 1 , Figure 1 This diagram illustrates a data relationship in lateral federated learning. Lateral federated learning is typically used when two datasets have significant feature overlap but relatively little user overlap. The essence of lateral federated learning is the joint learning of samples, that is, jointly learning from the molecular samples possessed by different institutions (e.g., institution A and institution B).

[0104] This application employs a federated learning framework, which allows different institutions to collaboratively train a global model for downstream tasks (e.g., molecular property prediction) without the constraint of sharing local data with other institutions. For easier understanding, please refer to [link to relevant documentation]. Figure 2 , Figure 2This is a schematic diagram of the architecture of a federated learning system in an embodiment of this application. As shown in the figure, the federated learning system includes local devices and a central device. The central device can be a server or a terminal device, and the local device can be either a server or a terminal device. The server involved in this application can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. The terminal device can be a smartphone, tablet, laptop, PDA, personal computer, smart TV, smartwatch, in-vehicle device, wearable device, etc., but is not limited to these. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, which is not limited here. The number of servers and terminal devices is also not limited. The solution provided in this application can be completed independently by the terminal device, independently by the server, or jointly by the terminal device and the server; this application does not specifically limit this.

[0105] by Figure 2 Taking the illustrated architecture as an example, in one iteration, local devices A, B, and C download the latest global model from the central device and use it as their local model. Each local device trains its local model using local data and obtains corresponding local gradient values. Each local device then encrypts its local gradient values ​​and uploads them to the central device. The central device aggregates the local gradient values ​​from all local devices and uses them to obtain a global gradient value, which is then used to update the model parameters of the global model. Based on this, the central device sends the updated model parameters to local devices A, B, and C, and each local device updates its local model.

[0106] Throughout the process, the training of the algorithm and the use of data were localized, which protected the user's data privacy and greatly accelerated the training process.

[0107] Based on the above description, the solutions provided in this application involve AI technologies such as machine learning (ML). The following section will describe the model training method in this application from the perspective of a local device. Please refer to [link to relevant documentation]. Figure 3 One embodiment of the model training method in this application includes:

[0108] 110. Obtain K molecular sample features and K molecular annotation features, where the molecular sample features and molecular annotation features have a corresponding relationship, and K is an integer greater than 1;

[0109] In one or more embodiments, a local device acquires K molecular samples, each of which has a set of pre-labeled annotations, typically determined during the experiment. Based on this, for each molecular sample, its molecular sample features and molecular annotation features are extracted.

[0110] Specifically, molecular samples are molecules used for model training. A molecule is a whole composed of atoms bonded together in a specific bonding order and spatial arrangement; this bonding order and spatial arrangement is called molecular structure. Due to the interactions between atoms within a molecule, the physical and chemical properties of a molecule depend not only on the types and numbers of its constituent atoms but also, and perhaps more importantly, on its structure. Based on this, for example, in one implementation, molecular motif features corresponding to the molecular samples are extracted, and after passing through a fully connected layer, the molecular sample features can be output.

[0111] It should be noted that the local device can be a server or a terminal device (e.g., a client), without limitation here.

[0112] 120. Determine K characteristic mixing cards based on the beta distribution, where the sum of the K characteristic mixing cards is 1;

[0113] In one or more embodiments, the local device can sample K feature mixing bases that satisfy a predetermined beta (β) distribution, the sum of which is 1.

[0114] Specifically, suppose a feature has a mixing cardinality of "t", where t ~ Beta(α,α), and α is a mixing hyperparameter used to control the degree of mixing, α∈(0,+∞). When α approaches 0, the distribution tends to concentrate at both ends of (0,1), while a larger α can easily lead to underfitting. Based on this, the value of α is usually greater than or equal to 0.1 and less than or equal to 0.4.

[0115] 130. Based on the K feature mixing cardinality, K molecular sample features, and K molecular annotation features, generate the target molecular sample features and target molecular annotation features of the mixed molecular sample;

[0116] In one or more embodiments, K feature mixing cardinality numbers are used to linearly combine K molecular sample features to construct the target molecular sample features of the mixed molecular sample. Similarly, K feature mixing cardinality numbers are used to linearly combine K molecular annotation features to construct the target molecular annotation features of the mixed molecular sample.

[0117] In actual training, a large number of mixed molecular samples are often required. This application takes the construction of a mixed molecular sample as an example for introduction. It is understood that the construction methods of other mixed molecular samples are similar, and will not be elaborated here.

[0118] 140. Based on the target molecule sample features and target molecule annotation features, obtain the first local gradient value for the local model. When the model training conditions are met, obtain the second local gradient value.

[0119] In one or more embodiments, during a training iteration, the target molecular sample features of the mixed molecular samples are used as input to the local model, and the predicted value is output by the local model. Thus, using the target molecular labeled features as the ground truth, the first local gradient value for the local model can be calculated based on the loss function.

[0120] Specifically, in one scenario, an iteration threshold (e.g., 20 iterations) can be preset for each local device. When the number of iterations reaches the preset threshold, the model training condition is considered met. In another scenario, the central device can set different iteration thresholds for different local devices and distribute them to the local devices. When the number of iterations on a local device reaches the preset threshold, the model training condition is considered met. Based on this, when the model training condition is met, the second local gradient value for the local device can be obtained. The second local gradient value represents the result of multiple gradient accumulations for the local model.

[0121] It is understood that model training conditions are not limited to whether the number of iterations reaches the iteration threshold. For example, the model training conditions are determined to be met when the model converges. It should be noted that in actual training, the first local gradient value is usually generated based on a batch of mixed molecular samples. This application uses any one mixed molecular sample in a batch as an example for illustration; however, this should not be construed as a limitation of this application.

[0122] 150. Send the second local gradient value to the central device.

[0123] In one or more embodiments, a local device sends second local gradient values ​​to a central device. After receiving second local gradient values ​​from at least one local device, the central device can aggregate these second local gradient values ​​(e.g., by weighted averaging) to obtain a global gradient value, and then use the global gradient value to update the model parameters of the global model. Thus, the central device can obtain the global model without accessing the original training data.

[0124] This application provides a model training method based on federated learning. While the training data from various institutions may contain noise, some training data is actually noise-free. Therefore, this application introduces feature fusion technology. By mixing noise-free and noisy training data, the influence of noise can be eliminated, thereby improving the robustness of the model. Simultaneously, on the one hand, data augmentation is achieved by constructing new training data; on the other hand, under the federated learning framework, multiple institutions can jointly train a global model, thus addressing the problem of insufficient training data in different environments to some extent, which is beneficial for improving the accuracy of model predictions.

[0125] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, obtaining K molecular sample features and K molecular annotation features may specifically include:

[0126] Obtain K molecular samples, where each molecular sample corresponds to at least one annotation result;

[0127] For each molecular sample, feature extraction is performed to obtain the molecular sample features corresponding to the molecular sample;

[0128] For each molecular sample, based on at least one annotation result corresponding to the molecular sample, a molecular annotation feature corresponding to the molecular sample is generated.

[0129] In one or more embodiments, a method for obtaining molecular sample features and molecular annotation features is described. As can be seen from the foregoing embodiments, each molecular sample has a set of annotation results, which can be understood to include, but are not limited to, water solubility, toxicity, hydrophobicity, and stability.

[0130] Specifically, a molecular sample can be represented by a molecular formula. For example, a water molecule sample can be represented as "H2O," meaning that one water molecule can be separated into two hydrogen atoms (H) and one oxygen atom (O) using electrolysis or other methods. Molecular fingerprinting technology can be used to partially decompose or amplify the molecular sample (e.g., enzymatic digestion and polymerase chain reaction amplification of proteins), followed by separation by chromatography and electrophoresis to obtain the molecular fingerprint. The molecular fingerprint is an abstract representation of a molecule. Based on this, the molecular fingerprint is used as input to a feature extraction network (e.g., several fully connected layers), and the molecular sample features are obtained through the feature extraction network.

[0131] For at least one annotation result corresponding to a molecular sample, molecular annotation features can be generated using one-hot encoding. Assuming that molecular sample X has annotation results including "water solubility" and "toxicity," but not "stability," the molecular annotation feature of molecular sample X can be represented as (1,1,0). Here, the first element of the molecular annotation feature indicates whether it has "water solubility"; if it does, the first element is "1," otherwise it is "0." The second element of the molecular annotation feature indicates whether it has "toxicity"; if it does, the second element is "1," otherwise it is "0." The third element of the molecular annotation feature indicates whether it has "stability"; if it does, the third element is "1," otherwise it is "0."

[0132] It should be noted that in practical applications, the annotation result can also be represented as the actual value under a preset label. For example, if the preset label is "boiling point" and the actual value of the boiling point is 95 degrees Celsius, then the annotation result is 95. This application may use other methods to encode the annotation result; this is merely an illustration and should not be construed as a limitation of this application.

[0133] Secondly, in this embodiment of the application, a method for obtaining molecular sample features and molecular annotation features is provided. Through the above method, the features and annotation results of each molecular sample can be encoded separately, which can be used for subsequent model training. Thus, a feasible processing solution is provided for the implementation of the scheme.

[0134] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, feature extraction is performed on the molecular sample to obtain the molecular sample features corresponding to the molecular sample, which may specifically include:

[0135] A molecular graph sample is generated from a molecular sample. The molecular graph sample includes vertices and edges, where vertices represent atoms and edges represent chemical bonds.

[0136] Based on molecular graph samples, molecular sample features are obtained through graph neural networks.

[0137] In one or more embodiments, a method for extracting molecular sample features using a graph neural network is described. As can be seen from the foregoing embodiments, a molecular sample can be represented as a chemical formula. Furthermore, a molecular sample can be converted into a molecular graph sample, that is, the molecular sample is stored in the form of a graph, where vertices in the molecular graph sample represent atoms and edges represent chemical bonds.

[0138] For specific details, please refer to [link / reference]. Figure 4 , Figure 4This is a schematic diagram illustrating molecular encoding based on a graph neural network in an embodiment of this application. As shown in the figure, molecules are treated as graph data, and molecular property prediction tasks can be accomplished by combining deep learning methods. Given a molecular graph sample, corresponding embedding features can be generated for each vertex. Optionally, corresponding embedding features can also be generated for edges of different types (ionic bonds, covalent bonds, or metallic bonds). Then, these embedding features are used as input to a graph neural network (GNN), and the GNN outputs the corresponding graph representation, i.e., the molecular sample features are obtained.

[0139] Furthermore, this application provides a method for extracting molecular sample features using a graph neural network. By converting the molecule into a molecular graph, the relationships between atoms can be better reflected. Therefore, the graph neural network can learn reasonable feature representations from the molecular graph, thus facilitating the extraction of molecular sample features with better characterization. In addition, GNN is a neural network structure that generates vector representations for nodes. Because the forward propagation of a neural network is extremely short, its computational time is less than that of traditional quantum chemical calculation methods.

[0140] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, obtaining K molecular samples may specifically include:

[0141] Send a data access request to the target local device;

[0142] If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1.

[0143] Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1;

[0144] P molecular samples and Q molecular samples are used as K molecular samples.

[0145] In one or more embodiments, a method for sharing samples between different local devices is described. As can be seen from the foregoing embodiments, at least two molecular samples need to be linearly combined, wherein the at least two molecular samples may originate from the same user (i.e., local device) or from different users (i.e., local devices).

[0146] For specific details, please refer to [link / reference]. Figure 5 , Figure 5This is a schematic diagram illustrating an architecture for sharing samples among multiple local devices in an embodiment of this application. Taking local device B as an example, local device A belongs to institution A, and local device B belongs to institution B. Assuming that institution A and institution B agree to share molecular samples, local device A can send a data request to local device B, which carries verification information. Local device B parses the data request and authenticates the verification information. If the authentication passes, local device A is allowed to obtain P molecular samples from local device B.

[0147] In addition, local device A can retrieve Q molecular samples from the local database. Thus, P molecular samples and Q molecular samples are combined as K molecular samples, i.e., P + Q = K.

[0148] It should be noted that organization A and organization B may share some or all of their molecular samples; this is not limited here. Furthermore, there are typically no identical molecular samples among the P molecular samples and the Q molecular samples.

[0149] Furthermore, in this embodiment of the application, a method for sharing samples between different local devices is provided. Through the above method, feature fusion technology can not only perform calculations based on the internal data of the same user (i.e., local device), but also perform corresponding fusion calculations by introducing features of other users (i.e., local devices) into the fusion features, thereby further improving model performance.

[0150] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, determining the K feature mixing cardinality according to the beta distribution may specifically include:

[0151] Obtain random numbers that satisfy a beta distribution and use these random numbers as the first feature mixing base.

[0152] The second feature mixing basis is determined based on the first feature mixing basis.

[0153] In one or more embodiments, a method for obtaining the feature mixing cardinality of two features is described. As can be seen from the foregoing embodiments, a local device can acquire at least two molecular samples. Before performing a linear combination of the at least two molecular samples, it is necessary to determine the feature mixing cardinality assigned to each molecular sample. The following will describe the method for obtaining the feature mixing cardinality of two features as an example.

[0154] Specifically, based on a predetermined β distribution, the local device can randomly sample a feature mixing cardinality that satisfies the β distribution, thus obtaining the first feature mixing cardinality. The first feature mixing cardinality is denoted as "t". Since the sum of the K feature mixing cardins is 1, the second feature mixing cardinality is denoted as "t-1".

[0155] Secondly, in this embodiment of the application, a method for obtaining the mixing cardinality of two features is provided. By using the above method, only one feature mixing cardinality needs to be randomly determined to achieve the combination of two molecular samples. That is, the computational resources required for random sampling are saved to a certain extent, which helps to improve processing efficiency.

[0156] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by the present application, the K molecular sample features include first molecular sample features and second molecular sample features, and the K molecular annotation features include first molecular annotation features and second molecular annotation features;

[0157] Based on the K feature mixing cardinality, K molecular sample features, and K molecular annotation features, the target molecular sample features and target molecular annotation features of the mixed molecular sample are generated, which may specifically include:

[0158] The first feature vector is obtained by weighting the features of the first molecular sample using the first feature mixing basis.

[0159] The second feature vector is obtained by weighting the features of the second molecular sample using the second feature mixing basis.

[0160] Based on the first feature vector and the second feature vector, generate the target molecular sample features of the mixed molecular sample;

[0161] The first feature vector is obtained by weighting the first feature mixing cardinality.

[0162] The second feature vector is obtained by weighting the second feature mixing basis of the second molecule annotation features;

[0163] Based on the third and fourth feature vectors, target molecular annotation features of the mixed molecular samples are generated.

[0164] In one or more embodiments, a method for linearly combining two molecular samples is described. As can be seen from the foregoing embodiments, a local device can linearly combine at least two molecular samples. The following description will take the linear combination of two molecular samples as an example.

[0165] Specifically, assuming a linear combination of the first molecular sample and the second molecular sample, where the first molecular sample undergoes feature extraction to obtain first molecular sample features, and the second molecular sample undergoes feature extraction to obtain second molecular sample features. First molecular labeled features can be generated based on the annotation results corresponding to the first molecular sample, and second molecular labeled features can be generated based on the annotation results corresponding to the second molecular sample. Based on this, the target molecular sample features of the mixed molecular samples are calculated using the following formula:

[0166] X = tx i +(t-1)x j Equation (1)

[0167] Where X represents the target molecule sample feature, t represents the mixing cardinality of the first feature, and x i Represents the characteristics of the first molecular sample, tx i Let x represent the first eigenvector. (t-1) represents the second eigenvector mixing cardinality. j Represents the characteristics of the second molecule sample, (t-1)x j This represents the second eigenvector.

[0168] For example, suppose the first feature mixing cardinality is 0.2, the second feature mixing cardinality is 0.8, the first molecular sample features are (0.2, 0.8, 0.9), and the second molecular sample features are (0.1, 0.1, 0.7). Weighting the first molecular sample features using the first feature mixing cardinality yields a first feature vector represented as (0.04, 0.16, 0.18). Similarly, weighting the second molecular sample features using the second feature mixing cardinality yields a second feature vector represented as (0.08, 0.08, 0.56). In one implementation, the first and second feature vectors are summed (i.e., corresponding elements are added) to obtain the target molecular sample features represented as (0.12, 0.24, 0.74).

[0169] Based on this, the target molecule annotation features of the mixed molecule samples are calculated using the following formula:

[0170] Y = ty i +(t-1)y j Equation (2)

[0171] Where Y represents the target molecule annotation feature, t represents the first feature mixing cardinality, and y i Indicates the first molecule annotation features, ty i Let y represent the third eigenvector. (t-1) represents the mixing cardinality of the second eigenvector. j Indicates the second molecule's annotation feature, (t-1)y j This represents the fourth eigenvector.

[0172] For example, suppose the first feature mixing cardinality is 0.2, the second feature mixing cardinality is 0.8, the first molecule annotation feature is (1,1,0), and the second molecule annotation feature is (1,0,0). The first molecule annotation feature is weighted using the first feature mixing cardinality to obtain a third feature vector represented as (0.2,0.2,0). Similarly, the second molecule annotation feature is weighted using the second feature mixing cardinality to obtain a fourth feature vector represented as (0.8,0,0). In one implementation, the third and fourth feature vectors are summed (i.e., corresponding elements are added) to obtain the target molecule annotation feature represented as (1,0.2,0).

[0173] It is understandable that a mixed molecular sample can be understood as a new molecular sample obtained by linearly combining K molecular samples.

[0174] It should be noted that molecular sample features can be represented as 128-dimensional vectors, or vectors with other dimensions; this is not limited here. Molecular annotation features can be represented as 20-dimensional vectors, or vectors with other dimensions; this is not limited here.

[0175] Furthermore, this application provides a method for linearly combining two molecular samples. This method requires fewer molecular samples, only two. Therefore, it reduces the difficulty of combining molecular samples, thereby saving computational resources.

[0176] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, determining the K feature mixing cardinality according to the beta distribution may specifically include:

[0177] Obtain at least three random numbers that satisfy a beta distribution;

[0178] Based on the sum of at least three random numbers, normalize each random number to obtain the feature mixing cardinality corresponding to each random number.

[0179] In one or more embodiments, a method for obtaining at least three feature mixing cards is described. As can be seen from the foregoing embodiments, a local device can obtain at least two molecular samples. Before performing a linear combination of the at least two molecular samples, it is necessary to determine the feature mixing cardinality assigned to each molecular sample. The following will describe obtaining at least three feature mixing cards as an example.

[0180] Specifically, the local device, based on a predetermined β distribution, can randomly sample at least three random numbers that satisfy the β distribution, i.e., obtain (R1, R2, ..., RK), where K is an integer greater than or equal to 3. Since the sum of R1 + R2 + ... + RK is not necessarily equal to 1, each random number can be normalized as follows:

[0181] tn=Rn / (R1+R2+…+RK); Equation (3)

[0182] Where tn represents the nth feature mixing base, R1 represents the first random number, R2 represents the second random number, RK represents the Kth random number, and Rn represents the nth random number.

[0183] For example, assuming K is 3, after sampling, the random numbers R1 are 0.3, R2 are 0.5, and R3 are 0.8. Combining equation (3), the following calculations are performed:

[0184] t1=0.3 / (0.3+0.5+0.8)=0.1875;

[0185] t2=0.5 / (0.3+0.5+0.8)=0.3125;

[0186] t3=0.8 / (0.3+0.5+0.8)=0.5;

[0187] Where t1 represents the feature mixing cardinality corresponding to random number R1, t2 represents the feature mixing cardinality corresponding to random number R2, and t3 represents the feature mixing cardinality corresponding to random number R3.

[0188] It is understandable that when K is greater than 3, the method for calculating the feature mixing cardinality corresponding to the random number is similar, and will not be elaborated here.

[0189] Secondly, in this embodiment of the application, a method for obtaining at least three feature mixing bases is provided. Through the above method, K feature mixing bases that satisfy the β distribution can be randomly determined. However, considering that the sum of the K randomly obtained feature mixing bases may not be equal to 1, a normalization method is used to further process these feature mixing bases, thereby providing a reasonable and feasible solution for the implementation of the scheme.

[0190] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by the present application, the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features;

[0191] Based on the K feature mixing cardinality, K molecular sample features, and K molecular annotation features, the target molecular sample features and target molecular annotation features of the mixed molecular sample are generated, which may specifically include:

[0192] Using the feature mixing base corresponding to each random number, the features of at least three molecular samples are weighted and summed to obtain the target molecular sample features of the mixed molecular sample;

[0193] By using the feature mixing cardinality corresponding to each random number, at least three molecular annotation features are weighted and summed to obtain the target molecular annotation features of the mixed molecular sample.

[0194] In one or more embodiments, a method for linearly combining at least three molecular samples is described. As can be seen from the foregoing embodiments, a local device can linearly combine at least three molecular samples. The following description will take the linear combination of three molecular samples as an example. The method for linearly combining more than three molecular samples is similar and will not be elaborated here.

[0195] Specifically, assume a linear combination of the first molecular sample, the second molecular sample, and the third molecular sample. The first molecular sample undergoes feature extraction to obtain its features, the second molecular sample undergoes feature extraction to obtain its features, and the third molecular sample undergoes feature extraction to obtain its features. Based on the annotation results corresponding to the first molecular sample, the first molecular annotation features can be generated; based on the annotation results corresponding to the second molecular sample, the second molecular annotation features can be generated; and based on the annotation results corresponding to the third molecular sample, the third molecular annotation features can be generated.

[0196] Based on this, the target molecular sample characteristics of the mixed molecular samples are calculated using the following formula:

[0197] X=t1x1+t2x2+t3x3, stt1+t2+t3=1; Formula (4)

[0198] Where X represents the target molecular sample feature, t1 represents the mixing cardinality of the first feature, and x1 represents the first molecular sample feature. t2 represents the mixing cardinality of the second feature, and x2 represents the second molecular sample feature. t3 represents the mixing cardinality of the third feature, and x3 represents the third molecular sample feature.

[0199] For example, suppose the first feature mixing cardinality is 0.2, the second feature mixing cardinality is 0.5, and the third feature mixing cardinality is 0.3. The first molecular sample features are (0.2, 0.8, 0.9), the second molecular sample features are (0.1, 0.1, 0.7), and the third molecular sample features are (0.8, 0.4, 0.6). The three molecular sample features are weighted and summed using the three feature mixing cardins, i.e., by adding corresponding elements, resulting in the target molecular sample features represented as (0.2, 0.8, 0.9).

[0200] Based on this, the target molecule annotation features of the mixed molecule samples are calculated using the following formula:

[0201] Y=t1y1+t2y2+t3y3, stt1+t2+t3=1; Formula (5)

[0202] Where Y represents the molecular annotation feature, t1 represents the first feature mixing cardinality, y1 represents the first molecular annotation feature, t2 represents the second feature mixing cardinality, y2 represents the second molecular annotation feature, t3 represents the third feature mixing cardinality, and y3 represents the third molecular annotation feature.

[0203] For example, suppose the first feature mixing cardinality is 0.2, the second feature mixing cardinality is 0.5, and the third feature mixing cardinality is 0.3. The first molecule label feature is (1,1,0), the second molecule label feature is (1,0,0), and the third molecule label feature is (1,1,1). The three molecule label features are weighted and summed using the three feature mixing cardins; that is, corresponding elements are added together. The resulting target molecule sample feature representation is (1,0.5,0.3).

[0204] It is understandable that a mixed molecular sample can be understood as a new molecular sample obtained by linearly combining K molecular samples.

[0205] It should be noted that molecular sample features can be represented as 128-dimensional vectors, or vectors with other dimensions; this is not limited here. Molecular annotation features can be represented as 20-dimensional vectors, or vectors with other dimensions; this is not limited here.

[0206] Furthermore, in this embodiment of the application, a method for linearly combining at least three molecular samples is provided. Through the above method, the number of molecular samples used for linear combination is relatively large. Therefore, it can better eliminate the noise present in the molecular samples to a certain extent. Using mixed molecular samples with less data noise in the local training process can improve the performance of robust optimization of the model.

[0207] Optionally, in the above Figure 3Based on the corresponding embodiments, in another optional embodiment provided by this application, before obtaining the first local gradient value for the local model according to the target molecule sample features and the target molecule annotation features, the following may be included:

[0208] In response to the model update command sent by the central device, the model parameters of the global model are obtained from the central device and used as the model parameters of the local model.

[0209] In one or more embodiments, a method is described for a central device to issue model update tasks to local devices. As can be seen from the foregoing embodiments, the central device selects one or more local devices and sends model update instructions to these local devices. The selected local devices then respond to the model update instructions sent by the central device and download the latest global model from the central device, that is, obtain the model parameters of the latest global model.

[0210] Specifically, taking any selected local device as an example, the local device uses the model parameters of the latest global model as the model parameters of its local model. Therefore, the local device calculates the loss function for the mixed molecular samples and performs backpropagation on the loss function to update the model parameters of its local model. After the local iteration is complete, the model training conditions are met, and the local device uploads the second local gradient value corresponding to the latest global model to the central device.

[0211] In practical applications, in order to improve the reliability of information transmission between local devices and central devices, symmetric encryption technology or asymmetric encryption technology can be used to encrypt model parameters and local gradient values, thereby improving the privacy of communication between devices.

[0212] Secondly, this application provides a method for a central device to distribute model update tasks to local devices. With the aforementioned method, due to considerations of data security and privacy, data from different institutions is often difficult to contribute directly, and each institution typically possesses a small amount of data, thus often resulting in data shortages. In this application, the central device distributes model update tasks to each local device, enabling each local device to train the model without sharing samples. This not only protects data privacy but also increases the sample size, which is beneficial for achieving better model fitting results.

[0213] Optionally, in the above Figure 3 Based on the corresponding embodiments, in another optional embodiment provided by this application, the first local gradient value for the local model is obtained according to the target molecule sample features and the target molecule annotation features, which may specifically include:

[0214] Based on the characteristics of the target molecular sample, the molecular attribute prediction scores of the mixed molecular sample are obtained through the local model. The molecular attribute prediction scores represent the predicted scores for the target molecular attributes.

[0215] Based on the predicted scores of molecular attributes and the labeled features of the target molecules, the first local gradient value for the local model is determined by the first loss function, where the labeled features of the target molecules represent the actual scores for the target molecular attributes.

[0216] In one or more embodiments, a model training method for implementing regression tasks is described. As can be seen from the foregoing embodiments, regression tasks are typically used to predict a value, and regression is an approximate prediction of the true value.

[0217] For specific details, please refer to [link / reference]. Figure 6 , Figure 6 This is a schematic diagram of a training framework based on a regression task in an embodiment of this application. As shown in the figure, assume the target molecule label feature of the mixed molecular sample is "96.5", where the target molecule label feature represents the actual score corresponding to the boiling point. Based on this, the target molecule sample feature of the mixed molecular sample is used as the input of the local model, and the local model outputs the predicted score of molecular attributes, for example, "90.5". Then, the first local gradient value can be calculated using the first loss function.

[0218] It should be noted that the first loss function used in this application may include the mean-square error (MSE) loss function, or the mean absolute error (MAE) loss function, or the root mean square error (RMSE) loss function, or other types of loss functions, which are not limited here.

[0219] Optionally, if the local model also includes a GNN, then the first loss function can be expressed as L(B; θ; w), where B represents the batch size of the local iterative sampling, θ represents the value used to calculate the graph representation, and w represents the value used to calculate the prediction result.

[0220] Secondly, in this embodiment of the application, a model training method for implementing regression tasks is provided. In this way, a local model for regression tasks is trained using the mixed molecular samples obtained by linear combination. That is, supervised learning can be supported based on the target molecule annotation features, thereby improving the flexibility of model training.

[0221] Optionally, in the above Figure 3Based on the corresponding embodiments, in another optional embodiment provided by this application, the first local gradient value for the local model is obtained according to the target molecule sample features and the target molecule annotation features, which may specifically include:

[0222] Based on the characteristics of the target molecular samples, the molecular attribute prediction distribution of the mixed molecular samples is obtained through a local model, where the molecular attribute prediction distribution represents the probability of predicting that the sample belongs to each molecular attribute.

[0223] Based on the molecular attribute prediction distribution and the target molecule annotation features, the first local gradient value for the local model is determined by the second loss function, where the target molecule annotation features represent the probability that the annotation belongs to each molecular attribute.

[0224] In one or more embodiments, a model training method for implementing a classification task is introduced. As can be seen from the foregoing embodiments, a classification task typically outputs a category; given one or more inputs, the classification task will predict one or more results. In classification tasks, the local model can employ decision trees, random forests, gradient boosting trees, multilayer perceptrons, or Naive Bayes, etc., and is not limited here.

[0225] For specific details, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram of a training framework based on a classification task in this application embodiment. As shown in the figure, assume that the mixed molecular sample has one molecular attribute of "water solubility" and another molecular attribute of "toxicity", but no molecular attribute of "stability". Therefore, its corresponding target molecular annotation feature can be represented as (1,1,0). Based on this, the target molecular sample feature of the mixed molecular sample is used as the input of the local model, and the local model outputs the molecular attribute prediction distribution, i.e., (0.7,0.2,0.1), where "0.7" represents the probability of belonging to "toxicity", "0.2" represents the probability of belonging to "water solubility", and "0.1" represents the probability of belonging to "stability". Thus, the first local gradient value can be calculated using the second loss function.

[0226] It should be noted that the second loss function used in this application may be the cross entropy (CE) loss function, or the hinge loss function, or other types of loss functions, which are not limited here.

[0227] Optionally, if the local model also includes a GNN, then the second loss function can be expressed as L(B; θ; w), where B represents the batch size of the local iterative sampling, θ represents the value used to calculate the graph representation, and w represents the value used to calculate the prediction result.

[0228] Secondly, in this embodiment of the application, a model training method for implementing classification tasks is provided. In this way, a local model for classification tasks is trained using the mixed molecular samples obtained by linear combination. That is, supervised learning can be supported based on the target molecule annotation features, thereby improving the flexibility of model training.

[0229] Based on the above introduction, the model training method in this application will be described from the perspective of the central device. Please refer to [link / reference]. Figure 8 Another embodiment of the model training method in this application includes:

[0230] 210. Send the model parameters of the global model to M local devices so that each local device uses the model parameters of the global model as the model parameters of its local model, and obtains the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features. When the model training conditions are met, the second local gradient value is obtained, where M is an integer greater than or equal to 1.

[0231] In one or more embodiments, the central device sends the model parameters of the global model to M local devices, and each local device uses the model parameters of the global model as the model parameters of its local model to obtain the second local gradient value.

[0232] For ease of understanding, we will use any one of the M local devices as an example. Specifically, the local device acquires K molecular samples, each with a set of pre-labeled annotations. For each molecular sample, we extract its molecular sample features and molecular annotation features. Furthermore, based on a pre-determined β distribution, the local device samples K feature mixing cards that satisfy this β distribution, and the sum of these K feature mixing cards is 1. The local device uses the K feature mixing cards to linearly combine the K molecular sample features to construct the target molecular sample features of the mixed molecular sample. Similarly, the local device uses the K feature mixing cards to linearly combine the K molecular annotation features to construct the target molecular annotation features of the mixed molecular sample.

[0233] In one training iteration, the local device uses the target molecular sample features from the mixed molecular samples as input to the local model, and outputs predicted values ​​through the local model. Thus, using the target molecular labeled features as the ground truth, the first local gradient value for the local model can be calculated based on the loss function. When the model training conditions are met, the second local gradient value for the local device can be obtained. The second local gradient value represents the result of multiple gradient accumulations for the local model.

[0234] It should be noted that the central device can be a server or a terminal device; there is no limitation here.

[0235] 220. Receive the second local gradient value sent by each of the M local devices;

[0236] In one or more embodiments, the central device may receive a second local gradient value sent by each of the M local devices. Typically, the different local devices respond with the second local gradient value at different times; therefore, the timing of the central device acquiring the second local gradient value is not limited here.

[0237] 230. Update the model parameters of the global model based on the second local gradient values ​​sent by each of the M local devices.

[0238] In one or more embodiments, the central device may aggregate these second local gradient values ​​(e.g., by weighted averaging) to obtain a global gradient value, and use the global gradient value to update the model parameters of the global model.

[0239] This application provides a model training method based on federated learning. While the training data from various institutions may contain noise, some training data is actually noise-free. Therefore, this application introduces feature fusion technology. By mixing noise-free and noisy training data, the influence of noise can be eliminated, thereby improving the robustness of the model. Simultaneously, on the one hand, data augmentation is achieved by constructing new training data; on the other hand, under the federated learning framework, multiple institutions can jointly train a global model, thus addressing the problem of insufficient training data in different environments to some extent, which is beneficial for improving the accuracy of model predictions.

[0240] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another optional embodiment provided by this application, before sending the model parameters of the global model to the M local devices, the following may be included:

[0241] Get the first weight of each of the T local devices, where T is an integer greater than or equal to M, and the sum of the first weights of the T local devices is 1;

[0242] Based on the first weight corresponding to each of the T local devices, M local devices are sampled from the T local devices using a probability density function.

[0243] In one or more embodiments, a method based on a probability density function using M local devices is introduced. As can be seen from the foregoing embodiments, the central device can initialize a weight (λi) for each local device, ∑λi=1, where i in λi is an integer greater than or equal to 1 and less than or equal to T.

[0244] Specifically, at the start of each iteration, the central device uses the weights (e.g., the first weight) of each of the current T local devices as the probability density function, samples M local devices, and passes the model parameters of the current global model to these M local devices. The probability density function of a continuous random variable is a function describing the likelihood of the output value of that random variable near a certain point. The probability that the value of the random variable falls within a certain region is the integral of the probability density function over that region.

[0245] Secondly, this application embodiment provides a method based on a probability density function using M local devices. This method takes into account that the molecular samples possessed by different users (i.e., local devices) will follow a specific probability distribution; that is, different users (i.e., local devices) often have different data distributions. Therefore, incorporating robust optimization techniques enables the model to achieve better performance on different data distributions, thus adapting to different local data distributions and facilitating better average model performance. Furthermore, noise reduction also increases the stability of out-of-distribution robust optimization, thereby enabling the trained model to achieve better performance.

[0246] Optionally, in the above Figure 8 Based on the corresponding embodiments, another optional embodiment provided in this application may further include:

[0247] Receive the loss value sent by each of the M local devices;

[0248] Based on the loss value sent by each of the M local devices, the first weight corresponding to each of the M local devices is updated to obtain the second weight corresponding to each of the M local devices.

[0249] In one or more embodiments, a method for updating the weights corresponding to local devices is described. As can be seen from the foregoing embodiments, the M local devices selected by the central device, when the model training conditions are met, can not only feed back the second local gradient value to the central device, but also feed back the loss value.

[0250] Specifically, the central device updates the first weight corresponding to each local device based on the loss values ​​returned by the M local devices. Taking the update of the first weight of the i-th local device as an example, the updated second weight can be calculated as follows:

[0251] λ i更新 =λ i历史 +γL i Equation (6)

[0252] Where, λ i更新 λ represents the second weight of the i-th local device.i历史 This represents the first weight of the i-th local device. γ represents a preset parameter, such as 0.02 or 0.1. L i This represents the loss value returned by the i-th local device.

[0253] Secondly, this application provides a method for updating the weights corresponding to local devices. Through this method, the central device dynamically updates the weights corresponding to local devices based on the loss values ​​fed back by the local devices. This is beneficial for better adapting to different data distributions and achieving better model performance.

[0254] Optionally, in the above Figure 8 Based on the corresponding embodiments, in another optional embodiment provided by this application, the first weight corresponding to each of the M local devices is updated according to the loss value sent by each of the M local devices to obtain the second weight corresponding to each of the M local devices. Specifically, this may include:

[0255] Based on the loss value sent by each of the M local devices, the first weight corresponding to each of the M local devices is updated to obtain the weight to be processed corresponding to each of the M local devices;

[0256] Based on the sum of the unprocessed weights corresponding to each of the M local devices, normalize each unprocessed weight to obtain the normalized weights corresponding to each of the M local devices.

[0257] Based on the sum of the remaining weights and the normalized weights corresponding to each of the M local devices, determine the second weight corresponding to each of the M local devices.

[0258] In one or more embodiments, a method for normalizing updated weights is described. As can be seen from the foregoing embodiments, the sum of the weights of T local devices is 1, but after weight adjustment, the weights of the M local devices may not be 1 for the T local devices. Therefore, it is necessary to adjust the weights of the M local devices.

[0259] Specifically, assuming T is 10, an average distribution method can be used when initially allocating weights. Assuming the initially allocated weight is the first weight, the first weight for each local device is 0.1. Assuming M is 3, meaning 3 local devices are selected from 10. When the model training conditions are met, the central device updates the first weight for each local device based on the loss values ​​returned by the M local devices, thus obtaining the second weight for each local device. For example, the unprocessed weight for the first local device out of the M local devices is 0.3, the unprocessed weight for the second local device out of the M local devices is 0.5, and the unprocessed weight for the first local device out of the M local devices is 0.8. Therefore, each unprocessed weight is normalized as follows:

[0260] W i =w i / (w1+w2+…+w T Equation (7)

[0261] Among them, W i This represents the normalized weight of the i-th local device. i Let w1+w2+…+w represent the weight to be processed for the i-th local device. T ) represents the sum of the unprocessed weights of M local devices.

[0262] For example, the processing weight of the first local device is 0.1875, the processing weight of the second local device is 0.3125, and the processing weight of the third local device is 0.5.

[0263] Since the sum of the first weights of the M local devices is 0.3, meaning the sum of the remaining weights is also 0.3, the remaining weights need to be proportionally allocated according to the unprocessed weights of each local device. Based on this, the second weight of the first local device is 0.3 * 0.1875 = 0.05625, the second weight of the second local device is 0.3 * 0.3125 = 0.09375, and the second weight of the third local device is 0.3 * 0.5 = 0.15.

[0264] Understandably, in practical applications, the weights of each local device can be reallocated based on the normalized weights of each of the M local devices and the first weights of each of the (TM) local devices, which will not be elaborated here.

[0265] Furthermore, this application embodiment provides a method for normalizing the updated weights. This method ensures that the sum of the weights corresponding to all local devices is a constant, thus guaranteeing that the quantitative result is independent of the injection volume. In addition, it allows all indicators to be on the same order of magnitude, facilitating a comprehensive comparison of various local devices.

[0266] The model training apparatus in this application is described in detail below. Please refer to [link / reference]. Figure 9 , Figure 9 This is a schematic diagram of one embodiment of the model training device in this application. The model training device 30 includes:

[0267] The acquisition module 310 is used to acquire K molecular sample features and K molecular annotation features, wherein the molecular sample features and molecular annotation features have a corresponding relationship, and K is an integer greater than 1;

[0268] The determination module 320 is used to determine the K feature mixing bases based on the beta distribution, wherein the sum of the K feature mixing bases is 1;

[0269] The generation module 330 is used to generate target molecular sample features and target molecular annotation features of the mixed molecular sample based on K feature mixing cardinality, K molecular sample features and K molecular annotation features;

[0270] The acquisition module 310 is also used to acquire the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features, and to obtain the second local gradient value when the model training conditions are met.

[0271] The sending module 340 is used to send the second local gradient value to the central device.

[0272] This application provides a model training apparatus. While the training data from various institutions may contain noise, some training data is actually noise-free. Therefore, this application introduces feature fusion technology. By mixing noise-free and noisy training data, the influence of noise can be eliminated, thereby improving the robustness of the model. Simultaneously, on the one hand, data augmentation is achieved by constructing new training data; on the other hand, under the federated learning framework, multiple institutions can jointly train a global model, thus addressing the problem of insufficient environmental training data to some extent and improving the accuracy of model predictions.

[0273] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0274] The acquisition module 310 is specifically used to acquire K molecular samples, wherein each molecular sample corresponds to at least one annotation result;

[0275] For each molecular sample, feature extraction is performed to obtain the molecular sample features corresponding to the molecular sample;

[0276] For each molecular sample, based on at least one annotation result corresponding to the molecular sample, a molecular annotation feature corresponding to the molecular sample is generated.

[0277] This application provides a model training device. Using this device, the features and annotation results of each molecular sample can be encoded separately, which can then be used for subsequent model training, thus providing a feasible processing solution for the implementation of the proposed method.

[0278] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0279] The acquisition module 310 is specifically used to generate a molecular graph sample based on the molecular sample. The molecular graph sample includes vertices and edges, where vertices represent atoms and edges represent chemical bonds.

[0280] Based on molecular graph samples, molecular sample features are obtained through graph neural networks.

[0281] This application provides a model training device. Using this device, after converting molecules into molecular graphs, the relationships between atoms can be better reflected. Therefore, graph neural networks can learn reasonable feature representations from the molecular graphs, which is beneficial for extracting molecular sample features with better characterization. Furthermore, GNNs are neural network structures that generate vector representations for nodes. Because the forward propagation of neural networks is extremely short, their computational time is less than that of traditional quantum chemical calculation methods.

[0282] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0283] The acquisition module 310 is specifically used to send a data access request to the target local device;

[0284] If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1.

[0285] Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1;

[0286] P molecular samples and Q molecular samples are used as K molecular samples.

[0287] In this embodiment, a model training apparatus is provided. Using this apparatus, feature fusion technology can not only perform calculations based on the internal data of the same user, but also further improve model performance by introducing features from other users into the fused features for corresponding fusion calculations.

[0288] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0289] The determination module 320 is specifically used to obtain random numbers that satisfy the beta distribution and use the random numbers as the first feature mixing base;

[0290] The second feature mixing basis is determined based on the first feature mixing basis.

[0291] This application provides a model training apparatus. Using this apparatus, only a feature mixing cardinality needs to be randomly determined to combine two molecular samples, thus saving computational resources required for random sampling and improving processing efficiency.

[0292] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application, the K molecular sample features include first molecular sample features and second molecular sample features, and the K molecular annotation features include first molecular annotation features and second molecular annotation features;

[0293] The generation module 330 is specifically used to weight the features of the first molecular sample using the first feature mixing basis to obtain the first feature vector;

[0294] The second feature vector is obtained by weighting the features of the second molecular sample using the second feature mixing basis.

[0295] Based on the first feature vector and the second feature vector, generate the target molecular sample features of the mixed molecular sample;

[0296] The first feature vector is obtained by weighting the first feature mixing cardinality.

[0297] The second feature vector is obtained by weighting the second feature mixing basis of the second molecule annotation features;

[0298] Based on the third and fourth feature vectors, target molecular annotation features of the mixed molecular samples are generated.

[0299] This application provides a model training device. Using this device, the number of molecular samples required for linear combinations is reduced to just two. Therefore, the difficulty of molecular sample combination is reduced, thereby saving computational resources.

[0300] Optionally, in the above Figure 9Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0301] Module 320 is specifically used to obtain at least three random numbers that satisfy a beta distribution;

[0302] Based on the sum of at least three random numbers, normalize each random number to obtain the feature mixing cardinality corresponding to each random number.

[0303] In this application embodiment, a model training device is provided. Using the above device, K feature mixture bases that satisfy the β distribution can be randomly determined. However, considering that the sum of the K randomly obtained feature mixture bases may not be equal to 1, a normalization method is used to further process these feature mixture bases, thereby providing a reasonable and feasible solution for the implementation of the scheme.

[0304] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application, the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features;

[0305] The generation module 330 is specifically used to use the feature mixing base corresponding to each random number to perform a weighted summation of the features of at least three molecular samples to obtain the target molecular sample features of the mixed molecular samples;

[0306] By using the feature mixing cardinality corresponding to each random number, at least three molecular annotation features are weighted and summed to obtain the target molecular annotation features of the mixed molecular sample.

[0307] This application provides a model training apparatus. Using this apparatus, the linear combination utilizes a larger number of molecular samples, thus better eliminating noise present in the molecular samples to a certain extent. Furthermore, using mixed molecular samples with less data noise during local training can improve the model's robust optimization performance.

[0308] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0309] The acquisition module 310 is also used to, before acquiring the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features, respond to the model update instruction sent by the central device, acquire the model parameters of the global model from the central device, and use the model parameters of the global model as the model parameters of the local model.

[0310] This application provides a model training apparatus. Using the aforementioned apparatus, due to considerations of data security and privacy, data from different institutions is often difficult to contribute directly, and each institution typically possesses a small amount of data, thus often resulting in data shortages. In this application, a central device distributes model update tasks to each local device, enabling each local device to train the model without sharing samples. This not only protects data privacy but also increases the sample size, which is beneficial for achieving better model fitting results.

[0311] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0312] The acquisition module 310 is specifically used to acquire the molecular attribute prediction scores of the mixed molecular samples based on the characteristics of the target molecular samples through a local model, wherein the molecular attribute prediction score represents the predicted score for the target molecular attribute.

[0313] Based on the predicted scores of molecular attributes and the labeled features of the target molecules, the first local gradient value for the local model is determined by the first loss function, where the labeled features of the target molecules represent the actual scores for the target molecular attributes.

[0314] In this embodiment, a model training apparatus is provided. Using this apparatus, a local model for a regression task is trained using mixed molecular samples obtained through linear combination. That is, supervised learning can be supported based on the target molecule annotation features, thereby improving the flexibility of model training.

[0315] Optionally, in the above Figure 9 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0316] The acquisition module 310 is specifically used to acquire the molecular attribute prediction distribution of the mixed molecular sample based on the characteristics of the target molecular sample through a local model, wherein the molecular attribute prediction distribution represents the probability of predicting that it belongs to each molecular attribute.

[0317] Based on the molecular attribute prediction distribution and the target molecule annotation features, the first local gradient value for the local model is determined by the second loss function, where the target molecule annotation features represent the probability that the annotation belongs to each molecular attribute.

[0318] In this embodiment, a model training apparatus is provided. Using this apparatus, a local model for a classification task is trained using mixed molecular samples obtained through linear combination. That is, supervised learning can be supported based on the target molecule annotation features, thereby improving the flexibility of model training.

[0319] The model training apparatus in this application is described in detail below. Please refer to [link / reference]. Figure 10 , Figure 10 This is a schematic diagram of another embodiment of the model training device in this application. The model training device 40 includes:

[0320] The sending module 410 is used to send the model parameters of the global model to M local devices, so that each local device uses the model parameters of the global model as the model parameters of its local model, and obtains the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features. When the model training conditions are met, the second local gradient value is obtained, where M is an integer greater than or equal to 1.

[0321] The receiving module 420 is used to receive the second local gradient value sent by each of the M local devices;

[0322] The update module 430 is used to update the model parameters of the global model based on the second local gradient value sent by each of the M local devices;

[0323] Among them, the target molecule sample features and the target molecule annotation features are generated by the local device based on the K feature mixing cardinality, the K molecule sample features and the K molecule annotation features. The K feature mixing cardinality is determined according to the beta distribution. The molecule sample features and the molecule annotation features have a corresponding relationship. The sum of the K feature mixing cardinality is 1, and K is an integer greater than 1.

[0324] This application provides a model training apparatus. While the training data from various institutions may contain noise, some training data is actually noise-free. Therefore, this application introduces feature fusion technology. By mixing noise-free and noisy training data, the influence of noise can be eliminated, thereby improving the robustness of the model. Simultaneously, on the one hand, data augmentation is achieved by constructing new training data; on the other hand, under the federated learning framework, multiple institutions can jointly train a global model, thus addressing the problem of insufficient environmental training data to some extent and improving the accuracy of model predictions.

[0325] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application, the model training device 40 further includes an acquisition module 440 and a sampling module 450;

[0326] The acquisition module 440 is used to acquire the first weight of each of the T local devices before sending the model parameters of the global model to the M local devices, where T is an integer greater than or equal to M, and the sum of the first weights of the T local devices is 1.

[0327] The sampling module 450 is used to sample M local devices from the T local devices based on the first weight corresponding to each local device in the T local devices and through the probability density function.

[0328] This application provides a model training apparatus. Using this apparatus, considering that the molecular samples possessed by different users (i.e., local devices) follow specific probability distributions, meaning that different users (i.e., local devices) often have different data distributions, robust optimization techniques are incorporated to enable the model to achieve better performance on different data distributions. This allows the model to adapt to different local data distributions, facilitating better average model performance. Furthermore, noise reduction increases the stability of out-of-distribution robust optimization, thereby enabling the trained model to achieve better performance.

[0329] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0330] The receiving module 420 is also used to receive the loss value sent by each of the M local devices;

[0331] The update module 430 is also used to update the first weight corresponding to each of the M local devices based on the loss value sent by each of the M local devices, so as to obtain the second weight corresponding to each of the M local devices.

[0332] This application provides a model training apparatus. Using this apparatus, the central device dynamically updates the weights corresponding to the local devices based on the loss values ​​fed back by the local devices. This facilitates better adaptation to different data distributions and achieves better model performance.

[0333] Optionally, in the above Figure 10 Based on the corresponding embodiments, in another embodiment of the model training device 30 provided in this application,

[0334] The update module 430 is specifically used to update the first weight corresponding to each of the M local devices based on the loss value sent by each of the M local devices, so as to obtain the weight to be processed corresponding to each of the M local devices.

[0335] Based on the sum of the unprocessed weights corresponding to each of the M local devices, normalize each unprocessed weight to obtain the normalized weights corresponding to each of the M local devices.

[0336] Based on the sum of the remaining weights and the normalized weights corresponding to each of the M local devices, determine the second weight corresponding to each of the M local devices.

[0337] This application provides a model training apparatus. Using this apparatus, the sum of the weights corresponding to all local devices is a constant, ensuring that the quantitative results are independent of the injection volume. Furthermore, it allows all indicators to be on the same order of magnitude, facilitating a comprehensive comparison of the various local devices.

[0338] This application provides a model training device that can be deployed on a server. Figure 11 This is a schematic diagram of a server structure provided in an embodiment of this application. The server 500 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 522 (e.g., one or more processors) and memory 532, and one or more storage media 530 (e.g., one or more mass storage devices) for storing application programs 542 or data 544. The memory 532 and storage media 530 can be temporary or persistent storage. The program stored in the storage media 530 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the server. Furthermore, the CPU 522 may be configured to communicate with the storage media 530 and execute the series of instruction operations in the storage media 530 on the server 500.

[0339] Server 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input / output interfaces 558, and / or one or more operating systems 541, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0340] The steps performed by the server in the above embodiments can be based on this Figure 11 The server structure shown.

[0341] This application provides a model training device that can be deployed on a terminal device. For example... Figure 12 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. In the embodiments of this application, a smartphone is used as an example for illustration:

[0342] Figure 12 This is a block diagram illustrating a portion of the structure of a smartphone related to the terminal device provided in the embodiments of this application. (Reference) Figure 12 The smartphone includes components such as a radio frequency (RF) circuit 610, a memory 620, an input unit 630, a display unit 640, a sensor 650, an audio circuit 660, a wireless fidelity (WiFi) module 670, a processor 680, and a power supply 690. Those skilled in the art will understand that... Figure 12 The smartphone structure shown does not constitute a limitation on smartphones and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0343] The following is combined Figure 12 A detailed introduction to the various components of a smartphone:

[0344] RF circuit 610 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 680; additionally, it transmits uplink data to the base station. Typically, RF circuit 610 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, RF circuit 610 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), etc.

[0345] The memory 620 can be used to store software programs and modules. The processor 680 executes various functions and data processing of the smartphone by running the software programs and modules stored in the memory 620. The memory 620 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created based on the use of the smartphone (such as audio data, phonebook, etc.). In addition, the memory 620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0346] The input unit 630 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the smartphone. Specifically, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, also known as a touch screen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 631), and drive the corresponding connected devices according to a pre-set program. Optionally, the touch panel 631 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 680, and can also receive and execute commands sent by the processor 680. In addition, the touch panel 631 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 631, the input unit 630 may also include other input devices 632. Specifically, other input devices 632 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0347] The display unit 640 can be used to display information input by the user or information provided to the user, as well as various menus of the smartphone. The display unit 640 may include a display panel 641, optionally configured as a liquid crystal display (LCD), organic light-emitting diode (OLED), or similar form. Further, a touch panel 631 may cover the display panel 641. When the touch panel 631 detects a touch operation on or near it, it transmits the information to the processor 680 to determine the type of touch event. Subsequently, the processor 680 provides corresponding visual output on the display panel 641 based on the type of touch event. Although in Figure 12 In this embodiment, the touch panel 631 and the display panel 641 are two separate components to realize the input and output functions of the smartphone. However, in some embodiments, the touch panel 631 and the display panel 641 can be integrated to realize the input and output functions of the smartphone.

[0348] The smartphone may also include at least one sensor 650, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 641 according to the ambient light level, and the proximity sensor can turn off the display panel 641 and / or backlight when the smartphone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that recognize the smartphone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the smartphone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0349] Audio circuit 660, speaker 661, and microphone 662 provide an audio interface between the user and the smartphone. Audio circuit 660 converts received audio data into electrical signals and transmits them to speaker 661, where speaker 661 converts them into sound signals for output. On the other hand, microphone 662 converts collected sound signals into electrical signals, which are received by audio circuit 660, converted into audio data, and then processed by processor 680 before being transmitted via RF circuit 610 to, for example, another smartphone, or the audio data can be output to memory 620 for further processing.

[0350] WiFi is a short-range wireless transmission technology. Smartphones, through their WiFi module 670, can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 12 WiFi module 670 is shown, but it is understood that it is not an essential component of a smartphone and can be omitted as needed without changing the nature of the invention.

[0351] The processor 680 is the control center of the smartphone, connecting various parts of the smartphone through various interfaces and lines. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 620, and by calling data stored in the memory 620. Optionally, the processor 680 may include one or more processing units; optionally, the processor 680 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the aforementioned modem processor may also not be integrated into the processor 680.

[0352] The smartphone also includes a power supply 690 (such as a battery) that powers various components. Optionally, the power supply can be logically connected to the processor 680 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0353] Although not shown, smartphones may also include a camera, Bluetooth module, etc., which will not be described in detail here.

[0354] The steps performed by the terminal device in the above embodiments can be based on this Figure 12 The terminal device structure is shown.

[0355] This application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0356] This application also provides a computer program product including a program, which, when run on a computer, causes the computer to perform the methods described in the foregoing embodiments.

[0357] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0358] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0359] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0360] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0361] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0362] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for training a model, characterized in that, Applied to a local device; the method includes: Obtain K molecular sample features and K molecular annotation features, wherein the molecular sample features and the molecular annotation features have a corresponding relationship, and K is an integer greater than 1; Obtain at least three random numbers that satisfy a beta distribution; Based on the sum of the at least three random numbers, each random number is normalized to obtain the feature mixing cardinality corresponding to each random number, wherein the sum of the feature mixing cardinality corresponding to the at least three random numbers is 1; the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features; Using the feature mixing base corresponding to each random number, the features of the at least three molecular samples are weighted and summed to obtain the target molecular sample features of the mixed molecular samples; Using the feature mixing cardinality corresponding to each random number, the at least three molecular annotation features are weighted and summed to obtain the target molecular annotation features of the mixed molecular sample; the K molecular sample features and the K molecular annotation features originate from the K molecular samples; the K molecular samples include noisy molecular samples and noise-free molecular samples; Based on the target molecule sample features and the target molecule annotation features, a first local gradient value for the local model is obtained, and a second local gradient value is obtained when the model training conditions are met. Send the second local gradient value to the central device; The K molecular samples were obtained by the local device in the following manner: Send a data access request to the target local device; the target local device and the local device belong to different organizations. If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1. Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1; The P molecular samples and the Q molecular samples are used as the K molecular samples.

2. The method according to claim 1, characterized in that, The acquisition of K molecular sample features and K molecular annotation features includes: Obtain K molecular samples, where each molecular sample corresponds to at least one annotation result; For each molecular sample, feature extraction is performed on the molecular sample to obtain the molecular sample features corresponding to the molecular sample; For each molecular sample, a molecular annotation feature corresponding to the molecular sample is generated based on at least one annotation result corresponding to the molecular sample.

3. The method according to claim 2, characterized in that, The step of extracting features from molecular samples to obtain the molecular sample features corresponding to the molecular samples includes: A molecular graph sample is generated based on the molecular sample, wherein the molecular graph sample includes vertices and edges, the vertices representing atoms and the edges representing chemical bonds; Based on the molecular graph samples, the features of the molecular samples are obtained through a graph neural network.

4. The method according to claim 1, characterized in that, Before obtaining the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features, the method further includes: In response to the model update command sent by the central device, the model parameters of the global model are obtained from the central device, and the model parameters of the global model are used as the model parameters of the local model.

5. The method according to any one of claims 1 to 4, characterized in that, The step of obtaining the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features includes: Based on the target molecule sample features, the molecular attribute prediction score of the mixed molecule sample is obtained through the local model, wherein the molecular attribute prediction score represents the predicted score for the target molecule attribute; Based on the predicted scores of the molecular attributes and the labeled features of the target molecules, the first local gradient value for the local model is determined by a first loss function, wherein the labeled features of the target molecules represent the actual scores for the target molecular attributes.

6. The method according to any one of claims 1 to 4, characterized in that, The step of obtaining the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features includes: Based on the target molecular sample features, the molecular attribute prediction distribution of the mixed molecular sample is obtained through the local model, wherein the molecular attribute prediction distribution represents the probability of predicting each molecular attribute. Based on the predicted distribution of the molecular attributes and the target molecule annotation features, the first local gradient value for the local model is determined by a second loss function, wherein the target molecule annotation features represent the probability that the annotation belongs to each of the molecular attributes.

7. A method for training a model, characterized in that, include: The model parameters of the global model are sent to M local devices so that each local device uses the model parameters of the global model as the model parameters of its local model. Based on the target molecular sample features and target molecular annotation features of the mixed molecular samples, a first local gradient value is obtained for the local model. When the model training conditions are met, a second local gradient value is obtained. Here, M is an integer greater than or equal to 1. Receive the second local gradient value sent by each of the M local devices; The model parameters of the global model are updated based on the second local gradient value sent by each of the M local devices. The target molecular sample features and target molecular annotation features of the mixed molecular sample are generated by the local device based on K feature mixing cardinality, K molecular sample features and K molecular annotation features. The molecular sample features and the molecular annotation features have a corresponding relationship, and K is an integer greater than 1. The target molecular sample features and target molecular annotation features of the hybrid molecular sample are generated by the local device in the following manner: Obtain at least three random numbers that satisfy a beta distribution; Based on the sum of the at least three random numbers, each random number is normalized to obtain the feature mixing cardinality corresponding to each random number, wherein the sum of the feature mixing cardinality corresponding to the at least three random numbers is 1; the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features; Using the feature mixing base corresponding to each random number, the features of the at least three molecular samples are weighted and summed to obtain the target molecular sample features of the mixed molecular samples; Using the feature mixing cardinality corresponding to each random number, the at least three molecular annotation features are weighted and summed to obtain the target molecular annotation features of the mixed molecular sample; the K molecular sample features and the K molecular annotation features originate from the K molecular samples; the K molecular samples include noisy molecular samples and noise-free molecular samples; The K molecular samples were obtained by the local device in the following manner: Send a data access request to the target local device; the target local device and the local device belong to different organizations. If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1. Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1; The P molecular samples and the Q molecular samples are used as the K molecular samples.

8. The method according to claim 7, characterized in that, Before sending the model parameters of the global model to the M local devices, the method further includes: Obtain the first weight corresponding to each of the T local devices, where T is an integer greater than or equal to M, and the sum of the first weights of the T local devices is 1; Based on the first weight corresponding to each of the T local devices, the M local devices are obtained by sampling from the T local devices through a probability density function.

9. The method according to claim 7 or 8, characterized in that, The method further includes: Receive the loss value sent by each of the M local devices; Based on the loss value sent by each of the M local devices, the first weight corresponding to each of the M local devices is updated to obtain the second weight corresponding to each of the M local devices.

10. The method according to claim 9, characterized in that, The step of updating the first weight corresponding to each of the M local devices based on the loss value sent by each of the M local devices to obtain the second weight corresponding to each of the M local devices includes: Based on the loss value sent by each of the M local devices, the first weight corresponding to each of the M local devices is updated to obtain the weight to be processed corresponding to each of the M local devices; Based on the sum of the unprocessed weights corresponding to each of the M local devices, each unprocessed weight is normalized to obtain the normalized weight corresponding to each of the M local devices. Based on the sum of the remaining weights and the normalized weights corresponding to each of the M local devices, the second weight corresponding to each of the M local devices is determined.

11. A model training device, characterized in that, Applied to a local device; the device includes: The acquisition module is used to acquire K molecular sample features and K molecular annotation features, wherein the molecular sample features and the molecular annotation features have a corresponding relationship, and K is an integer greater than 1; The determination module is used to obtain at least three random numbers that satisfy a beta distribution; based on the sum of the at least three random numbers, normalize each random number to obtain the feature mixing cardinality corresponding to each random number, wherein the sum of the feature mixing cardinality corresponding to the at least three random numbers is 1; the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features; The generation module is used to perform a weighted summation of the features of the at least three molecular samples using the feature mixing base corresponding to each random number to obtain the target molecular sample features of the mixed molecular sample; and to perform a weighted summation of the label features of the at least three molecular samples using the feature mixing base corresponding to each random number to obtain the target molecular label features of the mixed molecular sample; the K molecular sample features and the K molecular label features originate from K molecular samples; the K molecular samples include noisy molecular samples and noise-free molecular samples; The acquisition module is also used to acquire a first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features, and to obtain a second local gradient value when the model training conditions are met. The sending module is used to send the second local gradient value to the central device; The acquisition module is specifically used for: Send a data access request to the target local device; the target local device and the local device belong to different organizations. If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1. Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1; The P molecular samples and the Q molecular samples are used as the K molecular samples.

12. The apparatus according to claim 11, characterized in that, The acquisition module is specifically used for: Obtain K molecular samples, where each molecular sample corresponds to at least one annotation result; For each molecular sample, feature extraction is performed on the molecular sample to obtain the molecular sample features corresponding to the molecular sample; For each molecular sample, a molecular annotation feature corresponding to the molecular sample is generated based on at least one annotation result corresponding to the molecular sample.

13. The apparatus according to claim 12, characterized in that, The acquisition module is specifically used for: A molecular graph sample is generated based on the molecular sample, wherein the molecular graph sample includes vertices and edges, the vertices representing atoms and the edges representing chemical bonds; Based on the molecular graph samples, the features of the molecular samples are obtained through a graph neural network.

14. The apparatus according to claim 11, characterized in that, The acquisition module is further configured to, before acquiring the first local gradient value for the local model based on the target molecule sample features and the target molecule annotation features, respond to the model update instruction sent by the central device, acquire the model parameters of the global model from the central device, and use the model parameters of the global model as the model parameters of the local model.

15. The apparatus according to any one of claims 11 to 14, characterized in that, The acquisition module is specifically used for: Based on the target molecule sample features, the molecular attribute prediction score of the mixed molecule sample is obtained through the local model, wherein the molecular attribute prediction score represents the predicted score for the target molecule attribute; Based on the predicted scores of the molecular attributes and the labeled features of the target molecules, the first local gradient value for the local model is determined by a first loss function, wherein the labeled features of the target molecules represent the actual scores for the target molecular attributes.

16. The apparatus according to any one of claims 11 to 14, characterized in that, The acquisition module is specifically used for: Based on the target molecular sample features, the molecular attribute prediction distribution of the mixed molecular sample is obtained through the local model, wherein the molecular attribute prediction distribution represents the probability of predicting each molecular attribute. Based on the predicted distribution of the molecular attributes and the target molecule annotation features, the first local gradient value for the local model is determined by a second loss function, wherein the target molecule annotation features represent the probability that the annotation belongs to each of the molecular attributes.

17. A model training device, characterized in that, include: The sending module is used to send the model parameters of the global model to M local devices, so that each local device uses the model parameters of the global model as the model parameters of its local model, and obtains a first local gradient value for the local model based on the target molecular sample features and target molecular annotation features of the mixed molecular samples. When the model training conditions are met, a second local gradient value is obtained, wherein M is an integer greater than or equal to 1. The receiving module is configured to receive the second local gradient value sent by each of the M local devices; The update module is used to update the model parameters of the global model based on the second local gradient value sent by each of the M local devices; The target molecular sample features and target molecular annotation features of the mixed molecular sample are generated by the local device based on K feature mixing cardinality, K molecular sample features and K molecular annotation features. The molecular sample features and the molecular annotation features have a corresponding relationship, and K is an integer greater than 1. The target molecular sample features and target molecular annotation features of the hybrid molecular sample are generated by the local device in the following manner: Obtain at least three random numbers that satisfy a beta distribution; Based on the sum of the at least three random numbers, each random number is normalized to obtain the feature mixing cardinality corresponding to each random number, wherein the sum of the feature mixing cardinality corresponding to the at least three random numbers is 1; the K molecular sample features include at least three molecular sample features, and the K molecular annotation features include at least three molecular annotation features; Using the feature mixing base corresponding to each random number, the features of the at least three molecular samples are weighted and summed to obtain the target molecular sample features of the mixed molecular samples; Using the feature mixing cardinality corresponding to each random number, the at least three molecular annotation features are weighted and summed to obtain the target molecular annotation features of the mixed molecular sample; the K molecular sample features and the K molecular annotation features originate from the K molecular samples; the K molecular samples include noisy molecular samples and noise-free molecular samples; The K molecular samples were obtained by the local device in the following manner: Send a data access request to the target local device; the target local device and the local device belong to different organizations. If the target local device authenticates the data access request, then P molecular samples are obtained from the target local device, where P is an integer greater than or equal to 1. Retrieve Q molecular samples from the local database, where Q is an integer greater than or equal to 1; The P molecular samples and the Q molecular samples are used as the K molecular samples.

18. The apparatus according to claim 17, characterized in that, The device further includes: an acquisition module and a sampling module; The acquisition module is used to acquire the first weight corresponding to each of the T local devices before sending the model parameters of the global model to the M local devices, wherein T is an integer greater than or equal to M, and the sum of the first weights of the T local devices is 1. The sampling module is used to sample the M local devices from the T local devices using a probability density function based on the first weight corresponding to each of the T local devices.

19. The apparatus according to claim 17 or 18, characterized in that, The receiving module is also used to receive the loss value sent by each of the M local devices; The update module is further configured to update the first weight corresponding to each of the M local devices based on the loss value sent by each of the M local devices, so as to obtain the second weight corresponding to each of the M local devices.

20. The apparatus according to claim 19, characterized in that, The update module is specifically used for: Based on the loss value sent by each of the M local devices, the first weight corresponding to each of the M local devices is updated to obtain the weight to be processed corresponding to each of the M local devices; Based on the sum of the unprocessed weights corresponding to each of the M local devices, each unprocessed weight is normalized to obtain the normalized weight corresponding to each of the M local devices. Based on the sum of the remaining weights and the normalized weights corresponding to each of the M local devices, the second weight corresponding to each of the M local devices is determined.

21. A computer device, characterized in that, include: Memory, processor, and bus system; The memory is used to store programs; The processor is configured to execute a program in the memory, and the processor is configured to execute the method of any one of claims 1 to 6 according to the instructions in the program code, or to execute the method of any one of claims 7 to 10; The bus system is used to connect the memory and the processor to enable communication between the memory and the processor.

22. A computer-readable storage medium comprising instructions, when executed on a computer, causing the computer to perform the method as claimed in any one of claims 1 to 6, or to perform the method as claimed in any one of claims 7 to 10.

23. A computer program product, comprising a computer program and instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the method as described in any one of claims 1 to 6, or implements the method as described in any one of claims 7 to 10.

Images