Super-network representation learning method based on multi-head attention mechanism applied to breeding

Abstract

The application discloses a multi-head attention mechanism-based super network representation learning method applied to breeding, relates to the technical field of crop breeding, and comprises the following steps: constructing a crop heterogeneous super network, and constructing an initial embedding matrix corresponding to all nodes; obtaining a normalized node representation vector corresponding to each tuple and a similarity score of each tuple; calculating the pairwise similarity between each two nodes; constructing a joint loss function, and optimizing the parameters of a target model; generating a final representation vector of a node in the crop heterogeneous super network by using the target model after parameter optimization; and predicting a crop variety according to the final representation vector. The application avoids complex graph conversion and explicit high-order combination calculation, improves the calculation efficiency and scalability, can more comprehensively and accurately encode semantic and structural information of phenotypic traits, and thus provides reliable and high-quality data basis for crop variety prediction based on the final representation vector.

Description

Technical Field

[0001] This invention relates to the field of crop breeding technology, and in particular to a hypernetwork representation learning method based on a multi-head attention mechanism for application in breeding. Background Technology

[0002] In the field of crop breeding, there is a pressing technical problem: how to automatically and accurately learn phenotypic representations that profoundly reflect breeding patterns from massive, multi-dimensional, and complex phenotypic and varietal data. This data naturally constitutes a heterogeneous supernetwork, where nodes are different types of phenotypic traits, and hyperedges are crop varieties defined by multiple traits. The essence of this problem is to achieve high-quality representation learning from such heterogeneous crop supernetworks to support accurate variety prediction and design.

[0003] To address the aforementioned issues, existing technologies offer various hypernetwork representation learning methods. These methods can be broadly categorized as follows: matrix factorization-based methods, which decompose the node-hyperedge association matrix to obtain a low-dimensional representation; random walk-based methods, which generate sequences by designing special walk strategies within hyperedges and then borrowing graph representation learning techniques; deep learning-based methods, including using graph neural networks to process the transformed ordinary graph, designing specialized hypergraph neural networks for message aggregation, and methods based on autoencoders to reconstruct structural information; and attention-based methods, which assign differentiated weights to different nodes or hyperedges during information aggregation.

[0004] However, existing methods exhibit significant drawbacks when applied to crop breeding scenarios. First, they are insufficient in modeling higher-order heterogeneity. Existing methods typically treat multiple phenotypic traits (nodes) constituting a variety (hyperedge) as equally important or perform simple weighting, ignoring potential differences in contribution and complex interaction patterns (such as complementarity and antagonism) between traits. This results in a "flattened" trait representation, losing the fine-grained structure within the variety. Second, they suffer from high computational complexity and poor scalability. The higher-order characteristics of hypernetworks make directly extending traditional graph neural networks and other methods face combinatorial explosions in computational and memory overhead, making it difficult to handle large-scale breeding data. Third, hyperedge fusion is insufficient. Many methods simply decompose varieties (hyperedges) into paired trait relationships, destroying the crucial higher-order collaborative information that "multiple traits collectively define a variety," leading to an inability to effectively capture and utilize the holistic semantics at the variety level.

[0005] Therefore, given the specific needs of heterogeneous supernetwork representation learning in crop breeding and the shortcomings of existing technologies, a new technical solution is urgently needed. This solution needs to effectively model complex pairwise and high-order tuple relationships between traits, fully integrate the overall information of hyperedges while ensuring computational efficiency, and thus learn more discriminative and semantically rich phenotypic representations, providing core technical support for data-driven intelligent breeding. Summary of the Invention

[0006] The technical problem to be solved by this invention is to address the shortcomings of existing technologies, and the following technical solution is provided: 1) In a first aspect, the present invention provides a hypernetwork representation learning method based on a multi-head attention mechanism for application in breeding, the specific technical solution of which is as follows: Construct a heterogeneous supernetwork for crops, where nodes in the heterogeneous supernetwork represent phenotypic traits of crop varieties, and superedges in the heterogeneous supernetwork represent crop varieties, with each superedge containing at least two nodes; Construct the initial embedding matrix for all nodes; For each tuple in the heterogeneous supernetwork of crops, based on the initial embedding matrix, the target model is used to process it to obtain the normalized node representation vector corresponding to each tuple and the similarity score of each tuple; the target model includes a multi-head attention layer, a feedforward network layer, a global average pooling layer and a fully connected layer containing an activation function, and has residual connection operation and layer normalization operation. Based on the normalized node representation vectors corresponding to all tuples, calculate the pairwise similarity between every two nodes in the crop heterogeneous supernetwork; Based on the similarity scores of all tuples and all pairwise similarities, a joint loss function is constructed, and the parameters of the target model are optimized based on the joint loss function. The final representation vectors of nodes in a heterogeneous supernetwork of crops are generated using the target model with optimized parameters. Crop variety prediction is performed based on the final representation vector.

[0007] The beneficial effects of the hypernetwork representation learning method based on multi-head attention mechanism for breeding provided by this invention are as follows: By constructing a heterogeneous supernetwork for crops, the complex relationship between phenotypic traits and varieties is formally represented. Utilizing a target model incorporating multi-head attention layers, feedforward layers, global average pooling layers, residual connections, and layer normalization operations, tuples are directly processed, effectively overcoming the problem of insufficient hyperedge fusion. This allows the model to capture high-order collaborative information where multiple phenotypic traits define a variety as a whole. The multi-head attention mechanism allows the model to automatically learn the heterogeneous relationships between nodes within a tuple, thereby improving the granularity of high-order heterogeneity modeling. By uniformly optimizing tuple similarity scores and pairwise similarities through a joint loss function, an organic unified learning of pairwise relationships and high-order tuple relationships is achieved. This method avoids complex graph transformations and explicit high-order combinatorial calculations, improving computational efficiency and scalability. Finally, the node final representation vectors generated by the parameter-optimized target model can more comprehensively and accurately encode the semantic and structural information of phenotypic traits, thus providing a reliable and high-quality data foundation for crop variety prediction based on the final representation vectors.

[0008] Based on the above scheme, the supernetwork representation learning method based on multi-head attention mechanism applied to breeding, as proposed in this invention, can be further improved as follows.

[0009] Furthermore, the process of obtaining the normalized node representation vector corresponding to each tuple and the similarity score of each tuple includes: Based on the initial embedding matrix, a multi-head attention mechanism is used to extract features and interact with nodes within any tuple, resulting in a multi-head attention output. This multi-head attention output is then input into a feedforward network for nonlinear transformation, yielding a feedforward output. The multi-head attention output and the feedforward output are then connected via residual connections to obtain a fused output. The fused output is then subjected to layer normalization to obtain a normalized node representation vector corresponding to the tuple. This normalized node representation vector is then input into the global average pooling layer of the target model for compression, resulting in a pooling vector. Finally, the pooling vector is input into a fully connected layer containing an activation function to obtain a similarity score for the tuple. This process is repeated for each tuple to obtain its corresponding normalized node representation vector and similarity score.

[0010] The beneficial effects of adopting the above-mentioned further scheme are as follows: By explicitly using a multi-head attention mechanism to extract features and interact with nodes within a tuple based on the initial embedding matrix, this step enables the model to capture multiple dependency patterns between phenotypic trait nodes in parallel and with fine precision. Residually connecting the multi-head attention output and the feedforward output and performing layer normalization effectively alleviates the information decay and gradient instability problems in deep network training, ensuring the stability of the learning process and the quality of the node representation vectors. Finally, generating tuple similarity scores through a global average pooling layer and a fully connected layer containing activation functions, this process achieves the compression and quantification of the overall semantics of the tuples, providing a direct and optimizable metric for evaluating the rationality of trait combinations within a variety.

[0011] Furthermore, the pairwise similarity between every two nodes is obtained, including: For any pair of nodes, calculate the conditional probability between the first and second nodes in the pair based on the normalized node representation vectors of the two nodes. Calculate the pairwise similarity of the pair based on the conditional probability between the first and second nodes in the pair. Iterate through each pair of nodes to obtain the pairwise similarity between every two nodes.

[0012] The beneficial effects of adopting the above-mentioned further scheme are as follows: By defining the conditional probability calculated based on the normalized node representation vectors of two nodes, and using this as the basis for calculating pairwise similarity, this method provides a clear mathematical foundation for quantifying the association strength between any two phenotypic trait nodes. The process of traversing each node pair to calculate all pairwise similarities ensures a comprehensive examination of potential pairwise relationships in the heterogeneous supernetwork of crops. This calculation method effectively compensates for the shortcomings of the tuple similarity model in characterizing fine-grained pairwise relationships, enabling the model to capture high-order cooperative information while not neglecting basic and important pairwise associations between traits, thereby enhancing the completeness and discriminative power of the final representation vector.

[0013] Furthermore, the fully connected layer containing the activation function uses the sigmoid function.

[0014] The advantages of adopting the above further approach are: specifying that the fully connected layer containing the activation function uses the sigmoid function, this choice restricts the value range of the tuple similarity score to a continuous interval between 0 and 1. This value range characteristic allows the scoring results to have a clear probabilistic interpretation, directly representing the probability that the combination of traits represented by a tuple constitutes a real or ideal crop variety. This design is not only compatible with subsequent loss function calculations based on negative sampling, but also provides a stable and interpretable optimization objective for model training, while making the final variety prediction results intuitively comparable and operable.

[0015] 2) In a second aspect, the present invention also provides a hypernetwork representation learning system based on a multi-head attention mechanism for use in breeding, the specific technical solution of which is as follows: It includes a first building module, a second building module, a scoring calculation module, a similarity calculation module, a parameter optimization module, a generation module, and a prediction module; The first construction module is used to: construct a heterogeneous supernetwork of crops, wherein the nodes in the heterogeneous supernetwork of crops represent the phenotypic traits of crop varieties, the superedges in the heterogeneous supernetwork of crops represent crop varieties, and each superedge contains at least two nodes; The second building module is used to: construct the initial embedding matrix corresponding to all nodes; The scoring calculation module is used to: for each tuple in the heterogeneous supernetwork of crops, based on the initial embedding matrix, process it using the target model to obtain the normalized node representation vector corresponding to each tuple and the similarity score of each tuple; the target model includes a multi-head attention layer, a feedforward network layer, a global average pooling layer and a fully connected layer containing activation functions, and has residual connection operation and layer normalization operation. The similarity calculation module is used to: calculate the pairwise similarity between every two nodes in the crop heterogeneous supernetwork based on the normalized node representation vectors corresponding to all tuples; The parameter optimization module is used to: construct a joint loss function based on the similarity scores of all tuples and all pairwise similarities, and optimize the parameters of the target model based on the joint loss function; The generation module is used to: generate the final representation vectors of nodes in the heterogeneous supernetwork of crops using the parameter-optimized target model; The prediction module is used to predict crop varieties based on the final representation vector.

[0016] Based on the above scheme, the supernetwork representation learning system based on multi-head attention mechanism applied to breeding, as proposed in this invention, can be further improved as follows.

[0017] Furthermore, the scoring calculation module is specifically used for: based on the initial embedding matrix, using a multi-head attention mechanism to extract features and interact with nodes within any tuple to obtain a multi-head attention output; inputting the multi-head attention output into a feedforward network for nonlinear transformation to obtain a feedforward output; performing a residual connection between the multi-head attention output and the feedforward output to obtain a fused output; performing layer normalization on the fused output to obtain a normalized node representation vector corresponding to the tuple; inputting the normalized node representation vector into the global average pooling layer of the target model for compression to obtain a pooling vector; inputting the pooling vector into a fully connected layer containing an activation function to obtain a similarity score for the tuple; traversing each tuple to obtain the normalized node representation vector and similarity score corresponding to each tuple.

[0018] Furthermore, the similarity calculation module is specifically used for: for any two nodes forming a node pair, calculating the conditional probability between the first and second nodes in the node pair based on the normalized node representation vectors of the two nodes, calculating the pairwise similarity of the node pair based on the conditional probability between the first and second nodes in the node pair, and traversing each node pair to obtain the pairwise similarity between every two nodes.

[0019] Furthermore, the fully connected layer containing the activation function uses the sigmoid function.

[0020] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so that the electronic device implements any of the above-mentioned hypernetwork representation learning methods based on multi-head attention mechanism applied to breeding.

[0021] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned hypernetwork representation learning methods based on multi-head attention mechanisms applied to breeding.

[0022] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below: Figure 1 This is a flowchart illustrating a hypernetwork representation learning method based on a multi-head attention mechanism applied to breeding, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a heterogeneous supernetwork. Figure 3 This is a schematic diagram of the HyperAtt neural network model; Figure 4 The experimental results for GPS hypernetwork reconstruction; Figure 5 Experimental results for drug hypernetic network reconstruction; Figure 6 This is a schematic diagram of the structure of a hypernetwork representation learning system based on a multi-head attention mechanism applied to breeding, according to an embodiment of the present invention. Detailed Implementation

[0024] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0025] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0026] like Figure 1 As shown in the figure, an embodiment of the present invention provides a hypernetwork representation learning method based on a multi-head attention mechanism for use in breeding, comprising the following steps: S1. Construct a heterogeneous supernetwork for crops, where nodes represent phenotypic traits of crop varieties, and hyperedges represent crop varieties. Each hyperedge contains at least two nodes. The specific implementation process is as follows: S10. Collect a batch of varieties of the target crop (such as rice and wheat) and their detailed phenotypic trait measurement data. The data is usually in matrix or tabular form, with each row corresponding to a crop variety and each column corresponding to a phenotypic trait measurement value or classification level. First, all phenotypic traits need to be sorted and classified. Assume a total of [number missing] traits are defined. Class traits, for example Indicates yield-related traits, Indicates quality-related traits. This represents traits such as resistance to biological stress. For each type of trait, enumerate all specific trait indicators it includes. Assuming that after organization, there are a total of Each trait is a specific trait indicator. Each trait indicator is uniquely assigned a type label. and a global node identifier Thus, a node set was established. And know each node Type of ,Right now Includes all types The node.

[0027] S11. Original phenotypic measurements (e.g., plant height 105.3 cm) are usually continuous or ordered categorical values. To construct a hypernetwork based on co-occurrence relationships, these values ​​need to be transformed into Boolean relationships. For each phenotypic node... You can set a threshold or standard for it. For example, for the node "plant height", a threshold can be set. For a specific crop variety, if its measured plant height is greater than... If the trait "possesses" the characteristic of tall stem, then this variety is considered to have this trait, and is recorded as the variety and node. Associated; otherwise, considered not to possess. In this way, the expression of all varieties in all traits is transformed into a single... A binary eigenvector with dimensions 1, where each dimension has a value of 1 indicating the presence of the trait and 0 indicating the absence of the trait.

[0028] S12, Iterate through each crop variety in the dataset. For the first... For each variety, examine its binary feature vector after S11 processing. Collect the phenotypic trait nodes corresponding to all dimensions with a value of 1 in this vector to form a set. This set represents the hyperedge of that variety. Formalistically, Clearly, the number of nodes contained in each hyperedge... Because a variety is defined by at least several traits. Assume the dataset has a total of... If there are multiple varieties, then a super-edge set is constructed. ,in .

[0029] S13. Integrate the results of the first three steps to form a complete formal definition of the crop heterogeneous supernetwork. This network is fully described by a triple: .in, It is a set of phenotypic trait nodes. It is a set of superedges for crop varieties, where each superedge is a subset of nodes. It is a mapping function for each node. Assign a type label, It is a collection of types, totaling Types, namely Represents a node Belongs to the Types. This structure It forms the input basis for all subsequent representation learning operations.

[0030] Among them, the heterogeneous hypernetwork of crops is a high-order relational structure used to model crop breeding data. It abstracts different types of crop phenotypic traits into nodes of different types, and each specific crop variety into a hyperedge. Mathematically, this network is formalized as a hypergraph. ,in It is a set of nodes. It is a set of hyperedges. Because the nodes represent a variety of different phenotypic traits (such as plant height, number of grains per ear, disease resistance, etc.), these nodes belong to different types, so the network is called a "heterogeneous" supernetwork. Its core purpose is to learn representations that can profoundly reflect breeding patterns by capturing the complex high-order co-occurrence relationships between phenotypic traits (i.e., which traits tend to appear simultaneously in the same variety), and thus guide the prediction of new varieties.

[0031] In a heterogeneous supernetwork of crops, a superedge represents a specific crop variety. It connects nodes representing multiple phenotypic traits possessed by that variety. Unlike a regular graph where an edge connects only two nodes, a superedge can contain two or more nodes, which corresponds to the fact that a crop variety typically possesses multiple traits. For example, a wheat variety may simultaneously possess traits such as "tall stalk," "large ears," and "rust resistance." The superedge representing this variety would connect these three nodes, representing "plant height," "ear morphology," and "disease resistance," respectively. (Superedge set) Each hyperedge in They are all subsets of a single node, that is ,in .

[0032] The specific crop varieties can be: rice variety "Nanjing 46", corn variety "Zhengdan 958" or wheat variety "Jimai 22", etc., and can also be set according to the actual situation.

[0033] Phenotypic traits of crop varieties refer to the morphological, physiological, and biochemical characteristics of the variety that can be observed and measured under specific environmental conditions. These traits serve as a bridge between genotype and phenotype and are key factors for breeders in their selection processes. In this technical solution, each specific phenotypic trait is modeled as a node in a heterogeneous supernetwork of crops. For example, in rice breeding, "thousand-grain weight," "heading date," and "rice blast resistance" are all important phenotypic trait nodes. (Node set) It is the set of all such trait nodes, and can be further divided into different type subsets according to the category of the trait (such as yield trait, quality trait, stress resistance trait). ,in Indicates type index, This represents the total number of node types, i.e. .

[0034] S2. Construct the initial embedding matrix for all nodes. The specific implementation process is as follows: S20. The initial embedding matrix is ​​mathematically defined as a real-valued matrix, denoted as . This matrix contains lines and Column. Among them, symbols This represents the total number of nodes displaying phenotypic traits in a heterogeneous crop supernetwork; this value is determined after the supernetwork is constructed. (Symbol) The dimension representing the embedding vector is a hyperparameter that needs to be predefined. (Matrix) The Okay, that is It is A row vector of dimension 1, specifically used to represent the 1st-order row vector in the network. The initial features of each phenotypic trait node. Therefore, this matrix forms a lookup table, accessed by node index. You can obtain its corresponding Dimensional initial embedding vector.

[0035] S21, Embedding Dimension Dimension is an important model hyperparameter that determines the capacity of the represented vector. The choice of needs to strike a balance between model expressiveness and computational complexity. A common practice is to set it to a power of 2, such as 64, 128, or 256, which is beneficial for computing devices (such as GPUs) to perform efficient memory alignment and parallel computation. Specific values ​​can be determined through small-scale experiments on a validation set, for example, by trying different ... Value and observe the model's initial performance on validation tasks (such as link prediction). In technical implementation, It is defined as a constant at the beginning of the program.

[0036] S22, Initialization is for a matrix Each element in the algorithm is assigned an initial random value. Good initialization is crucial for stable model training and final performance. A common strategy is to use Xavier uniform initialization. This strategy sets the range of random numbers based on the dimensions of the input and output, helping to maintain stable variance of activation values ​​in each layer during the early stages of training. Specifically, it involves using a uniform distribution... Random sampling in the matrix Each element is assigned a value. The range parameter... The calculation formula is: In this formula, It is the total number of nodes. It is the embedded dimension. This represents the square root operation. The initial values ​​obtained by sampling in this way have a moderate distribution range, which can effectively promote the flow of gradients in the subsequent neural network and avoid gradient explosion or vanishing problems in the early stage of training.

[0037] S23. To accurately obtain the corresponding embedding vector for each tuple or node pair in subsequent calculations, a stable and consistent mapping rule must be established. When constructing a heterogeneous supernetwork for crops, each phenotypic node... It has been assigned a unique integer index ,in, This index This will be directly used from the initial embedding matrix. Find the row number of its corresponding vector. For example, when the model needs to process a vector containing nodes... When dealing with tuples, the technical implementation involves an index operation from the matrix. Extract the first Rows are then assembled into a submatrix. This serves as the initial feature input for the tuple. This mapping relationship is definite and one-to-one.

[0038] S24. Within the framework of deep learning, the initial embedding matrix... It is not fixed. It needs to be defined as a trainable parameter tensor. During model training, through backpropagation algorithms and optimizers (such as Adam), the matrix... Each element in the model is iteratively updated based on the gradient of the loss function (such as the joint loss function). This process is the core of "representation learning," where the model adjusts these values ​​to ultimately learn the node representation vector. It can increasingly accurately capture the structural roles and semantic information of phenotypic node in heterogeneous crop supernetworks. Therefore, the initial embedding matrix The construction essentially involves setting a random, but data-driven, starting coordinate for each node in a continuous vector space.

[0039] S3. For each tuple in the heterogeneous supernetwork of crops, based on the initial embedding matrix, the target model is used to process it to obtain the normalized node representation vector corresponding to each tuple and the similarity score of each tuple; the target model includes a multi-head attention layer, a feedforward network layer, a global average pooling layer and a fully connected layer containing an activation function, and has residual connection operation and layer normalization operation. The process of obtaining the normalized node representation vector for each tuple and the similarity score for each tuple includes: Based on the initial embedding matrix, a multi-head attention mechanism is used to extract features and interact with nodes within any tuple, resulting in a multi-head attention output. This multi-head attention output is then input into a feedforward network for nonlinear transformation, yielding a feedforward output. The multi-head attention output and the feedforward output are then concatenated using residual connections to obtain a fused output. This fused output is then normalized to obtain a normalized node representation vector for that tuple. This normalized node representation vector is then input into the global average pooling layer of the target model for compression, resulting in a pooling vector. Finally, this pooling vector is input into a fully connected layer containing an activation function to obtain a similarity score for that tuple. This process is repeated for each tuple, yielding a normalized node representation vector and a similarity score for each tuple. The fully connected layer using the activation function is the sigmoid function. The specific implementation process is as follows: S30. Construct and train the HyperAtt neural network model to obtain the target model, specifically: The input to the HyperAtt neural network model is a tuple. First, based on each node The unique index in the hypernetwork, derived from the constructed initial embedding matrix. Find its corresponding initial value in the middle. 1-dimensional vectors. These vectors are stacked using indexing operations to form the initial feature matrix of the tuple. This matrix It will serve as the input for all subsequent neural network layers.

[0040] The multi-head attention layer is the core of the HyperAtt neural network model, used to extract complex interaction features between nodes within a tuple. This layer uses [variable name] in parallel. Each independent attention head. For the first Each attention head has three independent sets of trainable parameter matrices: query projection matrix. Key projection matrix Value projection matrix For simplicity, usually... The calculation process for each head is as follows: ① Input Projected onto the query, key, and value spaces respectively: .symbol Representing the first The query, key, and value matrix of each head.

[0041] ② Calculate the scaled dot product attention weights and apply them to the value matrix: .in It is a scaling factor used to stabilize the gradient.

[0042] all The output of each attention head The data is concatenated along the feature dimension and then passed through a trainable linear projection matrix. By integrating the results, we obtain the final output of the multi-head attention layer: .symbol It is the updated representation matrix of each node in the tuple after multi-head attention interaction.

[0043] The feedforward network layer of the HyperAtt neural network model Each node in the algorithm represents a vector that undergoes two independent linear transformations, with nonlinearity introduced in between using the ReLU activation function. Its computational definition is: .in, and It's the first layer of weights and biases, which change the dimension from... Mapping to higher (e.g., feedforward network dimension). and The second layer of weights and biases maps the dimensions back to... . It is the output matrix of the feedforward network.

[0044] To prevent network degradation in the HyperAtt neural network model and to facilitate training, the output of the multi-head attention layer is... With the output of the feedforward network layer Perform element-wise additive residual join: Next, regarding Each row of the matrix (i.e., the representation vector of each node) Perform layer normalization to stabilize the training process: . and They are vectors The mean and variance of. It is a very small constant (e.g., 1e-12) used to prevent division by zero. and These are learnable scaling factors and bias parameters. This operation yields a normalized node representation vector matrix. This is the representation that the model learns for each node in the current tuple, after a layer of deep feature interaction.

[0045] Normalized node representation matrix Global average pooling is performed along the node dimension (i.e., the row direction) to compress the information of all nodes into a single pooling vector: .symbol This represents the overall feature vector of the tuple. Then, this pooled vector is fed into a fully connected layer containing an activation function. This fully connected layer consists of a weight matrix... and a bias scalar Constructed and using the sigmoid function As an activation function, the output is constrained to the (0,1) interval to obtain the similarity score of the tuples: .

[0046] To train the HyperAtt neural network model, we need to define the objective function of the tuple similarity model. The negative sampling strategy is adopted, with the goal of minimizing the following loss: .in, It is a set of positive elements constructed from the superedges of real varieties. It is a set of negative tuples constructed by randomly replacing some nodes in the positive tuples. The model (including the initial embedding matrix) All attention head parameters Projection matrix Feedforward network parameters Normalized parameters and rating layer parameters All parameters of the algorithm are iteratively updated using backpropagation and optimizers (such as Adam) to minimize the error. Training stops when the loss function converges on the validation set or reaches the preset number of training rounds. The model with frozen parameters obtained at this point is the "target model".

[0047] S31. After completing the previous step, a parameter-optimized target model is obtained. This model is then used to perform forward propagation computation on each tuple that needs to be processed in the crop heterogeneous supernetwork. Specifically: Prepare the set of tuples to be processed (e.g., tuples corresponding to variety superedges in all training and test sets, or new tuples constructed for prediction). Simultaneously, load the saved target model parameters and set the model to evaluation mode. In this mode, all model parameters are frozen and no longer updated, and randomization operations such as Dropout, which are only used during training, are disabled. For each tuple in the set... Repeat the following operations: Convert tuples The input is fed into the loaded target model. Internally, the model sequentially performs embedding lookup, multi-head attention calculation, feedforward network transformation, residual connections, layer normalization, global average pooling, and sigmoid fully connected layer calculation. This complete forward propagation process is deterministic. The forward propagation process of the target model produces two key outputs. The first output is the node representation matrix obtained after the layer normalization operation. It provides the normalized node representation vector for each phenotypic node within the tuple, after deep model processing. The second output is a scalar value. This represents the similarity score of the tuple. After processing all tuples, we obtain the normalized set of node representation vectors for each tuple and the set of similarity scores for each tuple. These outputs will serve as the basis for subsequent calculations of pairwise similarity, construction of joint loss functions (during the training phase), or direct use in variety prediction analysis.

[0048] In this context, a tuple specifically refers to an ordered or unordered combination of nodes representing multiple phenotypic traits. These nodes originate from the same crop variety superedge or are a set of nodes constructed for modeling purposes. Formally, a tuple... It can be represented as ,in arrive yes The identifier, symbol of each phenotypic trait node. This represents the number of nodes contained in the tuple. Tuples are the basic processing unit for the model to learn higher-order relations. For example, a tuple representing a certain rice variety might contain nodes such as "thousand-grain weight," "heading date," and "rice blast resistance." The model learns how these traits collectively define the variety by analyzing the interactions between the nodes within this tuple.

[0049] The tuple similarity score is a scalar value between 0 and 1, calculated by the target model, and is used to quantify the tightness and reasonableness of the relationships between all phenotypic trait nodes within a tuple. A higher score means the model considers the trait combination represented by the tuple to be more likely to constitute a real, reasonable, or "ideal" crop variety. This score is obtained by aggregating the deep representations of the nodes within the tuple and mapping them through a sigmoid activation function. During the training phase, it is used to distinguish between real-world varieties (positive tuples) and randomly constructed, non-existent trait combinations (negative tuples), thereby driving the model to learn effective node representations. Its calculation process is represented as follows: .

[0050] S4. Based on the normalized node representation vectors corresponding to all tuples, calculate the pairwise similarity between every two nodes in the crop heterogeneous supernetwork. Specifically, for any pair of nodes, calculate the conditional probability between the first and second nodes in the pair based on the normalized node representation vectors of the two nodes. Based on the conditional probability between the first and second nodes in the pair, calculate the pairwise similarity of the pair. Iterate through each pair of nodes to obtain the pairwise similarity between every two nodes. The specific implementation process is as follows: S40. For any two nodes to be calculated and First, we need to obtain their depth representation vectors. These vectors are derived from the output of the target model after processing the tuples containing the node in step S3. Specifically, for a node... It is in a certain tuple After passing through a multi-head attention layer, a feedforward network layer, and layer normalization operations, a stable and semantically rich normalized node representation vector is obtained, denoted as . Similarly, nodes The representation vector is denoted as .symbol The final representation vector function of the node. This represents the dimension of the vector. When calculating pairwise similarity, these learned vectors are used directly. and .

[0051] S41, A node needs to be calculated. Appears on another node Conditional probability in context In pairwise relationship modeling, nodes are... and They form a context for each other. Conditional probability. Indicates at a given node Under the condition, node The probability of appearing in its context is theoretically defined based on the softmax function: in, The dot product of two vector representations is used to measure their similarity. The denominator is... It is for all nodes in the network Summation is performed, which is a computationally expensive normalization factor. (Symbol) It represents the entire set of nodes.

[0052] S42, Node and Pairwise similarity They are defined as being associated with all possible context nodes. The joint product of conditional probabilities: For ease of calculation, the logarithmic form is usually taken, as shown in equation (11): Substituting the definition of conditional probability, we get: in, , These are respectively called nodes. and The partition function.

[0053] S43, Direct Calculation and The entire set of nodes needs to be processed. Summation, in the number of nodes This is not feasible when the value is very large. Therefore, negative sampling is used for efficient approximation. For nodes... No longer enumerate all context nodes Instead, it uses an objective function to distinguish positive sample pairs. and negative sample pairs Specifically, for each positive sample pair (i.e., a real association), several negative sample nodes are sampled. (Usually randomly selected from a set of nodes), and the sigmoid function is used. This strategy aims to approximately maximize the similarity of positive sample pairs and minimize the similarity of negative sample pairs. After applying this strategy, The calculation is approximated as: in, and These are nodes and The set of negative sample nodes generated by random sampling and It is a negative sample node.

[0054] S44. In actual training and evaluation, each... Instead of scalar values, it optimizes an overall objective function. To implicitly learn a representation vector that makes pairwise relationships reasonable. The objective function integrates information from all positive and negative sample pairs: In this formula, It is a set of positive sample node pairs, where each element This represents a pair of related nodes. It is the target node. It is its context node. It is a set of negative sample node pairs, where each element From the target node and a randomly sampled negative sample node Composition. Symbols This represents the number of positive sample pairs. By minimizing... The model parameters were adjusted to increase the dot product value of positive sample pairs and decrease the dot product value of negative sample pairs, thereby indirectly increasing the dot product value of node pairs with true correlations. Achieving higher implicit similarity .

[0055] S45. Within the overall training framework of the model, a pairwise relation component needs to be prepared for the calculation of the joint loss function. This process involves traversing a large number of node pairs sampled from the training data. For each sampled positive node pair... Perform calculations S43 and S44: that is, obtain the node representation vector. and For each positive sample pair, generate several negative samples, calculate their dot product, apply the sigmoid function, and finally substitute the result into... In the cumulative formula, by traversing all sampled node pairs and completing the above calculation, the model learning process of "obtaining the pairwise similarity between every two nodes" is achieved, and the result is reflected in the node representation vector learned by the model. In this way, closely related nodes are placed close to each other in the vector space. These representation vectors and the computed... The loss value will be used in step S5 for joint optimization with the tuple similarity model.

[0056] S5. Based on the similarity scores of all tuples and all pairwise similarities, construct a joint loss function, and optimize the parameters of the target model based on the joint loss function; The parameters of the target model refer to the set of all weights and bias variables in the HyperAtt neural network model that need to be determined through data learning after training. These parameters are not fixed; their initial values ​​are usually randomly set and continuously adjusted during training using a gradient descent algorithm guided by the joint loss function, so that the model can effectively capture pairwise and higher-order tuple relationships between phenotypic trait nodes from heterogeneous crop hypernetwork data. Specifically, the parameters of the target model include, but are not limited to: the initial embedding matrix. All elements in; the independent query projection matrix for each attention head in a multi-head attention layer. Key projection matrix Value projection matrix and the final output projection matrix Weight matrix in feedforward network layer , and bias vector , Learnable scaling factor in layer normalization operation and bias ; and the weights of the fully connected layers used to calculate the tuple similarity score. and bias Optimizing these parameters is the essence of model learning.

[0057] The specific implementation process of S5 is as follows: S50, Joint Loss Function It is the overall goal that integrates the two tasks of tuple relation optimization and pairwise relation optimization. It is constructed as the objective function of the tuple similarity model. Objective function of pairwise similarity model A weighted linear combination of . Its mathematical expression is defined as: In this formula, This represents the joint loss function, which is the overall objective that needs to be minimized. The calculation depends on the similarity score of all tuples. and The formula is . The calculation relies on the dot product and sigmoid function calculated based on the node representation vectors, as shown in the formula: .symbol It is a non-negative balancing hyperparameter used to adjust... and The relative importance of the total loss. The value is usually determined on the validation set through cross-validation.

[0058] S51. In each training iteration, a small batch of data is sampled from the training set. This small batch contains two parts of data: one part is from the set of positive tuples. A batch of positive units obtained from sampling And for each positive tuple, a corresponding negative tuple is constructed according to a preset negative sampling ratio. This forms the batch of tuple data; the other part is from the set of positive nodes. A batch of positive node pairs obtained from sampling and for each target node Construct corresponding negative sample node pairs according to the preset negative sampling number. This forms the node pair data for that batch. These two parts of data together constitute a training batch, used for loss calculation in the current iteration.

[0059] S52. Input the tuple data and node pair data of the current batch into the target model under the current parameter state. For each tuple... and The model performs a complete S3 forward propagation and calculates the corresponding... and Then, the current batch is calculated according to the formula. Value. For each node pair and Using the node representation vectors already generated by the model Calculate the dot product and And through the sigmoid function After conversion, the current batch is calculated according to the formula. The value. Finally, the calculated value. and Substitute into the joint loss function formula This yields the joint loss value for the current training batch. .

[0060] S53, after calculating the combined loss of the current batch Then, the backpropagation algorithm is initiated. This algorithm, based on the chain rule, automatically calculates the joint loss. The gradient is the partial derivative with respect to each trainable parameter in the target model. Specifically, the computation graph starts from the final joint loss. Begin by tracing back the process sequentially. and This process traces back to tuple similarity score calculation, pairwise similarity calculation, layer normalization, feedforward network, multi-head attention mechanism, and finally back to the initial node embedding lookup. This process calculates the initial embedding matrix. Gradients of relevant rows and all attention head parameters gradient, feedforward network parameters gradient, layer normalization parameters gradient and scoring layer parameters The gradients indicate how each parameter should be adjusted to reduce the joint loss. .

[0061] S54. After obtaining the gradients of all parameters, an optimization algorithm is used to update the parameters of the target model. An adaptive moment estimator optimizer, such as the Adam optimizer, is typically used. The optimizer calculates an update step size based on the first-order and second-order moment estimates of the gradient of each parameter. For any parameter in the target model... (It can represent) One of the elements, or One of the weights), whose update rules follow the following form: .here, It is the current value of the parameter. This is the updated value. It is the global learning rate, which is an important hyperparameter. and The Adam optimizer is used as a parameter. The maintained, bias-corrected estimates of the first and second moments of the gradient. This is a minimal constant used for numerical stability. Through this step, all parameters of the target model are aligned towards reducing the joint loss. The direction was slightly adjusted.

[0062] S55. Repeat steps S51 to S54, i.e., "prepare batch data -> forward propagation to calculate loss -> backpropagation to calculate gradient -> optimizer to update parameters," constituting a complete training iteration. Perform multiple epochs of this iteration across the entire training set. After each epoch, model performance can be evaluated on an independent validation set, for example, calculating the AUC value for a link prediction task. When the joint loss... The training process ends when the performance on the validation set no longer decreases significantly, or when the validation performance stabilizes. At this point, the parameters of the target model have been sufficiently optimized by the joint loss function, achieving a good balance between capturing higher-order tuple relations and pairwise relations. The model has thus evolved from its initial state into an optimized state that can be used for final prediction.

[0063] S6. Using the optimized target model, generate the final representation vectors of nodes in the heterogeneous supernetwork of crops. Specifically: S60. Since the target model is designed with tuples as input units, to obtain the final representation vector of a single node, the node needs to be placed in a specific tuple context for forward propagation. Therefore, for each node in the network... This requires selecting one or more tuples containing that node. The most direct and reliable strategy is: for each node... Select a real sample from the training set that contains... Given a crop variety superedge, form a tuple by connecting all nodes connected to that superedge. For example, for the phenotypic trait node "thousand-grain weight", we select a hyperedge corresponding to a real rice variety possessing this trait (such as "Nanjing 46"), and construct a tuple containing all trait nodes within this hyperedge. This ensures that the node is processed by the model within a real and semantically complete high-order relation. Assume the node... Existence Among the different real hyperedges, one can usually be chosen as the most representative or the first one to appear to construct the tuple. .

[0064] S61. Load the optimized target model, saved after the training phase, into memory. Set the model to evaluation mode. In evaluation mode, all model parameters, including the initial embedding matrix, are evaluated. The projection matrices of the multi-head attention layer, the weights and biases of the feedforward network layers, and the scaling and bias parameters for layer normalization are all frozen and will no longer be updated. Simultaneously, any stochastic operations in the model, such as Dropout (if previously used), are disabled to ensure deterministic output. Next, for each node... Prepare the corresponding tuples This forms a list of tuples that need to be processed.

[0065] S62. Iterate through the prepared list of tuples. For each tuple... Perform the following deterministic forward propagation computation, specifically, based on the index of each node in the tuple, from the model's already optimized initial embedding matrix. Extract the corresponding initial vector to form the input matrix. ,Will The input model uses a multi-head attention layer. This layer uses optimized parameters. Calculations are performed to obtain the interactive feature matrix. .Will Input with optimization parameters The feedforward network layer yields the output. .calculate Subsequently, regarding Each row (i.e., the middle vector of each node) Application-layer normalization operation. This operation uses optimized parameters. and The calculation formula is: , It is a vector The mean, It is its variance. It is a very small constant that prevents division by zero. (Symbol) This indicates element-wise multiplication. After this step, the normalized node representation vector matrix corresponding to the tuple is obtained. .

[0066] S63, For the currently being processed node The constructed tuple The normalized node representation matrix obtained in S62 In the middle, node The corresponding representation vector is located in a specific row. This row is found through the mapping relationship, and then... The dimensional vector is extracted and denoted as... This vector It is a node The final representation vector. It combines the node's initial information, its attentional interactions with other nodes within a specific tuple, and deep features after nonlinear transformation and stabilization by the feedforward network.

[0067] S64. Repeat S62 and S63, iterating through each phenotypic node. (in from arrive tuples constructed by ) For each node Generate its corresponding final representation vector In this process, each node only needs to perform one forward propagation computation through the model. Finally, the final representation vectors of all nodes are arranged in node index order and assembled into a final node representation matrix. The first of this matrix A row is a node The final representation vector .

[0068] S65. Generate the final node representation matrix. The data is persistently saved to disk in file format (such as a .npy file containing NumPy arrays or in text format). Simultaneously, this matrix is ​​also retained in memory as directly usable data, input into the subsequent step of "predicting crop varieties based on the final representation vectors." This completes the entire process of generating final representation vectors with rich structural semantics and domain knowledge for all phenotypic trait nodes in the entire heterogeneous crop supernetwork using the parameter-optimized target model. These vectors encode the higher-order law of "which traits tend to co-occur to form superior varieties," and are key data assets for intelligent breeding prediction.

[0069] S7. Predict crop varieties based on the final representation vector. The specific implementation process is as follows: S70. The core of variety prediction is to assess whether a virtual variety composed of a specific combination of phenotypic traits has superior potential. Therefore, it is first necessary to define the combination of traits to be evaluated. This combination is represented as a target tuple. ,in arrive From the set of nodes Selected The identifier, symbol of each phenotypic trait node. This represents the number of traits contained in the candidate variety. These trait combinations may be proposed by breeding experts based on experience, or generated through a search strategy (such as a genetic algorithm). Each target tuple This corresponds to a virtual crop variety to be predicted.

[0070] S71. For each target tuple to be predicted It is necessary to obtain each node. The final representation vectors. These vectors are directly derived from the generated and stored final node representation matrix. In this context, the node's unique index is used for lookup. Assuming... In the matrix The corresponding row index is Then its final representation vector is . All of the target tuple The final representation vectors of the nodes are stacked sequentially to form an input matrix. This matrix It will serve as the basic input for prediction calculations.

[0071] S72, A function is needed. , input matrix Mapped to a scalar prediction value This is used to represent the superior potential of the candidate variety. The design of the prediction function should fully utilize the high-order relational information encoded in the final representation vector. An effective approach is to reuse key parts of the tuple similarity scoring calculation path in a pre-trained HyperAtt neural network model, excluding parameters that need to be retrained. Specifically, a lightweight neural network layer can be defined or metric learning can be used directly. For example, a weighted aggregation layer based on attention weights and a fully connected layer can be employed: in, , It is a learnable query vector (or, in simple scenarios, a mean vector). Then, the aggregated vector... The prediction result is obtained by using a fully connected layer and a sigmoid activation function: here, and These are the weights and bias parameters of the prediction layer. It's the sigmoid function. Parameters and It can be obtained by fine-tuning on a small amount of historical variety data that has been labeled as excellent or good, or by directly initializing and fixing the parameters of the last layer of the HyperAtt neural network model with similar parameters.

[0072] S73. When multiple candidate varieties need to be evaluated, construct a batch processing tensor. ,in This refers to the batch size; each row represents a candidate variety. Input matrix. Convert the entire tensor. Input to prediction function The prediction function needs to support batch processing, i.e., for... Each sample in the calculation is performed independently. and The final output is a prediction result vector. , of which element That is the first Potential score of each candidate variety.

[0073] S74. Prediction Results It is a value between 0 and 1. A threshold needs to be set according to the application scenario. (For example For each candidate variety, if If the candidate variety meets the criteria, it is classified as a "variety with excellent potential"; otherwise, it is classified as "variety with insufficient potential". Furthermore, all candidate varieties can be classified according to... The values ​​are sorted in descending order to generate a ranking list of variety potential, providing breeding experts with a priority testing order. To enhance interpretability, the attention weights calculated in the prediction function can be further analyzed. Weight The size indicates the corresponding phenotypic trait. The relative importance of this variety combination provides a basis for understanding model decision-making.

[0074] S75. Compare the prediction results with subsequent actual field trial data (if available), and calculate metrics such as prediction accuracy, precision, recall, F1 score, or root mean square error to evaluate the effectiveness of the prediction model. Based on the validation results, it may be necessary to adjust the prediction function. Structure, adjusting threshold Or even re-examine and adjust the HyperAtt neural network model training process to form a continuously improving closed loop. Ultimately, the optimized prediction process can be integrated into the breeding decision support system to achieve data-driven prediction of new variety performance.

[0075] In this context, the prediction result refers to the quantitative output regarding the potential of a new variety, derived through a specific calculation process based on the model's final representation vector. This output is typically a continuous numerical value or a classification label, used to guide breeding decisions. In variety prediction tasks based on the final representation vector, the prediction result can be specifically represented as a comprehensive score, a probability value, or a ranking list, indicating the likelihood that a virtual variety composed of a combination of phenotypic traits will become a superior variety. The prediction result is the endpoint of the entire technical solution, providing breeding experts with data-driven decision-making support.

[0076] This invention relates to the following technical content: (1) In the field of crop breeding, in order to model the complex high-order relationships between varieties and traits, the mathematical model of heterogeneous hypernetworks for crops is introduced. It abstracts breeding data into a hypergraph. .in, yes A set of nodes representing various phenotypic traits. Representing the A set of nodes for each type (e.g., yield traits, quality traits, stress resistance traits). It is the size of the node set. It is a set of superedges, where It is the size of the set of hyperedges. In this invention, each hyperedge... A set of nodes representing a specific crop variety. That is, the characteristics possessed by this variety. A phenotypic trait. If for any All Then it is called for -Uniform hypernet. If Then the hypernetwork degenerates into a traditional network that only describes pairwise relationships. If the number of node types... Then, a hypernetwork is defined as a heterogeneous hypernetwork, which accurately reflects the diversity of crop traits. Figure 2 A heterogeneous supernetwork is shown, in which, , These represent three different node types, and each hyperedge connects multiple nodes of different types.

[0077] (2) Multi-head attention is a powerful feature extraction and interaction tool that initially performed well in tasks involving sequential data. In this invention, it is innovatively applied to the tuple structure in heterogeneous supernetworks for crops. The core principle of this mechanism is to capture different types of relationship patterns between input elements in parallel using multiple independent attention subspaces. Specifically, it computes the attention distribution in parallel using multiple independent weight matrices, each set of weights being called an "attention head." Each attention head learns the association patterns between input elements in an independent subspace, enabling the model to simultaneously identify and integrate various complex relationship features, such as direct associations, complementary effects, or role differences between phenotypic nodes within a tuple. This parallel learning strategy greatly enhances the model's ability to model high-order heterogeneous relationships.

[0078] (3) To learn high-quality representation vectors of phenotypic trait nodes in heterogeneous supernetworks of crops, this invention proposes the HyperAtt neural network model. The overall design goal of this model is to simultaneously and fully capture both pairwise relationships and higher-order tuple relationships between nodes. For example... Figure 3 As shown, in the HyperAtt neural network model, the "multi-head attention mechanism" directly corresponds to the "multi-head attention layer," which is responsible for node feature interaction; the "feedforward network" corresponds to the "feedforward network layer," which is used for nonlinear transformation; "layer normalization" and "global average pooling" are conceptually consistent in both, respectively responsible for stabilizing numerical distribution and compressing node dimensions; and the scoring function in "tuple relation optimization," especially the step of generating scores through the sigmoid function, is implemented by the "fully connected layer containing activation functions" in the final scheme. In addition, the operation of "residually connecting the multi-head attention output and the feedforward output" corresponds to the "residual connection operation."

[0079] Figure 3 The HyperAtt neural network model design diagram clearly outlines the overall architecture of the model to learn high-quality node representation vectors through dual-path collaborative optimization. The core of this architecture lies in the parallel and organic integration of modeling pairwise relations and higher-order tuple relations. Specifically, the model input consists of two parts. The first is the initial vector representation provided to the nodes through "embedding layers (center word)" and "embedding layers (surrounding words)." These representations are used in the "pairwise relation optimization" path. In this path, the model calculates the "dot product" between node representation vectors to measure their similarity and optimizes the "non-tuple relations," i.e., pairwise associations, between nodes based on this. Simultaneously, the other core path of the model is "tuple relation optimization," which specifically handles "tuple relations" composed of multiple nodes (such as a1, a2, a3, b1, b3, C1, etc.). This path begins with a "higher-order tuple filter" specifically handling such higher-order structures, which first utilizes... The "multi-head attention mechanism" performs deep feature extraction and interaction on all nodes within a tuple, allowing the model to learn complex dependency patterns between nodes in parallel from different subspaces. Subsequently, the output of the "multi-head attention mechanism" is fed into a "feedforward network" for nonlinear transformation to enhance the model's expressive power. Afterward, the model performs "layer normalization" to stabilize the numerical distribution and ensure the quality of the node representation vectors. Next, "global average pooling" is applied to the normalized representations of all nodes within the tuple, compressing them into a pooled vector representing the overall semantics of the tuple. Finally, this pooled vector is used for "tuple relation optimization." The entire design, by jointly optimizing both "pairwise relation optimization" and "tuple relation optimization," drives the node representation vectors learned by the model to accurately capture subtle pairwise associations and deeply understand the high-order collaborative semantics of "multiple traits defining a variety as a whole," thus laying a solid foundation for the final crop variety prediction task.

[0080] The HyperAtt neural network model achieves comprehensive learning of the network structure through the collaborative work of two components: a tuple similarity model and a pairwise similarity model. Specifically, this includes: 1) The tuple similarity model is the core of the HyperAtt neural network model, responsible for modeling higher-order tuple relationships. It uses tuples... As input, where This represents the number of nodes contained in the tuple. The specific implementation process is as follows: ① The model starts from the node embedding matrix (i.e., the initial embedding matrix) Extract the representation vectors of the corresponding nodes from (or its updated version) to form the input matrix. ,in It is the embedded dimension.

[0081] ② Input matrix It is fed into a multi-head attention layer. This layer is used in parallel. The first point of attention. For the first Each attention head, its output The calculation formula is: in, , and They are the first Each attention head can be trained to generate query, key, and value projection matrices. and This refers to the dimension of each head, typically set to... . It is a scaling factor. The outputs of all attention heads are concatenated and then passed through a trainable projection matrix. By integrating these elements, we can obtain multi-head attention output. : ③ The data is fed into a feedforward network layer for nonlinear transformation to enhance the model's expressive power. This layer consists of two linear transformations and a ReLU activation function. in, , , , These are trainable parameters. It is the internal dimension of the feedforward network.

[0082] ④ To preserve original features and ensure stable training, the model performs residual connections and layer normalization operations: for Middle representative node vector Its layer normalized output for: in, and It is a vector The mean and variance, It is a very small constant. and These are learnable scaling factors and bias parameters. That is, a node The normalized node representation vector after processing by this layer.

[0083] ⑤ To generate tuple similarity scores, global average pooling is performed on the normalized representation vectors of all nodes within a tuple, compressing them into a single pooled vector: Then, the pooling vector is fed into a fully connected layer containing a sigmoid activation function to obtain a score: in, It is the sigmoid function. and These are the trainable weights and biases of the fully connected layer. An indicator function is introduced. The objective function of the final tuple similarity model is to distinguish effective tuples during training. The negative sampling strategy is defined as follows: in, and These are sets of positive and negative tuples, respectively.

[0084] 2) Pairwise similarity models are used as an auxiliary method to capture and optimize pairwise relationships between phenotypic trait nodes, thus compensating for the limitations of tuple similarity models in this regard. Specifically, they include: ① For any two nodes and Theoretically, their pairwise similarity is defined as the joint probability of their association with all possible context nodes: in, , The representation vector of a node.

[0085] ② To avoid calculating the normalized denominator To mitigate the significant overhead, a negative sampling strategy is introduced for efficient approximation. The sigmoid function is employed. For nodes and Construct its connection to the positive context node. and negative sampling node set , The target. The approximate logarithmic similarity is calculated as follows: ③ Based on the above approximation, the objective function of the overall similarity model is defined. : in, and These are the sets of positive and negative sample node pairs, respectively. This represents a positive sample node pair. This represents a negative sample node pair. By minimizing... The node representation vectors learned by the model-driven learning process It can accurately reflect the pairwise correlation strength between nodes.

[0086] To verify the potential effectiveness and generalization ability of the proposed hypernetwork representation learning method based on multi-head attention mechanism in crop breeding, a comprehensive experimental evaluation was conducted on several publicly available hypernetwork datasets. Although these datasets come from different fields, the high-order heterogeneous relationships they contain are structurally similar to those in heterogeneous crop hypernetworks (where nodes are phenotypic traits and hyperedges are varieties). By validating the model's ability to capture complex relationships on these datasets, the feasibility of applying this method to breeding data modeling can be indirectly demonstrated.

[0087] The experiment used four hypernetwork datasets from different domains: GPS network, drug network, MovieLens social network, and WordNet semantic network, to comprehensively evaluate the performance of the HyperAtt neural network model. Detailed statistics of the datasets are shown in Table 1.

[0088] Table 1: As shown in Table 1, these datasets exhibit diversity in node type, number of nodes, and number of hyperedges, enabling the testing of model performance under different scales and heterogeneity. Specifically, the GPS dataset records user trajectory information. Higher-order tuples (user, location, activity) are used as hyperedges to construct the hypernetwork. This "entity-location-behavior" ternary relationship is analogous to the multivariate interaction relationship of "variety-environment-phenotypic expression" in breeding. The drug dataset records adverse events and drug error report information. Higher-order tuples (user, drug, response) are used as hyperedges to construct the hypernetwork. This is similar to the complex correspondence in breeding where a "variety" (drug) triggers a series of "phenotypic responses" under specific conditions (user). The MovieLens dataset records user activity information. Higher-order tuples (user, movie, tag) are used as hyperedges to construct the hypernetwork. This collaborative relationship of "user-item-tag" is structurally similar to the association pattern of "breeder-variety-phenotypic evaluation." The WordNet dataset records semantic network information from WordNet 3.0, with higher-order tuples (head entities, relations, tail entities) treated as hyperedges to construct the hypernetwork. This structured knowledge representation is isomorphic to the logical network of "trait-genetic relationship-gene" in the field of breeding knowledge.

[0089] The HyperAtt neural network model is further compared with several advanced network representation learning methods to demonstrate its superiority. Specifically, the methods employed include: ① Deepwalk: This method applies a Skip-gram model to node sequences generated by random walks to learn low-dimensional embeddings of nodes in the network. It represents a class of traditional graph representation learning methods based on random walks. ② Node2Vec: This method obtains node representation vectors by performing biased random walks using depth-first and breadth-first strategies. It is an improvement on Deepwalk, capturing richer network structural features. ③ HPSG: This method uses a sequence of node nodes from hyperpath-based random walks as input to a Skip-gram model to learn node representation vectors. It is a random walk method specifically designed for hypernetworks. ④ Hyper2vec: This method captures relationships between nodes by performing biased second-order random walks on the hypernetwork, thereby learning node representation vectors. It further considers the higher-order properties of hypernetworks. ⑤ HyperS2V: This method learns node representation vectors by quantifying the structural similarity between nodes based on node hyperdegree. It utilizes the node degree information unique to hypernetworks. ⑥DHNE: This method learns node representation vectors using a neural network model incorporating a multilayer perceptron. It represents a class of deep learning-based hypernetwork representation learning methods.

[0090] Link prediction tasks are used to evaluate the ability of learned node representation vectors to predict unknown associations. In the context of crop breeding, this is similar to predicting whether a potential variety composed of a specific combination of phenotypic traits, which has not yet been observed, is likely to exist.

[0091] In the link prediction experiments, four hypernetic datasets, including GPS, drug, MovieLens, and WordNet, were selected as the research objects. 80% of the hyperedges were randomly selected as the training set, and the remaining 20% ​​as the test set. Ten independent experiments were conducted on all methods, and the average AUC (Area Under Curve) metric was calculated to eliminate the influence of randomness. A higher AUC value indicates a stronger ability of the model to predict missing links. Detailed experimental data are shown in Table 2. As shown in Table 2, the HyperAtt neural network model performs excellently on the drug and WordNet datasets, and also outperforms most baseline methods on the GPS and MovieLens datasets. This result is mainly due to the fact that the HyperAtt neural network model not only captures pairwise relationships between nodes through a pairwise similarity model, but also enhances the capture of higher-order tuple relationships through the multi-head attention mechanism in the tuple similarity model. Therefore, the HyperAtt neural network model not only has strong performance in capturing pairwise relationships between hypernetwork nodes, but can also be effectively applied to hypernetworks with complex high-order tuple relationships, which provides strong evidence for handling complex trait-variety high-order relationships in crop breeding.

[0092] Table 2: The hypernetwork reconstruction task is used to evaluate the quality of node representation vectors. Its goal is to ensure that the reconstructed hypernetwork from the learned representation vectors retains the structural information of the original hypernetwork to the greatest extent possible. In a breeding context, this is equivalent to verifying whether the trait representations learned by the model can infer the true varietal combinations. Hypernetwork reconstruction experiments were conducted on the GPS and drug datasets. The evaluation metric for hypernetwork reconstruction is the reconstruction accuracy ACC(ρ), which is calculated as follows: in, The number of hyperedges to be rebuilt according to the plan is relative to the total number of hyperedges in the original hypernetwork. The proportion. Represents the first reconstructed based on the model score The hyperedge does indeed exist in the original hypernetwork, otherwise This metric measures the accuracy with which the model recovers the true hyperedges at a given reconstruction scale. Experimental results for hypernetic network reconstruction are as follows: Figure 4 and Figure 5 As shown.

[0093] according to Figure 4 and Figure 5 As can be seen, the HyperAtt neural network model outperforms most baseline methods on both datasets, including traditional network representation learning methods like Deepwalk, node2vec, and the existing hypernetwork representation learning method HyperS2V. This advantage stems from the HyperAtt neural network model's use of a multi-head attention mechanism. This mechanism significantly improves the quality of the learned node representation vectors by effectively integrating the different features and interaction information of nodes within tuples, thus enabling a more accurate reconstruction of the original high-order network structure. This further confirms that the representation vectors generated by the method of this invention have high structural fidelity, laying a solid foundation for their use in crop breeding for precise trait characterization and variety prediction.

[0094] In this application, ① a multi-head attention mechanism is applied to the hypernetwork representation learning task. Specifically, a multi-head attention layer is innovatively introduced as a core feature interaction component in the tuple similarity model of the HyperAtt neural network model. This component works in parallel through multiple independent attention heads, each learning different types of relationship patterns between phenotypic trait nodes within a tuple in an independent subspace. This enables the model to simultaneously capture and integrate complex, heterogeneous, high-order dependencies between nodes, overcoming the shortcomings of existing methods in modeling the fine structure inside hyperedges, thus providing a more powerful feature extraction and representation capability for the complex breeding patterns contained in heterogeneous hypernetworks of crops. ② The output of the multi-head attention mechanism is residually connected to the output of the feedforward network to ensure a more comprehensive capture of the hypernetwork structure. In the model design, a residual connection operation is explicitly introduced, specifically in the form of connecting the output of the multi-head attention layer... With the output of the feedforward network layer Perform element-wise addition to obtain the fused output. This design effectively mitigates the gradient vanishing or network degradation problems that may occur in deep neural networks, ensuring that the structural information of the original supernetwork and the important interaction features learned in the multi-head attention stage can be losslessly transferred to deeper network layers. Combined with subsequent layer normalization operations, this strategy significantly improves the training stability, convergence speed, and quality of the finally learned node representation vectors, enabling the model to capture and retain the overall and local structural information of the supernetwork more comprehensively and deeply. ③ Addressing the key issue of modeling pairwise relations and higher-order tuple relations in supernetwork representation learning, a HyperAtt neural network model based on the multi-head attention mechanism is proposed. This model creatively utilizes the powerful feature extraction and interaction capabilities of multi-head attention to design a framework for joint learning of tuple similarity models and pairwise similarity models, effectively balancing the model's representation depth, robustness to data noise, and overall computational efficiency. Systematic experiments on four heterogeneous hypernetwork datasets—GPS, drug, MovieLens, and WordNet—show that the HyperAtt neural network model outperforms most baseline methods in both link prediction and hypernetwork reconstruction tasks. This result fully validates the effectiveness and advancement of the model design and indirectly proves that the method is suitable for modeling crop breeding data with similar structures.

[0095] In the above embodiments, although the steps are numbered S1, S2, etc., they are only specific embodiments given by the present invention. Those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation. The scheme after adjusting the order is also within the protection scope of the present invention. It can be understood that in some embodiments, some or all of the above embodiments may be included.

[0096] like Figure 6 As shown, an embodiment of the present invention provides a hypernetwork representation learning system 200 based on multi-head attention mechanism for breeding, which includes a first construction module 201, a second construction module 202, a scoring calculation module 203, a similarity calculation module 204, a parameter optimization module 205, a generation module 206, and a prediction module 207. The first construction module 201 is used to: construct a heterogeneous supernetwork of crops, wherein the nodes in the heterogeneous supernetwork of crops represent the phenotypic traits of crop varieties, the superedges in the heterogeneous supernetwork of crops represent crop varieties, and each superedge contains at least two nodes; The second construction module 202 is used to: construct the initial embedding matrix corresponding to all nodes; The scoring calculation module 203 is used to: for each tuple in the heterogeneous supernetwork of crops, based on the initial embedding matrix, process it using the target model to obtain the normalized node representation vector corresponding to each tuple and the similarity score of each tuple; the target model includes a multi-head attention layer, a feedforward network layer, a global average pooling layer and a fully connected layer containing an activation function, and has residual connection operation and layer normalization operation. The similarity calculation module 204 is used to: calculate the pairwise similarity between every two nodes in the crop heterogeneous supernetwork based on the normalized node representation vectors corresponding to all tuples; The parameter optimization module 205 is used to: construct a joint loss function based on the similarity scores of all tuples and all pairwise similarities, and optimize the parameters of the target model based on the joint loss function; The generation module 206 is used to: generate the final representation vector of nodes in the heterogeneous supernetwork of crops using the target model with optimized parameters; The prediction module 207 is used to predict crop varieties based on the final representation vector.

[0097] Optionally, in the above technical solution, the scoring calculation module 203 is specifically used for: based on the initial embedding matrix, using a multi-head attention mechanism to extract features and interact with nodes within any tuple to obtain a multi-head attention output; inputting the multi-head attention output into a feedforward network for nonlinear transformation to obtain a feedforward output; performing a residual connection between the multi-head attention output and the feedforward output to obtain a fused output; performing layer normalization processing on the fused output to obtain a normalized node representation vector corresponding to the tuple; inputting the normalized node representation vector into the global average pooling layer of the target model for compression to obtain a pooling vector; inputting the pooling vector into a fully connected layer containing an activation function to obtain a similarity score for the tuple; traversing each tuple to obtain a normalized node representation vector and a similarity score corresponding to each tuple.

[0098] Optionally, in the above technical solution, the similarity calculation module 204 is specifically used to: for any two nodes forming a node pair, calculate the conditional probability between the first node and the second node in the node pair based on the normalized node representation vector of the two nodes, calculate the pairwise similarity of the node pair based on the conditional probability between the first node and the second node in the node pair, and traverse each node pair to obtain the pairwise similarity between every two nodes.

[0099] Optionally, in the above technical solution, the activation function used in the fully connected layer containing the activation function is the sigmoid function.

[0100] It should be noted that the beneficial effects of the hypernetwork representation learning system 200 based on multi-head attention mechanism for breeding provided in the above embodiments are the same as the beneficial effects of the hypernetwork representation learning method based on multi-head attention mechanism for breeding provided above, and will not be repeated here. Furthermore, the system provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the system can be divided into different functional modules according to the actual situation to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, and will not be repeated here.

[0101] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned hypernetwork representation learning methods based on multi-head attention mechanism applied to breeding.

[0102] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-mentioned hypernetwork representation learning methods based on multi-head attention mechanisms applied to breeding.

[0103] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0104] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A hypernetwork representation learning method based on multi-head attention mechanism for application in breeding, characterized in that, include: Construct a heterogeneous supernetwork for crops, wherein nodes in the heterogeneous supernetwork represent phenotypic traits of crop varieties, and superedges in the heterogeneous supernetwork represent crop varieties, and each superedge contains at least two nodes; Construct the initial embedding matrix for all nodes; For each tuple in the crop heterogeneous supernetwork, based on the initial embedding matrix, the target model is used to process it to obtain the normalized node representation vector corresponding to each tuple and the similarity score of each tuple; the target model includes a multi-head attention layer, a feedforward network layer, a global average pooling layer and a fully connected layer containing an activation function, and has residual connection operation and layer normalization operation. Based on the normalized node representation vectors corresponding to all tuples, calculate the pairwise similarity between every two nodes in the crop heterogeneous supernetwork. Based on the similarity scores of all tuples and all pairwise similarities, a joint loss function is constructed, and the parameters of the target model are optimized based on the joint loss function. The final representation vectors of nodes in the crop heterogeneous supernetwork are generated using the target model with optimized parameters. Crop variety prediction is performed based on the final representation vector.

2. The hypernetwork representation learning method based on multi-head attention mechanism applied to breeding, as described in claim 1, is characterized in that... The process of obtaining the normalized node representation vector for each tuple and the similarity score for each tuple includes: Based on the initial embedding matrix, a multi-head attention mechanism is used to extract features and interact with nodes within any tuple to obtain a multi-head attention output. This multi-head attention output is then input into a feedforward network for nonlinear transformation to obtain a feedforward output. The multi-head attention output and the feedforward output are then joined via residual connections to obtain a fused output. The fused output is then subjected to layer normalization to obtain a normalized node representation vector corresponding to the tuple. This normalized node representation vector is then input into the global average pooling layer of the target model for compression to obtain a pooling vector. Finally, the pooling vector is input into the fully connected layer containing the activation function to obtain a similarity score for the tuple. This process is repeated for each tuple to obtain a normalized node representation vector and a similarity score for each tuple.

3. A hypernetwork representation learning method based on multi-head attention mechanism applied to breeding, as described in claim 1 or 2, characterized in that, The pairwise similarity between each pair of nodes is obtained, including: For any pair of nodes, calculate the conditional probability between the first and second nodes in the pair based on the normalized node representation vectors of the two nodes. Calculate the pairwise similarity of the pair based on the conditional probability between the first and second nodes in the pair. Iterate through each pair of nodes to obtain the pairwise similarity between every two nodes.

4. A hypernetwork representation learning method based on multi-head attention mechanism applied to breeding, as described in claim 1 or 2, characterized in that, The fully connected layer containing the activation function uses the sigmoid function.

5. A hypernetwork representation learning system based on a multi-head attention mechanism for application in breeding, characterized in that, It includes a first building module, a second building module, a scoring calculation module, a similarity calculation module, a parameter optimization module, a generation module, and a prediction module; The first construction module is used to: construct a heterogeneous supernetwork of crops, wherein the nodes in the heterogeneous supernetwork of crops represent phenotypic traits of crop varieties, the hyperedges in the heterogeneous supernetwork of crops represent crop varieties, and each hyperedge contains at least two nodes; The second building module is used to: construct the initial embedding matrix corresponding to all nodes; The scoring calculation module is used to: for each tuple in the crop heterogeneous supernetwork, based on the initial embedding matrix, process it using the target model to obtain the normalized node representation vector corresponding to each tuple and the similarity score of each tuple; the target model includes a multi-head attention layer, a feedforward network layer, a global average pooling layer and a fully connected layer containing an activation function, and has residual connection operation and layer normalization operation. The similarity calculation module is used to: calculate the pairwise similarity between every two nodes in the crop heterogeneous supernetwork based on the normalized node representation vectors corresponding to all tuples; The parameter optimization module is used to: construct a joint loss function based on the similarity scores of all tuples and all pairwise similarities, and optimize the parameters of the target model based on the joint loss function; The generation module is used to: generate the final representation vector of the nodes in the crop heterogeneous supernetwork using the parameter-optimized target model; The prediction module is used to predict crop varieties based on the final representation vector.

6. A hypernetwork representation learning system based on multi-head attention mechanism for breeding, as described in claim 5, is characterized in that... The scoring calculation module is specifically used to: based on the initial embedding matrix, use a multi-head attention mechanism to extract features and interact with nodes in any tuple to obtain multi-head attention output; The multi-head attention output is input into the feedforward network and subjected to a nonlinear transformation to obtain the feedforward output; The multi-head attention output and the feedforward output are residually concatenated to obtain a fused output. The fused output is then subjected to layer normalization to obtain a normalized node representation vector corresponding to the tuple. This normalized node representation vector is input into the global average pooling layer of the target model for compression to obtain a pooling vector. The pooling vector is then input into the fully connected layer containing the activation function to obtain a similarity score for the tuple. This process is repeated for each tuple to obtain a normalized node representation vector and a similarity score for each tuple.

7. A hypernetwork representation learning system based on a multi-head attention mechanism for breeding, as described in claim 5 or 6, characterized in that, The similarity calculation module is specifically used for: for any two nodes forming a node pair, calculating the conditional probability between the first and second nodes in the node pair based on the normalized node representation vectors of the two nodes, calculating the pairwise similarity of the node pair based on the conditional probability between the first and second nodes in the node pair, and traversing each node pair to obtain the pairwise similarity between every two nodes.

8. A hypernetwork representation learning system based on a multi-head attention mechanism for breeding, as described in claim 5 or 6, characterized in that, The fully connected layer containing the activation function uses the sigmoid function.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the hypernetwork representation learning method based on a multi-head attention mechanism for breeding as described in any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the hypernetwork representation learning method based on a multi-head attention mechanism for breeding as described in any one of claims 1 to 4.

Images