Super-computing interconnection resource coding method based on resource hierarchical relationship description

Abstract

The invention provides a supercomputing interconnection resource coding method based on resource hierarchical relationship description. The supercomputing interconnection resource coding method comprises the following steps: feeding resource records, corresponding codes and hierarchical codes into a shared encoder; obtaining a representation specific to a supercomputing resource code based on a perceptual attention mechanism; resource coding records are extracted through hierarchical relation description, resource information closely related to given codes is effectively screened, and effective coding of supercomputing resource related information is achieved by designing a hierarchical coding description distillation mode. According to the invention, the classification precision of resource automatic coding is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of automatic coding of resources in supercomputing Internet, and in particular to a supercomputing Internet resource coding method based on resource hierarchical relationship description. Background Art

[0002] With the development of technologies such as 5G, the Internet of Things, and artificial intelligence, the amount of data has exploded, and the demand for computing power has also increased. According to the "Computing 2030" released by Huawei, by 2030, mankind will enter the era of yottabyte (YB) data, and general computing power will be 10 times the current computing power. In this context, the construction of computing power networks and supercomputing Internet has become an important driving force for the development of the digital economy. Supercomputing Internet aggregates the software and hardware resources of multiple supercomputing centers by connecting supercomputing centers in various places. Resource coding can achieve unified management and scheduling of these heterogeneous resources. In order to achieve high-speed interconnection between computing centers in various places, it is necessary to build a high-speed, low-latency, and stable network support environment. However, the manual nature of resource coding has brought major challenges. First, the supercomputing Internet needs to realize the description of computing power resources across heterogeneous and multi-centers. Manual coding is difficult to handle such complex scenarios, and may lead to the lack of unified standards and specifications for resource description schemes, making it difficult to achieve interconnection and resource sharing between different supercomputing centers. Automated coding can promote resource integration and management by following unified standards and specifications, providing a more effective and reliable solution. Therefore, there is a growing interest in automated resource encoding in both industry and academia.

[0003] Existing research treats the resource automatic encoding task as a multi-label text classification task, and various existing machine learning-based methods, including rule-based, support vector machine-based, and decision tree-based methods. With the emergence of deep learning in NLP tasks, people pay special attention to the deep representation of resource encoding strategies, which is achieved through recurrent neural networks, convolutional neural networks, or Transformer encoders. There are also attention mechanisms that determine representative words in resource encoding strategies, and then use words with higher weights as important evidence for making predictions.

[0004] Although these methods are effective, they do not effectively address the textual noise problem in resource encoding strategies, i.e., resource descriptions are long and noisy, with only a few key phrases highly relevant to the encoding task. Training models with such noisy text may lead to confusion of attention and incorrect decisions due to semantic bias.

[0005] With the popularity of neural networks, many researchers have focused on using RNN, CNN, and Transformer models for resource encoding. Relevant information for each encoding is extracted from the source resource record using convolutional layers with different kernel sizes. In order to further improve performance, some work has explored combining encoding descriptions with external knowledge. For example, a teacher network composed of encoding descriptions and modeling encoding co-occurrences is trained using interactive shared attention. There are also methods that use encoding synonyms in the supercomputing Internet knowledge graph and integrate external Wikipedia knowledge to implement supercomputing Internet resource-driven attention mechanisms. However, they all ignore the encoding hierarchy because the parent-child encodings are very similar on distributed high-performance computing, which can easily lead to incorrect encoding assignments.

[0006] In terms of encoding hierarchy, existing works have proposed hyperbolic representation methods that utilize encoding hierarchy. Other works have introduced a hierarchical joint learning architecture that considers the hierarchical relationship between encodings. However, these methods only use the encoding hierarchy at the output layer to avoid inappropriate encoding allocation, which is not enough to capture specific domain keywords. Summary of the invention

[0007] The purpose of the present invention is to provide a supercomputing interconnected resource encoding method based on resource hierarchical relationship description, and to provide a unified semantic representation of the data, storage, software and hardware resources of the supercomputing system. In order to solve the problem of lengthy text and high noise in the encoding process, a supercomputing resource encoding representation method based on encoding description perception is designed by converting the supercomputing resource into a compact low-dimensional word embedding, which effectively screens resource information closely related to a given encoding, and realizes the encoding of supercomputing resource related information by designing a hierarchical encoding description distillation method. The adopted technical scheme is as follows:

[0008] A supercomputing interconnection resource encoding method based on resource hierarchical relationship description comprises the following steps:

[0009] Step 1: Encode the resource description text, the corresponding coding description and the hierarchical coding description into a vector;

[0010] Step 2: Calculate the attention representation V of the resource description text based on the attention mechanism driven by the encoding description d , corresponding to the coding description representation Hierarchical coding description representation

[0011] Step 3: Describe the representation based on the corresponding code and hierarchical coding description representation Distillation, obtain the optimal attention representation V′ of resource description text d ;

[0012] Step 4: Input the optimal attention representation of the resource description text into the encoding description driven classifier to predict the resource encoding.

[0013] Preferably, in step 2, the attention representation V of the resource description text is calculated d The specific process includes:

[0014]

[0015]

[0016] α C =Attention(H C W Q , H d , H d )

[0017] V d =H d α C

[0018] in, - an encoded representation generated based on the hidden layer representation of each resource encoding;

[0019] - Each resource code Description;

[0020] c i - the i-th resource code;

[0021] N l -The number of resource codes;

[0022] H C -Hidden layer representations of all resource encodings;

[0023] W Q -Linear projection matrix with respect to the Query matrix;

[0024] α C -Attention weight matrix.

[0025] Preferably, in step 4:

[0026]

[0027] in - score vector for all resource encodings;

[0028] W l - Linear transformation matrix of resource encoding in the classifier;

[0029] V′ d-Optimal attention representation of resource description text;

[0030] W - weight matrix for direct classification in the classifier;

[0031] Preferably, in step 3, based on the corresponding coding description representation Distillation involves the following steps:

[0032] The attention representation in the resource description text is combined with the corresponding encoded description representation The distance is reduced;

[0033] Use cosine distance to measure and V d The similarities between.

[0034] Preferably, in step 3, the hierarchical coding description representation The loss of distillation is the resource description text representation V d and hierarchical coding description The dissimilarity between them.

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] The resource is encoded by the corresponding encoding and the hierarchical encoding description. Specifically, the resource record, the corresponding encoding and the hierarchical encoding are fed into the shared encoder.

[0037] The present invention uses a coding description-aware attention mechanism to obtain coding-specific representations, then applies corresponding coding and hierarchical coding to extract resource records, and finally uses the coding description-driven classifier output for classification, effectively improving the classification accuracy of resource automatic coding. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the AHDD model;

[0039] Figure 2 Experimental results and comparative experiments with other classification methods;

[0040] Figure 3 The experimental results are based on different text lengths;

[0041] Figure 4 The experimental results of ablation learning are shown in Figure 2. DETAILED DESCRIPTION

[0042] The following is a more detailed description of the supercomputing interconnection resource encoding method based on resource hierarchical relationship description of the present invention in conjunction with a schematic diagram, wherein a preferred embodiment of the present invention is shown, and it should be understood that those skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art, and not as a limitation of the present invention.

[0043] This embodiment explores the resource automatic encoding task and proposes an effective resource encoding prediction model (AHDD) based on corresponding encoding and hierarchical encoding description.

[0044] like Figure 1 As shown in the figure, the AHDD model structure and algorithm process diagram, the model includes four parts: encoding text, obtaining attention representation, corresponding encoding and hierarchical encoding distillation, and encoding description driven classification.

[0045] First, the resource record (resource description text), the corresponding coding description and the hierarchical coding description are encoded into vectors;

[0046] Then, the corresponding resource description text attention representation is calculated based on the attention mechanism driven by the encoded description;

[0047] Extract noisy resource records through the description of corresponding codes and their hierarchical codes;

[0048] Finally, the input is sent to the encoding description driven classifier to predict the resource encoding.

[0049] The specific algorithm is as follows:

[0050] Step 1: Encode resource description text, corresponding encoding description and hierarchical encoding description.

[0051] Taking the encoding resource description text as an example, step 1 is explained.

[0052] Step 1 requires converting each word in the resource description text into a compact, low-dimensional word embedding.

[0053] As far as the encoder is concerned, in theory, it can be implemented using any type of neural encoder, such as a CNN-based encoder, an RNN-based encoder, or a Transformer-based encoder.

[0054] In order to more clearly illustrate the specific implementation method of the present invention, the following definitions are made:

[0055] Resource description text and encoding description definition: given an N d words of resource description text The corresponding total is N L Resource code Each resource code i There is a brief description

[0056] Word embedding and hidden layer representation definition: The corresponding resource description text d will become a word embedding vector The encoder then transforms it into a hidden layer representation The corresponding formula is as follows:

[0057]

[0058] Among them, Enc represents the encoder

[0059] like Figure 1 As shown, the "corresponding code" refers to the associated node, that is, codes 1 to 3.

[0060] "Hierarchical coding" refers to sibling nodes, i.e. codes 4 to 6.

[0061] Step 2: Get the attention representation V of the resource description text d , corresponding to the coding description representation Hierarchical coding description representation

[0062] In this embodiment, the attention representation V of the resource description text is obtained d Take 2000 as an example to explain step 2.

[0063] The hidden layer representation generated in step 1 is encoded into the description-driven attention mechanism to obtain the corresponding attention representation.

[0064] In resource description text, not every word is equally helpful for the resource encoding task. Therefore, attention weights are used to refine the resource description text representation and align it with the encoded description representation.

[0065] Most previous works obtain the corresponding attention representation through a randomly initialized attention mechanism. This process consists of two steps, as follows:

[0066] First, calculate the attention weight matrix α according to the encoding, and the calculation formula is as follows:

[0067] Attention(Q,K,V)=softmax(QK T )V (2)

[0068] α=Attention(U,H d , H d )(3)

[0069] This includes learning parameters related to encoding attention as attention weights. However, due to limited training data, the weight matrix for rare encodings may not be learned effectively.

[0070] In order to solve this problem, the present invention calculates each resource code l i The hidden layer representation of the description is then followed by a max pooling operation to generate the encoded representation, which is then concatenated to form H c .

[0071] This matrix H c After a linear projection to replace U, the only parameter that needs to be learned in this module is the linear projection matrix W Q ∈R h×h , the corresponding calculation formula is as follows:

[0072]

[0073] At this time, formula (3) is modified as follows:

[0074] α C =Attention(HCW Q M, H d , H d ) (6)

[0075] Among them, the linear projection matrix W Q The acquisition process is as follows:

[0076] Initial W Q It is initialized with a random Gaussian and then continuously optimized during the training process.

[0077] The second step is to apply the matrix α C To calculate H d The attention weighted sum of , creates an encoding-specific document representation, namely, the attention representation V of the resource description text d :

[0078]

[0079] Among them, H in formula (7) d , using formula (1) to obtain H d .

[0080] Among them, use and calculate V d The same encoder and attention layer obtain the corresponding encoding description representation and

[0081] Step 3: Corresponding encoding description distillation and hierarchical encoding description distillation are performed to obtain the optimal attention representation of the resource description text.

[0082] In order to solve the problem of lengthy text and high noise in resource coding tasks, the present invention uses the description of the corresponding code to extract resource records, and effectively screens and removes irrelevant words in the resource records.

[0083] Furthermore, to ensure that resource terms that are particularly relevant to a given encoding and distinct from similar hierarchical encodings are captured, the present invention incorporates descriptions of these hierarchical encodings into the distillation.

[0084] This distillation is crucial in removing words that are more closely related to the hierarchical encoding rather than the corresponding encoding.

[0085] In step 3, the following sub-steps are further included:

[0086] a) Corresponding coding description distillation.

[0087] The purpose of corresponding code description distillation is to obtain information related to the corresponding code by extracting keywords from the resource description text.

[0088] The present invention reduces the distance between the attention representation in the resource description text and the representation of the corresponding encoded description, and uses the latter as training data for classification.

[0089] The present invention uses cosine distance to measure and V d The similarity between them is to minimize this distance. The specific calculation method is as follows:

[0090]

[0091]

[0092] in, is an encoding-specific representation of the resource description text, c A is the description of the corresponding encoding, is the representation of the corresponding encoding description.

[0093] and V d The similarity loss between them.

[0094] In formula (8), Calculated by formula (7), the output V of formula (7) is d ,for A collection of .

[0095] Similarly, Also calculated through steps 1 and 2.

[0096] The loss of the corresponding encoding description distillation includes two parts: and V d, and the binary cross entropy loss using the corresponding encoded descriptions as training data.

[0097]

[0098] L ades =λ sim ·L sim +Loss(c A ,y) (11)

[0099] Among them, y is the target code, The corresponding encoding description c is input A predictive coding.

[0100] L ades - corresponds to the coding description of the distillation loss; ades- corresponds to the label description;

[0101] sim-similarity;

[0102] λ sim - represents the weight of the corresponding encoding description similarity loss, which is a manually set hyperparameter.

[0103] b) Hierarchical coding description distillation

[0104] Resource codes are organized in a tree-like hierarchy, reflecting parent-child and sibling relationships. High-level nodes represent broader resource categories, while lower-level nodes detail specific resources. This hierarchy helps identify mutually exclusive codes. For example, if codes X and Y are sibling codes under Z, it is impossible to assign them to a user at the same time.

[0105] To effectively exploit this hierarchical structure and ensure that resource terms associated with a specific encoding are accurately captured and distinguished from terms associated with similar sibling encodings, the present invention utilizes distillation of hierarchical encoding descriptions.

[0106] The present invention distinguishes a specific representation in a supercomputing Internet resource from its sibling encoding related description. The loss of the hierarchical encoding description distillation is the resource description text representation V d With brother encoding description representation The dissimilarity between (hierarchical coding description representations) is sought to maximize the dissimilarity.

[0107] V d and The dissimilarity between them is similar to the similarity measure mentioned above, and the specific calculation method is as follows:

[0108]

[0109]

[0110] L sdes =λ dis ·L dis (14)

[0111] where c S Represents a hierarchical coding description, It is a representation of a hierarchical coding description.

[0112] Step 4: Encode the description driver classification.

[0113] The final resource encoding assignment is determined by evaluating the similarity between the encoding and the resource description text representation.

[0114] The previous approach is to convert the encoding-specific representation V d Input linear layer, then perform sum pooling operation to generate score vectors for all resource encodings Then, the probability vector is derived using the sigmoid function The specific calculation formula is as follows:

[0115]

[0116] Among them, it is necessary to learn the classification matrix related to the encoding to classify.

[0117] However, learning the weight matrix W can be challenging due to the imbalanced data distribution.

[0118] To solve this problem, the present invention combines the similarity between the coding description and the supercomputing Internet resource text. The hidden layer representation of all coding descriptions is calculated, and the W obtained from it is c , the calculation formula is as follows:

[0119]

[0120] Among them, the matrix W l The acquisition process is as follows:

[0121] Initial W l It is initialized with a random Gaussian and then continuously optimized during the training process.

[0122] Then pass through a linear layer. V d W c +V d W instead of V d W, so as to focus more on those rare codes, then formula (15) is changed to:

[0123]

[0124] At this time, the optimal resource description text attention representation V′ obtained in step 3 is d , substitute it into formula (18).

[0125] To verify the effectiveness of the present invention, the method of predicting resource coding based on corresponding coding and hierarchical coding description is compared with other methods, and micro and macro AUC (area under the ROC curve), micro and macro F1 scores, and Precision@K are used as the main indicators. For the MIMIC-III 50 dataset, Precision@5 (P@5) is used, and for the comprehensive MIMIC-III Full, Precision@8 (P@8) is used. The evaluation results can be found in Figure 2 .

[0126] From the comparative experimental results, the performance comparison of the baseline models and the proposed AHDD framework can be found in the MIMIC-III Full and MIMIC-III 50 settings. In general, it can be observed that the implementation of the AHDD framework enhances the performance of all baseline models.

[0127] This proves the effectiveness and necessity of extracting resource description text through correspondence and hierarchical coding description.

[0128] The significant improvement in F1 and Precision@K scores further confirms the effectiveness of the proposed framework in capturing relevant resource vocabulary for system administrators as well as operation and maintenance teams.

[0129] The AHDD framework accurately captures the representation of resource descriptions, which is a key factor in handling noisy text scenarios.

[0130] To further verify this inference, we divided the resources into five different groups according to the average text length associated with the encoding: [0,500], [501,1000], [1001,1500], [1501,2000], and [2001,+∞]. We calculated the micro F1 scores of all baseline methods and compared them with the corresponding scores within the AHDD framework. The results can be found in Figure 3 .

[0131] A notable observation from this analysis is that AHDD achieves significant improvements in the F1 scores for the [0, 500], [501, 1000], [1001, 1500], and [1501, 2000] groups.

[0132] Furthermore, these relative improvements become increasingly more pronounced as the average resource description text length grows. This trend emphasizes the effectiveness of the AHDD framework in identifying keywords in noisy and complex resource description texts, validating its ability to effectively handle such data.

[0133] In order to better explain the effectiveness of the relevant strategies proposed in this invention on the resource coding task, the model is subjected to ablation learning, and the coding description driven attention, corresponding coding distillation, hierarchical coding distillation and coding description driven classification are removed from the model for experimental analysis.

[0134] like Figure 4 As shown in the table, each data represents the experimental result after removing the relevant modules.

[0135] In the removal corresponding to the description of distillation ( Figure 4 w / o ADD), the performance indicators drop sharply, especially in the P@5 indicator. This shows the ability of corresponding encoding description distillation in capturing keywords in noisy and complex resource descriptions;

[0136] In removing the hierarchical encoding description distillation ( Figure 4 w / o HDD), the performance indicators drop significantly, especially in the F1 indicator. However, the P@5 metric remains unchanged. This shows the effectiveness of hierarchical description distillation in avoiding inappropriate encoding allocation;

[0137] No encoding drives attention ( Figure 4 w / o D-att in the training set, the attention layer degenerates into the attention layer in previous work. In various indicators, the performance is significantly reduced, especially in the F1 and P@5 indicators, indicating that the ability to capture important words is significantly reduced;

[0138] Remove the encoding description perception output ( Figure 4 w / o D-output in , the output layer degenerates to the output layer in previous work. The performance is significantly degraded in various indicators, especially in F1 and P@5 indicators. This indicates that the ability to predict rare codes is reduced.

[0139] In summary, the resource encoding method based on correspondence and hierarchical coding description distillation proposed in the present invention has greater advantages in capturing keywords in noisy resource description texts and avoiding improper coding allocation compared to existing commonly used resource encoding methods. In addition, the attention layer and classification layer designed in the present invention can make full use of the coding description, thereby improving the final coding accuracy.

[0140] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.

Claims

1. A supercomputing interconnection resource encoding method based on resource hierarchical relationship description, characterized in that: The following steps are involved: Step 1: Encode the resource description text, the corresponding coding description and the hierarchical coding description into a vector; Step 2: Calculate the attention representation V of the resource description text based on the attention mechanism driven by the encoding description d , corresponding to the coding description representation Hierarchical coding description representation Step 3: Describe the representation based on the corresponding code and hierarchical coding description representation Distillation, obtain the optimal attention representation V′ of resource description text d ; Step 4: Input the optimal attention representation of the resource description text into the encoding description driven classifier to predict the resource encoding.

2. The supercomputing interconnection resource encoding method based on resource hierarchical relationship description according to claim 1 is characterized in that: In step 2, the attention representation V of the resource description text is calculated d The specific process includes: a C =Attention(H C W Q ,H d ,H d ) V d =H d ·a C in, - an encoded representation generated based on the hidden layer representation of each resource encoding; - Each resource code Description; c i - the i-th resource code; N l -The number of resource codes; H C -Hidden layer representations of all resource encodings; W Q -Linear projection matrix with respect to the Query matrix; α C -Attention weight matrix.

3. The supercomputing interconnection resource encoding method based on resource hierarchical relationship description according to claim 2 is characterized in that: In step 4: in - score vector for all resource encodings; W l - Linear transformation matrix of resource encoding in the classifier; V′ d -Optimal attention representation of resource description text; W - weight matrix for direct classification in the classifier.

4. The supercomputing interconnection resource encoding method based on resource hierarchical relationship description according to claim 1 is characterized in that: In step 3, based on the corresponding encoding description representation Distillation involves the following steps: The attention representation in the resource description text is combined with the corresponding encoded description representation The distance is reduced; Use cosine distance to measure and V d The similarities between.

5. The supercomputing interconnection resource encoding method based on resource hierarchical relationship description according to claim 1 is characterized in that: In step 3, the hierarchical coding description representation The loss of distillation is the resource description text representation V d and hierarchical coding description The dissimilarity between them.

Images