A post-processing-based supercomputing internet resource description and query method

Abstract

The application relates to the field of supercomputer internet resource description and query, and provides a supercomputer internet resource description and query method based on post-processing, which comprises the following steps: obtaining an initial query result according to a query question and a resource description context; re-querying the resource description context for the obtained initial query result, and classifying the initial query result into 'correct information','semi-correct information' and 'wrong information'; correcting the'semi-correct information' by re-querying the resource description context; and finally combining the 'correct information' and the modified'semi-correct information' to obtain a final query result. The method can effectively improve the accuracy of resource description and query.

Description

Technical Field

[0001] The present invention relates to the field of supercomputing Internet resource description and query. Background Art

[0002] In recent years, with the rapid development of internet technology and artificial intelligence applications, traditional computing resource platforms have been unable to meet the computing power requirements of high-performance computing applications. Supercomputing internet technology provides a solution to this problem. Supercomputing internet refers to an emerging computing model that deeply integrates supercomputing technology with the internet. It utilizes high-speed network connections to integrate geographically distributed supercomputing resources to form a powerful computing platform. Through supercomputing internet, scientific research institutions and enterprises can efficiently share computing resources and achieve cross-disciplinary and cross-sector collaboration. It also provides strong support for applications such as big data analysis, artificial intelligence research, and climate simulation, promoting the rapid development of science and technology.

[0003] In order to better build a supercomputing Internet platform, it is crucial to describe, manage, and schedule available computing resources. Among them, resource description technology can help users and managers understand and utilize available resources by defining and managing computing resources, storage resources, and network resources in the supercomputing Internet platform in detail.

[0004] Traditional resource description technologies primarily rely on structured description languages ​​(such as XML and WDSL) and are stored in databases. When querying resources, SQL statements are used to retrieve relevant information from the database based on the resource information. With the development of large language models (LLMs), resource description methods based on natural language have become the current mainstream research topic. By describing computing resources in text form and leveraging the language understanding capabilities of LLMs, information about computing resources can be obtained in the form of questions and answers, allowing users and managers to better understand and schedule resources. Although LLMs have achieved good performance in various language processing tasks, they do not consider the real-time updates of computing resources in resource description and query tasks, and therefore often provide inaccurate resource description information. Summary of the Invention

[0005] To address the query errors caused by outdated resource information, this paper proposes a post-processing-based supercomputing internet resource description and query method. Specifically, this post-processing method utilizes a classification model and a modification model to classify resource information retrieved by LLM, remove erroneous information, and modify some answer information, thereby updating the retrieved resource information.

[0006] Technical solution of the present invention:

[0007] A post-processing-based supercomputing Internet resource description and query method comprises the following steps:

[0008] Step 1: Get the initial query results;

[0009] Step 2: For the initial query results obtained in step 1, the query results are classified according to their correctness based on the corresponding search question and resource description context, and the query results are classified into "correct information", "semi-correct information" and "incorrect information";

[0010] Step 3: For "semi-correct information", modify the query results by re-querying the resource description context;

[0011] Step 4: Finally, the query results obtained by combining the "correct information" of step 2 and the modified "semi-correct information" of step 3 to obtain the final query results.

[0012] The present invention's post-processing-based supercomputing internet resource description and query method trains a classification model to classify query results into three categories and eliminates predicted "incorrect information." It then trains a modification model to modify the "semi-correct information" predicted by the classification model and merges the modified answer with the "correct information" predicted by the classification model as the final query result. Therefore, this method can effectively improve the accuracy of resource description and query.

[0013] The present invention can be applied to resource description and query and other information retrieval tasks, reducing the amount of erroneous information and improving the quality of query results through post-processing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagram of the process of the present invention.

[0015] Figure 2 Schematic diagram of the structure of the classification model in an embodiment of the present invention.

[0016] Figure 3 Schematic diagram of the structure of the modification model in an embodiment of the present invention.

[0017] Figure 4 These are the experimental results based on MultiSpanQA and other datasets in the embodiments of the present invention.

[0018] Figure 5 This is the ablation experiment result in the embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solution provided by this application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of this application will become more apparent with reference to the following description.

[0020] This paper explores resource description and query tasks and proposes the ACC (Anser Classifier Corrector) framework. The ACC framework consists of a query model (Reader), a classification model (Classifier), and a correction model (Corrector). Given a query question and resource description context, the ACC framework first uses the query model to obtain initial query results. It then uses the classification model to classify each query result into "correct information," "semi-correct information," and "incorrect information." Finally, it uses the correction model to modify all "semi-correct information." Ultimately, the "correct information" predicted by the classification model and the "semi-correct information" modified by the correction model are retained as the final output of the ACC framework.

[0021] The specific steps are as follows:

[0022] Step 1: Get the initial query results.

[0023] Step 1: Given a query question Q and a resource description context C, use the query model to extract several text fragments P = {p1, p2, ..., p n} as the initial query result:

[0024] P=Reader(Q,C) (1)

[0025] Among them, the query model can adopt the existing multi-segment question-answering model, such as Tagger-BERT.

[0026] Step 2: Classify the query results.

[0027] Step 2 uses the classification model to classify the initial query results obtained in step 1 into three categories: "correct information," "semi-correct information," and "incorrect information." The initial query results contain multiple query results. Each query result is classified into three categories: "correct information," "semi-correct information," or "incorrect information," depending on the accuracy of the query result. "Correct information" will be retained, "semi-correct information" will be corrected in step 3, and "incorrect information" will be excluded. This process can be expressed as follows:

[0028] P t ,P p ,P w =Classifier(Q,C,P) (2)

[0029] Among them, P t ,P p ,P w They are "correct information", "half-correct information" and "wrong information" respectively.

[0030] The classification model includes an encoder, a cross-attention layer, an average pooling layer, an MLP and a normalization layer. Its input is the query result o, the query question q and the resource description context p. Its output is the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "incorrect information", which are s1, s2 and s3 respectively.

[0031] like Figure 2 As shown in the figure, for each query result o, it is concatenated with the query question q and sent to the encoder of the PLM (Pre-trained Language Model) together with the resource description context p to obtain the corresponding vector representation H qo and H c Then perform cross-attention calculation and average pooling the calculation results to obtain h qc and h cq . Finally, h qc and h cq The concatenation is performed and fed into the MLP. After softmax normalization, the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "incorrect information" are obtained, which are s1, s2, and s3 respectively. The above process can be expressed as:

[0032] H qo ,H c =Encoder([q;o],c) (3)

[0033] h qc =MeanPooling(CrossAttention(H qo ,H qo ,H c )) (4)

[0034] h cq =MeanPooling(CrossAttention(H c ,H c ,H qo )) (5)

[0035] s1,s2,s3=Softmax(MLP([P qc ;h cq ])) (6)where [·;·] represents a vector concatenation operation.

[0036] Furthermore, the encoder may adopt a RoBERTa encoder.

[0037] In the training, the classification model is optimized using the cross-entropy loss function. Preferably, the MultiSpanQA, MultiSpanQA-Expand, MAMRC and MAMRC-Multi datasets can be used for training the classification model.

[0038] When using the trained classification model to classify the query results, the category with the maximum value among the three category corresponding probability regression values is selected as the classification result.

[0039] Step 3, modifying the semi-correct information

[0040] After classifying the query results in step 2, step 3 uses the modification model to modify the "semi-correct information" and obtains the modified query results The above process can be represented as:

[0041]

[0042] The modification model includes an encoder and a linear transformation layer, and the input is the query result o, the query question q and the resource description context p, and the output is the probability value ps of each token of the resource description context as the beginning of the new query result and the probability value pe of the end.

[0043] As shown in Figure 3 , the modification model realizes modification by re-predicting the position of the text segment corresponding to the new query result. Like the classification model, the modification model concatenates each query result o with the query question q, and sends it together with the resource description context p into the encoder (Encoder) of the PLM to obtain the corresponding vector representation H qo and H c . Then, the probability value ps of each token of the resource description context as the beginning of the new query result and the probability value pe of the end are obtained through two linear transformation layers (Linear). The above process can be represented as:

[0044] ps=W1·H c +b1 (8)

[0045] pe=W2·H c +b2 (9)

[0046] Where W1, W1, b1, b2 are the learnable parameters of the linear transformation layer.

[0047] Further, the encoder can adopt the RoBERTa encoder.

[0048] During the training phase, the cross entropy loss function is used to optimize ps and pe. Preferably, the modified model can be trained using datasets such as MultiSpanQA, MultiSpanQA-Expand, MAMRC, and MAMRC-Multi.

[0049] When using the trained modification model to modify "semi-correct information", first use the modification model to obtain the probability value ps of each token in the resource description context as the beginning of the new query result. i And the probability value pe of the ending j , then get the ps i ·pe j The largest subscript pair (i, j) is finally extracted, and the corresponding text fragment is used as the modified query result, as shown in formula (10):

[0050] (i,j)=argmax i＜j (ps i ·pe j ) (10)

[0051] Step 4: Get the final query results.

[0052] Query results output by the ACC framework The “correct information” P predicted by the classification model in step 2 t And the query results obtained after modifying the "semi-correct information" using the modified model in step 3 Right now:

[0053]

[0054] Experimental verification

[0055] To verify the effectiveness of the present invention, the method was tested on a multi-answer reading comprehension task and compared with other methods. This experiment uses the accuracy (Precision), recall (Recall) and F1 score under the exact match (Exact Match) as evaluation indicators. The scores of the evaluation indicators can be found in Figure 3 .

[0056] In the comparative experiments, the present invention combines the methods of other multi-segment question answering models and uses the ACC framework to modify their prediction results (i.e., other multi-segment question answering models are used as query models) on four datasets: MultiSpanQA, MultiSpanQA-Expand, MAMRC, and MAMRC-Multi. The scores before and after the post-processing method are compared. The results are as follows: Figure 4As shown in the figure, the ACC framework can effectively improve the F1 scores of various multi-span question answering models. For example, on the MultiSpanQA dataset, the F1 score of Tagger-BERT increased by 7% and that of Tagger-RoBERTa by 3.2%. This improvement in F1 score is primarily due to the increase in accuracy, demonstrating that the post-processing method proposed in this paper can effectively reduce the number of erroneous information and improve the quality of query results.

[0057] In order to better illustrate the effectiveness of the post-processing strategy proposed in this paper, ablation learning is performed on the ACC framework, removing the modification model / classification model (cls / cor only), swapping the order of the classification model and the modification model (cor&cls), and modifying the "correct information" and "semi-correct information" (binary cls&cor). The results are shown in the figure. Figure 5 As shown in the figure, removing the classification model and modifying the model both resulted in a decrease in the F1 score, indicating that the classification model and the modification model play an equally important role in the ACC framework. Reversing the order in which the classification model and the modification model are used also resulted in a certain decrease in the F1 score. This may be because applying the modification model first modifies the query results, thus affecting the judgment of the classification model. Furthermore, modifying both "correct information" and "semi-correct information" simultaneously also affects the performance of the ACC framework, as the ACC framework may change already correct information into incorrect information.

[0058] In summary, the post-processing supercomputing internet resource description and query method proposed in this invention can improve the quality of query results by eliminating erroneous information and modifying partially correct information. Furthermore, the invention can be combined with large language models, which provides certain scalability and versatility.

[0059] The above description is only a description of the preferred embodiments of the present application and does not limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical content should be regarded as equivalent valid embodiments and fall within the scope of protection of the technical solution of the present application.

Claims

1. A post-processing-based supercomputing Internet resource description and query method, characterized in that: Including steps: Step 1: Get the initial query results; Step 2: For the initial query results obtained in step 1, the query results are classified according to their correctness, combining the corresponding search question and resource description context, and classified into "correct information", "semi-correct information" and "incorrect information"; Step 3: For "semi-correct information", modify the query results by re-querying the resource description context; Step 4: Finally, the query results obtained by combining the "correct information" in step 2 and the modified "semi-correct information" in step 3 to obtain the final query results; The step 1 comprises: For a given query and resource description context , using the query model to extract several text fragments from the resource description context As an initial query result: The step 2 includes: Use the classification model to classify the initial query results: The initial query results contain multiple query results. For each query result, it is classified as "correct information", "semi-correct information", or "incorrect information" based on the correctness of the query result. "Correct information" will be retained, "semi-correct information" will be corrected in step 3, and "incorrect information" will be excluded. This process is expressed as follows: in They are "correct information", "half-correct information" and "wrong information"; The classification model includes an encoder, a cross attention layer, an average pooling layer, an MLP layer, and a normalization layer, and its input is the query result. , query questions and resource description context , the output is the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "wrong information", which are ; The process is: For each query result , and compare it to the query Splice and describe the context with resources Send them together to the encoder of PLM (Pre-trained Language Model) to obtain the corresponding vector representation and ; Then perform cross-attention calculation and average pooling on the result to get and ; will eventually and The concatenation is performed and fed into MLP. After softmax normalization, the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "wrong information" are obtained, which are ; The above process is expressed as: in Represents vector concatenation operation; During training, the classification model is optimized using the cross entropy loss function; When using the trained classification model to classify the query results, the category with the maximum probability regression value among the three categories is selected as the classification result; The step 3 includes: Use the modification model to modify the "semi-correct information" and obtain the modified query results ; The above process is expressed as: The modification model includes an encoder and a linear transformation layer, and its input is the query result. , query questions and resource description context The output is the probability value of each token in the resource description context as the beginning of a new query result and the probability value of the ending ; The modification model is implemented by re-predicting the position of the text fragment corresponding to the new query result. The process is as follows: for each query result , and compare it to the query Splice and describe the context with resources Send them together to the PLM encoder to obtain the corresponding vector representation and ; Afterwards, two linear transformation layers are used to obtain the probability value of each token in the resource description context as the beginning of a new query result And the probability value of the ending ; The above process is expressed as: Among them, W1, W1, b1, and b2 are the learnable parameters of the linear transformation layer; During the training phase, the cross entropy loss function is used to optimize and ; When using the trained modification model to modify "semi-correct information", first use the modification model to obtain the probability value of each token in the resource description context as the beginning of the new query result And the probability value of the ending , then obtain The largest subscript pair , and finally extract the corresponding text fragment as the modified query result, as shown in formula (10): The step 3 includes: Final query results The "correct information" obtained in step 2 And the query results obtained after modifying the "semi-correct information" in step 3 ,Right now: 。

Images