Supercomputing Internet resource description and query method based on post-processing

Abstract

The invention relates to the field of supercomputing Internet resource description and query, and provides a postprocessing-based supercomputing Internet resource description and query method, which comprises the following steps of: obtaining an initial query result according to a query problem and a resource description context; re-querying a resource description context for the obtained initial query result, and classifying the resource description context into correct information, semi-correct information and error information; for the semi-correct information, correcting by re-querying a resource description context; and finally, combining the'correct information 'and the modified'semi-correct information' to obtain a final query result. The method can effectively improve the accuracy of resource description and query.

Description

Technical Field

[0001] The present invention relates to the field of supercomputing Internet resource description and query. Background Art

[0002] In recent years, with the rapid development of Internet technology and artificial intelligence applications, traditional computing resource platforms can no longer meet the computing power requirements of high-performance computing applications. Supercomputing Internet technology provides a solution to this problem. Supercomputing Internet refers to an emerging computing model that deeply integrates supercomputing technology with the Internet. It uses high-speed network connections to integrate supercomputing resources distributed in different geographical locations to form a powerful computing platform. Through supercomputing Internet, scientific research institutions and enterprises can efficiently share computing resources and achieve cross-disciplinary and cross-field cooperation. At the same time, supercomputing Internet also provides strong support for applications such as big data analysis, artificial intelligence research, and climate simulation, promoting the rapid development of science and technology.

[0003] In order to better build a supercomputing Internet platform, the description, management and scheduling of available computing resources are crucial. Among them, resource description technology can help users and managers understand and utilize available resources through detailed definition and management of computing resources, storage resources and network resources in the supercomputing Internet platform.

[0004] Traditional resource description technology mainly relies on structured description languages ​​(such as XML, WDSL, etc.) and is stored in a database. When querying, SQL statements are used to retrieve relevant information from the database based on the information of the resource to achieve resource query. With the development of large language models (LLM), resource description methods based on natural language have become the mainstream research content. By describing computing resources in text form and relying on the language understanding ability of LLM, computing resource information is obtained in the form of questions and answers, which makes it easier for users and managers to better understand resources and schedule resources. Although LLM has achieved good performance in various language processing tasks, in resource description and query tasks, large language models do not consider the real-time update of computing resources, and therefore usually provide inaccurate resource description information. Summary of the invention

[0005] In order to solve the query error problem caused by the lag of resource information, the present invention proposes a supercomputing Internet resource description and query method based on post-processing. Specifically, by using the post-processing method, by designing a classification model and a modification model, the resource information retrieved by LLM is classified, the error information is removed and some answer information is modified, so as to update the retrieved resource information.

[0006] Technical solution of the present invention:

[0007] A post-processing-based supercomputing Internet resource description and query method, comprising the following steps:

[0008] Step 1: Get the initial query results;

[0009] Step 2: For the initial query results obtained in step 1, the query results are classified according to their correctness in combination with the corresponding search question and resource description context, and the query results are classified into "correct information", "semi-correct information" and "wrong information";

[0010] Step 3: For "semi-correct information", modify the query results by re-querying the resource description context;

[0011] Step 4: Finally, the query results obtained by modifying the "correct information" in step 2 and the "semi-correct information" in step 3 are merged to obtain the final query results.

[0012] The present invention provides a post-processing-based supercomputing Internet resource description and query method. By training a classification model, the query results are classified into three categories and the predicted "wrong information" is excluded; by training a modification model, the "semi-correct information" predicted by the classification model is modified, and the modified answer is merged with the "correct information" predicted by the classification model as the final query result. Therefore, the method can effectively improve the accuracy of resource description and query.

[0013] The present invention can be applied to resource description and query and other information retrieval tasks, and can reduce the amount of erroneous information and improve the quality of query results through post-processing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the process of the present invention.

[0015] Figure 2 Schematic diagram of the structure of the classification model in the embodiment of the present invention.

[0016] Figure 3 It is a structural diagram of the modification model in the embodiment of the present invention.

[0017] Figure 4 These are the experimental results based on MultiSpanQA and other datasets in the embodiments of the present invention.

[0018] Figure 5 This is the ablation experiment result in the embodiment of the present invention. DETAILED DESCRIPTION

[0019] The technical solution provided by the present application will be further described below in conjunction with specific embodiments and accompanying drawings. The advantages and features of the present application will become more apparent with the following description.

[0020] The present invention explores resource description and query tasks, and proposes an ACC (Anser ClassifierCorrector) framework. The ACC framework includes a query model Reader, a classification model Classifier, and a modification model Corrector. For a given query question and resource description context, the ACC framework first uses the query model to obtain the initial query results, then uses the classification model to classify each query result into "correct information", "semi-correct information" and "wrong information", and finally uses the modification model to modify all "semi-correct information". Finally, the "correct information" predicted by the classification model and the "semi-correct information" modified by the modification model are retained as the final output of the ACC framework.

[0021] The specific steps are as follows:

[0022] Step 1: Get the initial query results.

[0023] Step 1: Given a query question Q and a resource description context C, use the query model to extract several text fragments P = {p1, p2, ..., p n} as the initial query result:

[0024] P=Reader(Q,C) (1)

[0025] Among them, the query model can adopt the existing multi-fragment question-answering model, such as Tagger-BERT.

[0026] Step 2: Classify the query results.

[0027] Step 2 uses the classification model to classify the initial query results obtained in step 1 into three categories: "correct information", "semi-correct information" and "wrong information". The initial query results contain multiple query results. For each query result, three categories are performed, and they are classified into "correct information", "semi-correct information" or "wrong information" according to the correctness of the query results. Among them, "correct information" will be retained, "semi-correct information" will be corrected in step 3, and "wrong information" will be excluded. This process can be expressed as:

[0028] P t ,P p ,P w =Classifier(Q,C,P) (2)

[0029] Where P t ,P p ,P w They are "correct information", "semi-correct information" and "wrong information" respectively.

[0030] The classification model includes an encoder, a cross attention layer, an average pooling layer, an MLP and a normalization layer. Its input is the query result o, the query question q and the resource description context p. Its output is the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "wrong information", which are s1, s2 and s3 respectively.

[0031] like Figure 2 As shown in Figure 1, for each query result o, it is concatenated with the query question q and sent to the encoder of the PLM (Pre-trained Language Model) together with the resource description context p to obtain the corresponding vector representation H qo and H c Then, we perform cross-attention calculation and average pool the calculation results to get h qc and h cq . Finally, h qc and h cq After concatenation and feeding into MLP, the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "wrong information" are obtained after softmax normalization, which are s1, s2, and s3 respectively. The above process can be expressed as:

[0032] H qo ,H c =Encoder([q;o],c) (3)

[0033] h qc =MeanPooling(CrossAttention(H qo ,H qo ,H c )) (4)

[0034] h cq =MeanPooling(CrossAttention(H c ,H c ,H qo )) (5)

[0035] s1,s2,s3=Softmax(MLP([P qc ;h cq ])) (6) where [·;·] represents a vector concatenation operation.

[0036] Furthermore, the encoder may adopt a RoBERTa encoder.

[0037] During training, the cross entropy loss function is used to optimize the classification model. Preferably, the classification model can be trained using datasets such as MultiSpanQA, MultiSpanQA-Expand, MAMRC, and MAMRC-Multi.

[0038] When using the trained classification model to classify the query results, the category with the maximum value of the probability regression values ​​corresponding to the three categories is selected as the classification result.

[0039] Step 3: Modify semi-correct information

[0040] After classifying the query results in step 2, step 3 uses the modification model to modify the "semi-correct information" and obtain the modified query results The above process can be expressed as:

[0041]

[0042] The modification model includes an encoder and a linear transformation layer, whose input is the query result o, the query question q and the resource description context p, and whose output is the probability value ps and the probability value pe of each token of the resource description context as the beginning of the new query result.

[0043] like Figure 3 As shown in Figure 1, the modification model implements the modification by re-predicting the position of the text segment corresponding to the new query result. Similar to the classification model, the modification model concatenates each query result o with the query question q and sends it to the encoder of the PLM together with the resource description context p to obtain the corresponding vector representation H qo and H c . Then, two linear transformation layers are used to obtain the probability value ps of each token in the resource description context as the beginning of the new query result and the probability value pe of the end. The above process can be expressed as:

[0044] ps=W1·H c +b1 (8)

[0045] pe=W2·H c +b2 (9)

[0046] Among them, W1, W1, b1, and b2 are learnable parameters of the linear transformation layer.

[0047] Furthermore, the encoder may adopt a RoBERTa encoder.

[0048] In the training phase, the cross entropy loss function is used to optimize ps and pe. Preferably, the modified model can be trained using datasets such as MultiSpanQA, MultiSpanQA-Expand, MAMRC, and MAMRC-Multi.

[0049] When using the trained modification model to modify the "semi-correct information", first use the modification model to obtain the probability value ps of each token in the resource description context as the beginning of the new query result i And the probability value pe of the ending j , then get the ps i ·pe j The largest subscript pair (i, j) is finally extracted to obtain the corresponding text fragment as the modified query result, as shown in formula (10):

[0050] (i,j)=argmax i＜j (ps i ·pe j ) (10)

[0051] Step 4: Get the final query result.

[0052] Query results output by the ACC framework The “correct information” P predicted by the classification model in step 2 t And the query results obtained after modifying the "semi-correct information" using the modification model in step 3 Right now:

[0053]

[0054] Experimental verification

[0055] To verify the effectiveness of the present invention, the method is verified on a multi-answer reading comprehension task and compared with other methods. This experiment uses the accuracy (Precision), recall (Recall) and F1 score under exact match as evaluation indicators. The scores of the evaluation indicators can be found in Figure 3 .

[0056] In the comparative experiment, the present invention combines the methods of other multi-segment question answering models and uses the ACC framework to modify their prediction results (i.e., other multi-segment question answering models are used as query models) on four datasets: MultiSpanQA, MultiSpanQA-Expand, MAMRC, and MAMRC-Multi. The scores before and after the post-processing method are compared. The results are as follows: Figure 4As shown. From the comparison results, it can be found that the ACC framework can effectively improve the F1 score of each multi-segment question answering model. For example, on the MultiSpanQA dataset, the F1 score on Tagger-BERT has increased by 7%, and on Tagger-RoBERTa by 3.2%. The improvement in the F1 score mainly comes from the improvement in accuracy. It can be seen that the post-processing method proposed in the present invention can effectively reduce the number of erroneous information and improve the quality of query results.

[0057] In order to better explain the effectiveness of the post-processing strategy proposed in this paper, ablation learning is performed on the ACC framework, removing the modification model / classification model (cls / cor only), swapping the order of using the classification model and the modification model (cor&cls), and modifying the "correct information" and "semi-correct information" (binary cls&cor). The results are shown in Figure 2. Figure 5 As shown in the figure. Removing the classification model and modifying the model both caused the F1 score to drop, which shows that the classification model and the improvement model are equally important to the ACC framework. After swapping the order of using the classification model and the modification model, the F1 score also dropped to a certain extent. This may be because if the modification model is used first, the query results will be modified, thus affecting the judgment of the classification model. In addition, modifying the "correct information" and "semi-correct information" at the same time will also affect the performance of the ACC framework, because the ACC framework may change the correct information into wrong information.

[0058] In summary, the post-processing supercomputing Internet resource description and query method proposed in the present invention can improve the quality of query results by eliminating erroneous information and modifying semi-correct information. In addition, the present invention can be combined with a large language model and has certain scalability and versatility.

[0059] The above description is only a description of the preferred embodiments of the present application, and is not intended to limit the scope of the present application. Any changes or modifications made by any person skilled in the art based on the above disclosed technical contents shall be deemed as equivalent effective embodiments and shall fall within the scope of protection of the technical solution of the present application.

Claims

1. A post-processing-based supercomputing Internet resource description and query method, characterized in that: Includes steps: Step 1: Get the initial query results; Step 2: For the initial query results obtained in step 1, the query results are classified according to their correctness in combination with the corresponding search question and resource description context, and the query results are classified into "correct information", "semi-correct information" and "wrong information"; Step 3: For "semi-correct information", modify the query results by re-querying the resource description context; Step 4: Finally, the "correct information" of step 2 and the query results obtained after modifying the "semi-correct information" in step 3 are merged to obtain the final query results.

2. A post-processing based supercomputing Internet resource description and query method as claimed in claim 1, characterized in that: Step 1: Given a query question Q and a resource description context C, the query model is used to extract several text fragments P = {p1, p2, ..., p n } as the initial query result: P = Reader (Q, C) (1).

3. A post-processing based supercomputing Internet resource description and query method as claimed in claim 1, characterized in that: Step 2: Use the classification model to classify the initial query results: The initial query results contain multiple query results. For each query result, it is classified into "correct information", "semi-correct information" or "wrong information" according to the correctness of the query result; "correct information" will be retained, "semi-correct information" will be corrected in step 3, and "wrong information" will be excluded; this process is expressed as: P t ,P p ,P w =Classifier(Q,C,P) (2) Where P t ,P p ,P w They are "correct information", "semi-correct information" and "wrong information" respectively; The classification model includes an encoder, a cross attention layer, an average pooling layer, an MLP and a normalization layer. Its input is the query result o, the query question q and the resource description context p. Its output is the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "wrong information", which are s1, s2 and s3 respectively. The process is as follows: For each query result o, it is concatenated with the query question q and sent to the encoder of the PLM (Pre-trained Language Model) together with the resource description context p to obtain the corresponding vector representation H qo and H c ; Then, the cross-attention calculation is performed, and the calculation results are average-pooled to obtain h qc and h cq ; Finally, h qc and h cq After concatenation and feeding into MLP, the probability regression values ​​corresponding to the three categories of "correct information", "semi-correct information" and "wrong information" are obtained after softmax normalization, which are s1, s2 and s3 respectively. The above process is expressed as: H qo ,H c =Encoder([q;o],c) (3) h qc =MeanPooling(CrossAttention(H qo ,H qo ,H c )) (4) h cq =MeanPooling(CrossAttention(H c ,H c ,H qo )) (5) <h2 style=";text-align:left;direction:ltr">s1,s2,s3=Softmax(MLP([h<h2 style=";text-align:left;direction:ltr"> qc <h2 style=";text-align:left;direction:ltr"> ;h<h2 style=";text-align:left;direction:ltr"> cq <h2 style=";text-align:left;direction:ltr"> ))) (6) Where [·;·] represents the vector concatenation operation; During training, the classification model is optimized using the cross entropy loss function; When the trained classification model is used to classify the query results, the category with the maximum value among the probability regression values ​​corresponding to the three categories is selected as the classification result.

4. A post-processing based supercomputing Internet resource description and query method as claimed in claim 1, characterized in that: Step 3: Use the modification model to modify the "semi-correct information" and obtain the modified query results The above process is expressed as: The modification model includes an encoder and a linear transformation layer, whose input is the query result o, the query question q and the resource description context p, and whose output is the probability value ps and the probability value pe of each token of the resource description context as the beginning and the end of the new query result; The modification model realizes the modification by re-predicting the position of the text fragment corresponding to the new query result. The process is as follows: for each query result o, it is concatenated with the query question q and sent to the encoder of the PLM together with the resource description context p to obtain the corresponding vector representation H qo and H c ; Afterwards, two linear transformation layers are used to obtain the probability value ps of each token in the resource description context as the beginning of the new query result and the probability value pe of the end; the above process is expressed as: ps=W1·H c +b1 (8) <h2 style=";text-align:left;direction:ltr">pe = W2·H<h2 style=";text-align:left;direction:ltr"> c <h2 style=";text-align:left;direction:ltr"> +b2 (9) Among them, W1, W1, b1, and b2 are the learnable parameters of the linear transformation layer; During the training phase, the cross entropy loss function is used to optimize ps and pe; When using the trained modification model to modify "semi-correct information", first use the modification model to obtain the probability value ps of each token in the resource description context as the beginning of the new query result i And the probability value pe of the ending j , then get the ps i ·pe j The largest subscript pair (i, j) is finally extracted to obtain the corresponding text fragment as the modified query result, as shown in formula (10): (i,j)=argmax i＜j (ps i ·pe j ) (10)。 5. A post-processing based supercomputing Internet resource description and query method as claimed in claim 1, characterized in that: Step 3: Final query results The "correct information" P obtained in step 2 t And the query results obtained after modifying the "semi-correct information" in step 3 Right now:

Images