A multi-hop question and answer model construction method based on table-text hybrid data

Abstract

The application discloses a kind of multi-hop question and answer model construction methods based on table-text hybrid data, the model constructed by text content filtering module, table row content screening module and fusion data extraction question and answer module are composed.Text content filtering module is used to screen the text sentence content related to question;Table row content screening module is used to construct table row-text fusion data and further screen out the content related to question;Fusion data extraction question and answer module is used to extract answer in table row content-text fusion data.The model constructed by the application is screened by content, greatly reduces redundant information, makes text content effectively supplement table content, accelerates model training speed, improves the accuracy of table-text hybrid data question and answer.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-hop question and answer models for table-text mixed data, specifically a model that can comprehensively perform multi-hop question and answer on a large amount of table and text content, which can be used in network database content retrieval, knowledge question and answer, and other scenarios. BACKGROUND

[0002] The table-text mixed data question and answer task aims to use machines to reason and answer questions based on table and text data, in order to improve data processing efficiency. However, due to the large amount of reference text data and the presence of two heterogeneous data, the table-text mixed data question and answer task often faces the problems of ultra-long text encoding and heterogeneous data alignment. However, as the mainstream framework in the field of natural language processing, the training of pre-trained language models is usually based on natural text training samples. Therefore, when facing question and answer tasks involving tables, the performance is not good. It has high research value to filter text and construct fusion data to improve the question and answer effect of pre-trained models on table-text mixed data. SUMMARY

[0003] The purpose of the present application is to provide a multi-hop question and answer model construction method based on table-text mixed data, which aims to filter the text sentence content related to the question through a text content filtering module, construct table row-text fusion data using a table row content filtering module and further filter the content, and extract answers from the fusion data based on a fusion data extraction question and answer module, in order to improve the question and answer ability of machines for table-text mixed data.

[0004] The specific technical solution to achieve the purpose of the present application is:

[0005] A multi-hop question and answer model construction method based on table-text mixed data, which filters relevant content in text and tables, and constructs table-text fusion data to make pre-trained language models generate accurate answers. The constructed model includes a text content filtering module, a table row content filtering module, and a fusion data extraction question and answer module, wherein:

[0006] The text content filtering module is used to filter sentence content that is linked to cell content and related to the question in units of lines;

[0007] The table row content filtering module is used to construct table row-text fusion data in units of lines, train a table row content filtering model, and further filter the content related to the question;

[0008] The fusion data extraction question and answer module is used to train a fusion data extraction question and answer model in the remaining table row content-text fusion data and extract answers.

[0009] The method requires a mixed table-text dataset containing the following content during its construction: [Issue] and its corresponding table data This includes the column names of the table. And the table OK Column data , No. Row data can be represented as ,lie in OK The column cells exist or do not exist Each corresponding entity title-text pair .

[0010] The text content filtering module is used to filter sentence segments that are linked to the cell content and are relevant to the question, line by line. Specifically, it includes: splitting all the linked text in a row of data in the table into sentence segments, and calculating the value of each sentence segment. With the question Text similarity Sort all text segments in the same line by similarity scores from highest to lowest, and select the segments with the highest similarity scores. Each sentence segment will be retained;

[0011] The table row content filtering module constructs table row-text fusion data row by row, trains a table row content filtering model, and further filters out content relevant to the question. Specifically, it includes: constructing each row of data in the table as a table-text fusion sequence. ,in For the first The concatenation result of the contents of each cell: If the cell contains text that was retained after the previous filtering step, then ,otherwise ,text This is the concatenated result of the "Entity Title: Text" key-value pair; iterate through all cell contents and text content in each row of the table, and if the answer text exists, define that row as related to the question and assign that row's data... With the question Relevance =1, otherwise =0; Calculate the first row using the table row content filtering model. row data With the question Relevance And calculate the model output. , With tags Deviation loss Iterate through all question-line text pairs in the training dataset and calculate the loss. Adjust the model parameters based on the loss until... We no longer reduce the number of rows, thus obtaining the final table row content filtering model. We then use this model to calculate the positive correlation of all rows of data in the same table. Sort by size from largest to smallest, and select the group with the highest similarity. The line is retained;

[0012] The fused data extraction question-answering module trains a fused data extraction question-answering model on the remaining table row content-text fused data and extracts answers, specifically including: [the following is a list of steps related to the module and its components]. Row content of table - text fusion data constructed into a sequence As a problem Reference text; in sequence Find the answer in the text and mark the starting position of the answer. and end position If not found and Let the initial probability of the starting position of the answer be... The probability of ending at the ending position The remaining positions of the sequence , All values ​​are 0; the question-answering model is used to calculate the question. Reference The answer location and ;Calculation model output , With the real answer , Deviation loss ; Iterate through all question-reference text pairs in the training dataset and calculate the loss, then adjust the model parameters based on the loss until... No further reduction is needed; the final fused data extraction question-answering model is obtained, and the model is used to calculate the reference text. , The largest position is used as the starting position of the answer. and end position Take out from arrive The text as a question The answer.

[0013] The question-answering model constructed in this invention significantly reduces redundant information by filtering content, enabling text content to effectively supplement table content, accelerating model training, and improving the accuracy of question-answering based on mixed table-text data. This also demonstrates that the table-text data fusion method proposed in this invention can utilize the natural text processing capabilities inherent in the pre-trained model to enhance the machine's understanding of cross-modal data. Attached Figure Description

[0014] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0015] The present invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the contents specifically mentioned below, the processes, conditions, and experimental methods for implementing the present invention are all common knowledge and general knowledge in the art, and the present invention does not have any particular limitations.

[0016] The following section details a method for constructing a multi-hop question-answering model based on mixed tabular and textual data. This method requires a mixed tabular and textual dataset containing the following information during its construction:

[0017] question and its corresponding table data This includes the column names of the table. And the table OK Column data , No. Row data can be represented as ,lie in OK The column cells exist or do not exist Each corresponding entity title-text pair .

[0018] like Figure 1 As shown in the flowchart, the model constructed in this invention mainly includes: a text content filtering module, a table row content filtering module, and a fusion data extraction and question-and-answer module. The specific steps of its operation are as follows:

[0019] 1) Use the text content filtering module to filter sentence segments that are linked to the cell content and relevant to the question, line by line. This module can filter out a large amount of noisy text data that is irrelevant to the question, reducing the workload of subsequent modules. The specific steps are as follows:

[0020] Step 1: Break down all the link text in a row of data in the table into sentence segments, and calculate the value of each sentence segment. With the question Text similarity The formula is as follows:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026] in Represents a segment of text With the entity title of the text and special characters , The sequence obtained after splicing; Indicate the problem With special characters , The sequence obtained after splicing; This represents a pre-trained BERT encoding model; express After being encoded by the BERT model, the following is taken The vector obtained from the characters is used as the sentence vector; express After being encoded by the BERT model, the following is taken The vector obtained from the characters is used as the sentence vector; This represents the vector dot product operation.

[0027] Step 2: Sort all text segments in the same line by similarity scores from highest to lowest, and select the segments with the highest similarity scores. Each sentence segment will be retained.

[0028] 2) Using a table row content filtering module, construct table row-text fusion data row by row, train a table row content filtering model, and further filter out content relevant to the question. When the question only involves a few rows of content in the table, the table content should be cropped. Simultaneously, fusing table data and text data enables the merging and alignment of heterogeneous data, achieving mutual complementarity between table content and text content, facilitating inference by the pre-trained language model. The specific steps are as follows:

[0029] Step 1: Construct a table-text fusion sequence from each row of data in the table. ,in For the first The concatenation result of the contents of each cell: If the cell contains text that was retained after the previous filtering step, then ,otherwise .text This is the concatenation result of the "Entity Title: Text" key-value pair.

[0030] Step 2: Iterate through all cell contents and text content in each row of the table. If the answer text exists, define that row as related to the question and assign that row's data... With the question Relevance =1, otherwise =0.

[0031] Step 3: Use the table row content filtering model to calculate the first... row data With the question Relevance The formula is as follows:

[0032]

[0033]

[0034] in For the question With merged text and special characters , The sequence obtained after splicing; Represents a linear operation; Represents the softmax activation function; and The first The positive and negative correlations are calculated from the row data.

[0035] Step 4: Calculate the output of the table row content filtering model , With tags Deviation loss The formula is as follows:

[0036]

[0037] in Filter model parameters for table row content.

[0038] Step 5: Iterate through all question-line text pairs in the training dataset and calculate the loss. Adjust the model parameters based on the loss until... We stop reducing the size and arrive at the final table row content filtering model.

[0039] Step Six: Using the table row content filtering model obtained in the previous step, calculate the positive correlation of all rows of data in the same table. Sort by size from largest to smallest, and select the group with the highest similarity. The line is retained.

[0040] 3) Using the fused data extraction question-answering module, a fused data extraction question-answering model is trained on the remaining table row content-text fused data to extract answers. After the filtering by the first two modules, the amount of reference content data related to the question has been significantly reduced, facilitating interaction with the question through cross-coding methods and improving question-answering accuracy. The specific steps are as follows:

[0041] Step 1: Select the filtered results Row content of table - text fusion data constructed into a sequence As a problem Reference text.

[0042] Step 2: In the sequence Find the answer in the text and mark the starting position of the answer. and end position If not found and Let the initial probability of the starting position of the answer be... The probability of ending at the ending position The remaining positions of the sequence , All are 0.

[0043] Step 3: Use the fused data extraction question-answering model to calculate the question. Reference The answer location and The formula is as follows:

[0044]

[0045]

[0046] in For the question with sequence and special characters , The sequence obtained by splicing; This represents the TransformerEncoder encoding model.

[0047] Step 4: Calculate the output of the fused data extraction question-answering model , With the real answer , Deviation loss The formula is as follows:

[0048]

[0049]

[0050]

[0051] in The CrossEntropy loss function is used. for The number of word segments, To extract parameters for the question-answering model from the fused data.

[0052] Step 5: Iterate through all question-reference text pairs in the training dataset and calculate the loss. Adjust the model parameters based on the loss until... We no longer reduce the size, and thus obtain the final fused data extraction question-answering model.

[0053] Step Six: Use the fused data obtained in the previous step to extract the question-answering model and calculate the value of the reference text. , The largest position is used as the starting position of the answer in the reference text. and end position Take out from arrive The text as a question The answer.

[0054] The method proposed in this invention significantly reduces redundant text information through a text content filtering module, enabling text content to effectively supplement table content; it constructs text-table fusion data through a table row content filtering module, solving the heterogeneous data alignment problem and further reducing redundant table and text data; and it achieves low-level interaction between multi-row table-text fusion data content and questions through a fusion data extraction question-answering module, improving the accuracy of question-answering for table-text mixed data. Furthermore, content filtering accelerates model training, saving training time and resources.

[0055] Example

[0056] Given the question "What is that address of the museum located in a Victorian House in an area whose Architectural styles within the district includeCraftsman Bungalow, Queen Anne, Stick style, Folk Victorian, AmericanFoursquare and Neoclassical Revival?" and the corresponding table:

[0057] Name Area Type Summary APEX Museum SweetAuburn AfricanAmerican Contributions of African Americans to the United States and theworld AtlantaContemporary ArtCenter WestMidtown Art Local, national, and international contemporary art ; education geared toward working artists and collectors of art Atlanta HistoryCenter Buckhead History History of Atlanta and Georgia ; includes the Centennial OlympicGames Museum and one of the nation's most complete Civil Warexhibitions Atlanta MonetaryMuseum MidtownAtlanta Numismatic History of money, banking in America, and the Federal Reserve, operated by the Federal Reserve Bank of Atlanta Booth WesternArt Museum Cartersville Art The worlds largest permanent exhibition space for Western art The BremanMuseum MidtownAtlanta Nation Nation history, with special emphasis on Georgia and historicaltragedy Callanwolde FineArts Center DruidHills Art Community arts center, gallery hosts one-person and occasionalgroup exhibitions by emerging local artists in varied media Center for Puppetry Arts Midtown Atlanta Puppetry Puppets from various time periods and countries around the world Children's Museum of Atlanta Luckie Marietta Children's Formerly known as Imagine It! The Children's Museum of Atlanta Clark Atlanta University Art Museum West End Art African American art College Football Hall of Fame Luckie Marietta Sports Honors collegiate athletes from around the country David J. Sencer CDC Museum Druid Hills Medical Public health issues, operated by the Centers for Disease Control and Prevention Delta Flight Museum Hapeville Aviation Aircraft, aviation, and history of Delta Air Lines Dignity Museum College Park Spirit Spirit, Justice Fernbank Museum of Natural History Druid Hills Natural history Natural history and the development of the planet Fernbank Science Center Druid Hills Science Planetarium, telescope, and Fernbank Forest, a 65-acre (260,000 m) natural forest Ferst Center for the Arts Midtown Atlanta Art Contemporary art and photography Georgia Capitol Museum South Downtown History Collections representing the natural and cultural history ofGeorgia Georgia Governor's Mansion Buckhead Historic house Mid 20th century official home of the state's governors Hammonds HouseMuseum WestEnd AfricanAmerican African American fine art, culture of the African diaspora ;located in a historic Queen Anne-style house

[0058] The link text for "Hammonds House Museum" in cell 1 of row 20 is:

[0059] Hammonds House Museum: The Hammonds House Museum is a museum for African American fine art, located at 503 Peeples Street SW in the West Endneighborhood of Atlanta, Georgia. It is located in the 1857 Victorian house, former residence of Dr. Otis Thrash Hammonds, a prominent Atlantaphysician and patron of the arts. The Victorian house is believed to have been built in 1857, and to be one of the three oldest houses in West End.

[0060] The link text for "West End" in cell 2 of row 20 is:

[0061] West End: West End is a historic neighborhood of Atlanta, one of the oldest outside Downtown Atlanta, and is listed on the National Register ofHistoric Places. West End residents are primarily (89%) African American and the neighborhood contains several prominent African American culturalinstitutions, in addition to being adjacent to the Atlanta University Centercomplex of HBCUs. West End is located southwest of Castleberry Hill, eastof Westview, west of Adair Park Historic District, and just north ofOakland City. Architectural styles within the district include CraftsmanBungalow, Queen Anne, Stick style, Folk Victorian, American Foursquareand Neoclassical Revival.

[0062] First, the text content filtering module is used to filter sentence segments that are linked to the cell content and relevant to the question, line by line. For row 20, the relevant linked text is split into sentence segments. Taking the second sentence of the linked text "Hammonds HouseMuseum" as an example, the constructed concatenation sequence is as follows: For: [CLS] Hammonds HouseMuseum: It is located in the 1857 Victorian house, former residence of Dr. Otis Thrash Hammonds, a prominent Atlanta physician and patron of the arts. [SEP]

[0063] And the problem splicing sequence For: [CLS] What is that address of the museum located in a Victorian House in an area whose Architectural styles within the district include Craftsman Bungalow , Queen Anne , Stick style , FolkVictorian , American Foursquare and Neoclassical Revival ? [SEP]

[0064] Recalculate Similarity to all text segments in line 20, retaining the first... A sentence segment. Take The remaining sentence / segment numbers in that row will follow the format [column number, linked text number, sentence number within the segment] as follows:

[0065] [1, 1, 2], [1, 1, 1], [1, 1, 3], [2, 1, 3], [2, 1, 1].

[0066] Next, the table row content filtering module is used to construct table row-text fusion data row by row, and the model is trained to filter out content relevant to the question. For the 20th row of table data, the constructed... for:

[0067] Name: Hammonds House Museum (Hammonds House Museum: It is located inthe 1857 Victorian house , former residence of Dr. Otis Thrash Hammonds , aprominent Atlanta physician and patron of the arts ;The Hammonds House Museumis a museum for African American fine art , located at 503 Peeples Street SWin the West End neighborhood of Atlanta , Georgia ; The Victorian house isbelieved to have been built in 1857 , and to be one of the three oldesthouses in West End .) , Area: West End (West End:West End is locatedsouthwest of Castleberry Hill , east of Westview , west of Adair ParkHistoric District , and just north of Oakland City ;West End is a historicneighborhood of Atlanta , one of the oldest outside Downtown Atlanta , and islisted on the National Register of Historic Places .), Type: AfricanAmerican, Summary: African American fine art , culture of the Africandiaspora ; located in a historic Queen Anne-style house

[0068] For the question in this table, the answer to which is "503 Peeples Street SW" is located. The row number containing the answer text is 20. Let the relevance of row 20 to the question be... =1, and lines 1-19 =0. Use the deviation loss calculation formula. Train the model until it is on the training dataset. It no longer decreases. The trained model outputs the top [number] items with the highest similarity. Okay. Take it. If =5, then the remaining table row numbers are as follows:

[0069] [20, 10, 3, 5, 18]

[0070] Finally, the question-answering module, which extracts fused data, is used to train the model on the selected data. Multi-hop inference is performed on the table content and text fusion data to extract the answer. For a given question, the content of rows 20, 10, 3, 5, and 18 of the corresponding table is used to construct the answer. This serves as a reference text for the question. The corresponding answer, "503 Peeples Street SW," has been pieced together. With word segmentation, the starting position of the answer and end position The values ​​are 119 and 124 respectively. The deviation loss calculation formula is used. Train the model until it is on the training dataset. It no longer decreases. The reference text is output using the trained model. , The largest position is used as the starting position of the answer in the reference text. and end position For a given problem, the starting position of the model output. and end position If the numbers are 119 and 124 respectively, then the model's predicted answer is "503 Peeples Street SW".

Claims

1.A method for constructing a multi-hop question and answer model based on table-text hybrid data, characterized in that, The method uses a table-text hybrid dataset including the following in the model construction process: Question q and its corresponding table data T = {H, D}, where H = {h1,..., hn} contains column names of the table and D = {r1,..., rm} contains m rows n columns data of the table, the i-th row data is denoted as r i = {d i1 ,..., d in}, and there are o corresponding entity title-text pairs P ij = {e ij1 : p ij1 ,..., e ijo : p ijo} in the cell of i-th row j-th column. n m i i1 in ij ij1 ij1 ijo ijo The constructed model includes a text content filtering module, a table row content screening module, and a fusion data extraction question and answer module; the text content filtering module is used to screen sentence segment content that is linked with cell content and related to the question in a unit of lines; the table row content screening module is used to construct table row-text fusion data in a unit of lines, train a table row content screening model, and further screen out content related to the question; the fusion data extraction question and answer module is used to train a fusion data extraction question and answer model in the remaining table row content-text fusion data and extract answers; wherein: The sentence segment content that is linked with cell content and related to the question is screened in a unit of lines, and the specific process is: A1: Split all the link texts of a table row data into sentence segments, calculate the similarity s of each sentence segment p with the question q ijk Similarity s of the text of the question q ijk ; The formula is as follows: p ijk-cat = [CLS] e ijk : p ijk [SEP] q cat = [CLS] q [SEP] u ijk = BERT(p ijk-cat ) v = BERT(q cat ) s ijk = u ijk · v wherein p ijk-cat denotes the sequence obtained by splicing the sentence segment text p ijk and the entity title e of the text where it is located ijk and special characters [CLS], [SEP]; q cat denotes the sequence obtained by splicing the question q and special characters [CLS], [SEP]; BERT denotes a pre-trained BERT encoding model; u ijk denotes p ijk-cat the vector obtained by taking the [CLS] character after BERT model encoding as a sentence vector; v denotes q cat the vector obtained by taking the [CLS] character after BERT model encoding as a sentence vector;. denotes vector dot product operation; A2: Sort the similarity of all text segments in the same row from large to small, and keep the top K segments with the highest similarity p ​ The table row-text fusion data is constructed in a unit of lines, a table row content screening model is trained, and content related to the question is further screened out, and the specific steps are: B1: construct each row of the table as a table-text fusion sequence r i-cat = c 1-cat ,..., c n-cat , where c j-cat is the concatenation of the jth cell content: if the cell contains text that is preserved by the previous step of filtering, then c j-cat = h j : d ij (p ij-cat ), otherwise c j-cat = h j : d ij ; text p ij-cat is the concatenation of the "entity title: text" key-value pairs; B2: Traverse all cell contents of each row of the table and the text contents, if there is an answer text in it, define that the row is related to the question, let the row data r i-cat The relevance s of the question q i = 1, otherwise s i = 0; B3: Calculate the i-th row data r using the table row content filtering model i-cat Relevance to question q The formula is as follows: q cat-r = [CLS] q [SEP] r i-cat [SEP] wherein q cat-r is the sequence obtained by splicing the question q and the fusion text r i-cat and special characters [CLS], [SEP]; FFN represents a linear operation; Softmax represents a softmax activation function; and respectively, are the positive and negative correlation degrees calculated for the i-th row of data; B4: Compute table row content filtering model output with label s i deviation loss L r ; the formula is as follows: where θ r is a table row content screening model parameter; B5: iterate through all question-text pairs in the training dataset and compute the loss, adjust the model parameters according to the loss until L r No longer reduced, get the final table row content screening model; B6: using the model obtained in step B5, calculate the positive correlation degree of all row data of the same table Sort from large to small, take the top K with the highest similarity r Line to be retained; The fusion data extraction question and answer model is trained in the remaining table row content-text fusion data and answers are extracted, and the specific process is: C1 : K r Row table row content - text fusion data is constructed as a sequence Reference text as question q; C2: find answer in sequence t, mark start position st and end position en, if not found then st = 0 and en = 0; let start probability p of answer start position be s (st) = 1, end probability p of end position e (en) = 1, p of remaining positions of sequence s , p e all 0; C3: use the fused data extraction and question answering model to compute the answer position of question q in reference t with The formula is as follows: q cat-t = [CLS] q [SEP] t [SEP] where q cat-t is the sequence obtained by concatenating the question q with the sequence t and the special characters [CLS], [SEP]; and TransformEncoder represents a TransformerEncoder encoding model; C4: computing the fused data extraction question answering model output with the true answer p s , p e loss L; the formula is as follows: L(θ a ) = (L s (θ a )+L e (θ a )) / 2 Where CrossEntropy is the CrossEntropy loss function, len(q) cat-t ) is q cat-t The number of word segments, θ a To extract parameters for the question-answering model from the fused data; C5: traverse all question-reference text pairs in the training dataset and calculate the loss, adjust the model parameters according to the loss until L no longer decreases, and obtain the final fusion data extraction question and answer model; C6: Using the model obtained in step C5, calculate the positions of the maximums in the reference text respectively as the start and end positions of the answer in the reference text extract the text from to as the answer to the question q.​

Images