Visualization processing method and device based on attention mechanism and storage medium
By adopting a visual processing method based on attention mechanism in the question-and-answer system, the problem of insufficient accuracy and generalization ability when highlighting answers is solved, and higher semantic matching accuracy and better ability are achieved Explanatory.
Patent Information
- Application Number
- CN202310935608.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-27
- Publication Date
- 2025-02-07
AI Technical Summary
When highlighting the parts of the answers related to the questions raised by users, existing question answers have problems with insufficient accuracy and generalization capabilities, making it difficult to deal with problems and articles in different fields and styles.
A visual processing method based on attention mechanism is adopted, the original text and the question are spliced and combined, and inputted into the question-and-answer model, answer information and attention matrix are generated, attention matrix within the focus range is calculated, heat maps are generated and visualized.
It improves the accuracy of semantic matching between the answer and the original text, enhances the generalization ability of the system, can better deal with problems and articles in different fields and styles, and provides better interpretability.
Smart Images
Figure CN119396963A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a visualization processing method and apparatus based on an attention mechanism, a terminal device, and a storage medium. Background Art
[0002] The big model question answering system is a natural language processing system based on deep neural network technology, which is used to answer questions raised by users. It is based on a large-scale language model trained with deep learning technology and a large-scale corpus, which enables the machine to answer questions raised by users in a way that humans can understand. Compared with traditional question answering systems, the big model question answering system uses more complex deep learning algorithms and larger data sets for training, so it can handle more complex questions and provide more accurate and comprehensive answers. The big model question answering model is widely used, such as search engines, intelligent customer service, virtual assistants, etc.
[0003] In a large-scale question-answering system based on a document library, it is usually necessary to highlight the parts of the answer that are directly related to the question asked by the user in different colors or other ways. Because when a user enters a question and gets a long answer, they may spend a lot of time and energy to find the truly useful information. Highlighting can help users find keywords or phrases related to the questions they care about more quickly, help users get faster and more accurate answers, enhance users' trust and satisfaction with the question-answering system, and deepen users' understanding and memory of information. In addition, highlighting can also make the answer look more professional and easy to read, and users can more easily understand the main points and key points of the answer, leaving a good impression on users. Summary of the invention
[0004] The present invention provides a visualization processing method, device and storage medium based on an attention mechanism.
[0005] A first aspect of an embodiment of the present disclosure provides a visualization processing method based on an attention mechanism, which is applied to an electronic device. The method includes:
[0006] The original text C is combined with the question Q to obtain the first input information;
[0007] Input the first input information into the question-answering model to obtain answer information A, result information T of the word segmentation answered by the question-answering model, and attention matrix Atten_m;
[0008] Given a focus range, calculating an attention matrix within the focus range;
[0009] Based on the attention matrix within the focus range, a heat map is calculated and visualized on the result information T after the question-answering model answers the word segmentation.
[0010] Based on the above scheme, the first input information is input into the question-answering model to obtain the answer information A, the result information T of the word segmentation of the question-answering model, and the attention matrix Atten_m, including:
[0011] The first input information is segmented to obtain second input information I with a length of Li.
[0012] Based on the above scheme, the first input information is input into the question-answering model to obtain the answer information A, the result information T of the word segmentation of the question-answering model, and the attention matrix Atten_m, including:
[0013] Input the second input information I into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after word segmentation of the first input information, and the suffix is the lexicalized sequence of the answer information A, with a length of La, and the length of T is the sum of Li and La.
[0014] Based on the above scheme, the first input information is input into the question-answering model to obtain the answer information A, the result information T of the word segmentation of the question-answering model, and the attention matrix Atten_m, including:
[0015] The attention mechanism in the question-answering model includes N layers and H heads. The dimension of the attention matrix is [N, H, Lt, Lt]. The elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m. The dimension of the attention matrix Atten_m is [Lt, Lt].
[0016] Based on the above scheme, given a focus range, calculating the attention matrix within the focus range includes:
[0017] The text range [Li+1, Lt] of the answer part or the text range [Ls, Le] of the custom selected part is used as the focus range, wherein the answer part is the content of the result information T;
[0018] Define a focus vector V, whose length is Lt, wherein the elements within the focus range are 1, and the elements in other ranges are 0;
[0019] The focus vector is multiplied by the attention matrix to obtain the attention matrix within the focus range.
[0020] Based on the above scheme, the heat map is calculated based on the attention matrix within the focus range, and visualization is performed on the result information T after the word segmentation of the question-answering model answer, including:
[0021] The attention matrix based on the focus range is matched with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
[0022] A second aspect of an embodiment of the present disclosure provides a visualization processing device based on an attention mechanism, which is applied to an electronic device, and the device includes:
[0023] A concatenation module, used for concatenating the original text C with the question Q to obtain first input information;
[0024] A first processing module is used to input the first input information into the question-answering model to obtain answer information A, result information T of the word segmentation answered by the question-answering model, and an attention matrix Atten_m;
[0025] A focusing module, configured to calculate an attention matrix within a given focusing range;
[0026] The second processing module is used to calculate a heat map based on the attention matrix within the focus range, and perform visualization processing on the result information T after the question-answering model answers the word segmentation.
[0027] Based on the above solution, the first processing module is specifically used to perform word segmentation processing on the first input information to obtain second input information I with a length of Li.
[0028] Based on the above scheme, the first processing module is specifically used to input the second input information I into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after word segmentation of the first input information, and the suffix is the lexicalized sequence of the answer information A, with a length of La, and the length of T is the sum of Li and La.
[0029] Based on the above scheme, the first processing module, the attention mechanism used in the question-answering model includes N layers and H heads, the dimension of the attention matrix is [N, H, Lt, Lt], and the elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m, and the dimension of the attention matrix Atten_m is [Lt, Lt].
[0030] Based on the above scheme, the focusing module is specifically used to take the text range [Li+1, Lt] of the answer part, or the text range [Ls, Le] of the custom selected part as the focusing range, wherein the answer part is the content of the result information T; define a focusing vector V with a length of Lt, wherein the elements within the focusing range are 1, and the elements in other ranges are 0; multiply the focusing vector by the attention matrix to obtain the attention matrix within the focusing range.
[0031] Based on the above scheme, the second processing module is specifically used to correspond the attention matrix based on the focus range with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
[0032] A third aspect of an embodiment of the present disclosure provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a computer, the computer is enabled to execute a visualization processing method based on an attention mechanism as described in any one of the above items.
[0033] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:
[0034] On the one hand, the disclosed embodiments can find the correspondence between the answer and the original text in the question-answering model, and highlight the original text through the attention weight of the answer and the original question-answering, so as to intuitively show which words play an important role in the matching process, and facilitate the explanation and interpretability of the answer. On the other hand, compared with the rule matching and similarity retrieval-based methods, the method of matching the answer and the original text based on the attention mechanism has the following advantages:
[0035] Higher accuracy: The attention-based method can dynamically calculate the importance of each word and more accurately capture the semantic relationship between the answer and the original text. Compared with rule matching and similarity-based retrieval methods, it can achieve higher matching accuracy.
[0036] Better generalization ability: The attention-based method does not rely on static matching templates or preset rules. It can handle problems and articles in different fields and styles and has better generalization ability.
[0037] Adaptive learning: The attention-based method can adaptively learn the weight of each word, give reasonable matching results for original texts and answers of different lengths, and adapt to changes in the data set.
[0038] Better explainability: The attention-based method can generate an attention distribution map, which can intuitively show which words play an important role in the matching process, facilitating the explanation and explainability of the answer. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0040] Figure 1 is a flowchart of a visualization processing method based on an attention mechanism according to an exemplary embodiment;
[0041] Figure 2 is a flowchart of a visualization processing method based on an attention mechanism according to an exemplary embodiment;
[0042] Figure 3 It is a flowchart of a visualization processing method based on an attention mechanism according to an exemplary embodiment. DETAILED DESCRIPTION
[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.
[0044] The answer can be matched with the original text based on the attention mechanism because the attention mechanism can solve the dependencies between the elements in the sequence, so that the model can better understand and process the input text.
[0045] Through the attention mechanism, the model can weight each element in the input sequence according to the current query vector (i.e., question), focusing on the parts related to the query, thereby generating a weighted vector, which can be used as a representation of the answer.
[0046] Specifically, in the question-answering model based on the attention mechanism, for each question, the model calculates each position in the input sequence, converts it into an attention weight, and then multiplies these weights with the elements in the input sequence and sums them to obtain a weighted vector that represents the most relevant part of the current question. Then, the model uses this weighted vector as a representation of the answer and converts it into a probability distribution through the softmax function to obtain the final answer.
[0047] In order to find the original text corresponding to the answer, existing question-answering systems include methods based on keyword matching, which uses rules such as regular expressions to search for keywords related to the answer in the original text, and then partially highlights these keywords; methods based on similarity retrieval use retrieval models to search for original text paragraphs related to the question, and highlight the parts that may contain the answer as a whole.
[0048] The existing methods have the following disadvantages:
[0049] 1. Unable to cover all questions: Highlighting based on keyword matching can only be matched according to pre-specified keywords. If no relevant keywords are set in advance, or the questions raised by users do not involve these keywords, the answers cannot be highlighted.
[0050] 2. Low matching accuracy: Keyword-based matching methods may mistake irrelevant content as relevant, or ignore relevant content. For example, when a keyword appears in a context that is irrelevant to the question, it will also be marked as a keyword, resulting in distorted answer highlighting.
[0051] 3. Difficulty in handling synonyms and language differences: Different people use different terms and languages to express the same concept. In addition, the same language may also differ in different regions. These factors will make the selection and matching of keywords more complicated, thus affecting the accuracy of answer highlighting.
[0052] Compared with rule matching and similarity-based retrieval methods, the method of matching answers with original texts based on the attention mechanism has the following advantages:
[0053] Higher accuracy: The attention-based method can dynamically calculate the importance of each word and more accurately capture the semantic relationship between the answer and the original text. Compared with rule matching and similarity-based retrieval methods, it can achieve higher matching accuracy.
[0054] Better generalization ability: The attention-based method does not rely on static matching templates or preset rules. It can handle problems and articles in different fields and styles and has better generalization ability.
[0055] Adaptive learning: The attention-based method can adaptively learn the weight of each word, give reasonable matching results for original texts and answers of different lengths, and adapt to changes in the data set.
[0056] Better explainability: The attention-based method can generate an attention distribution map, which can intuitively show which words play an important role in the matching process, facilitating the explanation and explainability of the answer.
[0057] like Figure 1 As shown, an embodiment of the present disclosure provides a visualization processing method based on an attention mechanism, which is applied to an electronic device, and the method includes:
[0058] S110: Concatenate the original text C and the question Q to obtain first input information;
[0059] S120: Input the first input information into the question-answering model to obtain answer information A, result information T of the word segmentation answered by the question-answering model, and attention matrix Atten_m;
[0060] S130: Given a focus range, calculating an attention matrix within the focus range;
[0061] S140: Based on the attention matrix within the focus range, a heat map is calculated and visualized on the result information T after the question-answering model answers the word segmentation.
[0062] The electronic device herein may also be referred to as a client. The electronic device may be various electronic devices. Exemplarily, the electronic device may include various smart home devices, wearable devices, tablet computers, smart office devices, or mobile phones.
[0063] For example, the above first input information may be: "Original text: The 2008 Summer Olympics torch relay was run from March 24 until August 8, 2008, prior to the 2008 Summer Olympics, with the theme of "one world, one dream". Plans for the relay were announced on April 26, 2007, in Beijing, China. The relay, also called by the organizers as the "Journey of Harmony'lasted 129days and carried the torch137,000km(85,000mi)--the longest distance of any Olympictorch relay since the tradition was started ahead of the 1936Summer Olympics. Question: What was the theme of the 2008Summer Olympics? "
[0064] The method provided by the embodiment of the present disclosure can also be called a method for matching answers with original texts based on an attention mechanism.
[0065] In some embodiments, the first input information is input into the question-answering model to obtain answer information A, result information T of the word segmentation of the question-answering model, and attention matrix Atten_m, including:
[0066] The first input information is segmented to obtain second input information I with a length of Li.
[0067] Wherein, Li is a positive integer greater than 0;
[0068] Exemplarily, the above-mentioned word segmentation processing may be tokenize processing.
[0069] Exemplarily, Tokenize: lexicalization processing, for example, "apple" becomes "app@@" and "le" after tokenization.
[0070] In some embodiments, the first input information is input into the question-answering model to obtain answer information A, result information T of the word segmentation of the question-answering model, and attention matrix Atten_m, including:
[0071] The second input information I is input into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after the first input information is segmented, and the suffix is the lexicalized sequence of the answer information A, and the length is La, and the length of T is the sum of Li and La. Wherein Li and La are both positive integers.
[0072] In some embodiments, the first input information is input into the question-answering model to obtain answer information A, result information T of the word segmentation of the question-answering model, and attention matrix Atten_m, including:
[0073] The attention mechanism in the question-answering model includes N layers and H heads, and the dimension of the attention matrix is [N, H, Lt, Lt]. The elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m, and the dimension of the attention matrix Atten_m is [Lt, Lt]. Where N, H, and Lt are all positive integers.
[0074] Specifically, the elements corresponding to the N layers and H heads of the attention matrix are averaged, that is, the first two dimensions are averaged, and the corresponding dimensions are converted to 1 dimension before dimensionality reduction.
[0075] In some embodiments, Figure 2 As shown, the S130 includes:
[0076] S131: taking the text range [Li+1, Lt] of the answer part or the text range [Ls, Le] of the custom selected part as the focus range, wherein the answer part is the content of the result information T;
[0077] S132: defining a focus vector V, with a length of Lt, wherein the elements within the focus range are 1, and the elements in other ranges are 0;
[0078] S133: Multiply the focus vector by the attention matrix to obtain the attention matrix within the focus range.
[0079] The above Ls and Le are both positive integers.
[0080] In some embodiments, the heat map is calculated based on the attention matrix within the focus range, and visualization is performed on the result information T after the word segmentation of the question-answering model answer, including:
[0081] The attention matrix based on the focus range is matched with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
[0082] In one embodiment, step 1: concatenate the original text C and the question Q as the first input information and input it into the question-answering model.
[0083] First input information: "Original text: The 2008 Summer Olympics torch relay was run from March 24 until August 8, 2008, prior to the 2008 Summer Olympics, with the theme of "one world, one dream". Plans for the relay were announced on April 26, 2007, in Beijing, China. The relay, also called by the organizers as the "Journey of Harmony' lasted 129 days and carried the torch137,000km(85,000mi)--the longest distance of any Olympic torch relay since the tradition was started ahead of the 1936Summer Olympics. Question: What was the theme of the 2008Summer Olympics? "
[0084] Step 2: Input the first input information into the model to calculate the model output answer A, the result T after word segmentation of the model answer, and the attention matrix Atten_m.
[0085] Step 2.1: Tokenize the first input information to obtain input I with a length of Li.
[0086] Tokenize: Lexicalization processing, for example, "apple" becomes "app@@" and "le" after tokenization.
[0087] Step 2.2: Input the input I into the model to obtain the model output T and the attention matrix, where the prefix of T is the lexicalized sequence I after the input word segmentation, and the suffix is the lexicalized sequence A of the answer output by the model, with a length of La. Then the length of T is Lt=Li+La.
[0088] The output of the model includes the first input information, that is, the original text + question, and the answer output by the model.
[0089] Step 2.3: Since the attention mechanism in the model contains N layers and H heads, the original attention matrix dimension is [N,H,Lt,Lt]. Take the average of the elements of the N layers and H heads of the attention matrix to get the new attention matrix Atten_m with dimension [Lt,Lt].
[0090] Take the mean of the elements corresponding to the N layers and H heads of the attention matrix respectively, that is, take the mean of the first two dimensions respectively, and the corresponding dimensions are converted to 1 dimension before dimensionality reduction.
[0091] Step 3: Given the focus range, calculate the attention matrix within the focus range.
[0092] Step 3.1: Given the attention matrix Atten_m, take the text range [La, Lt] of the answer part or the text range [Ls, Le] of the custom selected part as the focus range.
[0093] Step 3.2: Define a focus vector V with a length of Lt. The elements within the focus range are 1, and the elements in other ranges are 0.
[0094] Step 3.3: Multiply the focus vector by the attention matrix to get the attention matrix within the range.
[0095] Step 4: Based on the attention matrix within the range, calculate the heat map and visualize it on the model’s answer after word segmentation.
[0096] Step 4.1: Match the range-based attention matrix with the model output result T, where the higher the element value of the matrix, the darker the color of the corresponding word block.
[0097] like Figure 3 As shown, the embodiment of the present disclosure provides a visualization processing device 200 based on an attention mechanism, which is applied to an electronic device, and the device includes:
[0098] A concatenation module 210 is used to concatenate the original text C with the question Q to obtain first input information;
[0099] The first processing module 220 is used to input the first input information into the question-answering model to obtain answer information A, result information T of the word segmentation of the question-answering model, and attention matrix Atten_m;
[0100] A focusing module 230, configured to calculate an attention matrix within a given focusing range;
[0101] The second processing module 240 is used to calculate a heat map based on the attention matrix within the focus range, and perform visualization processing on the result information T after the question-answering model answers the word segmentation.
[0102] Based on the above solution, the first processing module is specifically used to perform word segmentation processing on the first input information to obtain second input information I with a length of Li.
[0103] Based on the above scheme, the first processing module is specifically used to input the second input information I into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after word segmentation of the first input information, and the suffix is the lexicalized sequence of the answer information A, with a length of La, and the length of T is the sum of Li and La.
[0104] Based on the above scheme, the first processing module, the attention mechanism used in the question-answering model includes N layers and H heads, the dimension of the attention matrix is [N, H, Lt, Lt], and the elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m, and the dimension of the attention matrix Atten_m is [Lt, Lt].
[0105] Based on the above scheme, the focusing module is specifically used to take the text range [Li+1, Lt] of the answer part, or the text range [Ls, Le] of the custom selected part as the focusing range, wherein the answer part is the content of the result information T; define a focusing vector V with a length of Lt, wherein the elements within the focusing range are 1, and the elements in other ranges are 0; multiply the focusing vector by the attention matrix to obtain the attention matrix within the focusing range.
[0106] Based on the above scheme, the second processing module is specifically used to correspond the attention matrix based on the focus range with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
[0107] The embodiment of the present disclosure provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the UE, the UE or the base station can execute the visualization processing method based on the attention mechanism provided by any of the above embodiments, and can execute Figure 1 , Figure 2 and Figure 3 At least one of any of the methods shown.
[0108] The attention-based visualization processing method applied to electronic devices may include:
[0109] The original text C is combined with the question Q to obtain the first input information;
[0110] Input the first input information into the question-answering model to obtain answer information A, result information T of the word segmentation answered by the question-answering model, and attention matrix Atten_m;
[0111] Given a focus range, calculating an attention matrix within the focus range;
[0112] Based on the attention matrix within the focus range, a heat map is calculated and visualized on the result information T after the question-answering model answers the word segmentation.
[0113] The electronic device herein may also be referred to as a client. The electronic device may be various electronic devices. Exemplarily, the electronic device may include various smart home devices, wearable devices, tablet computers, smart office devices, or mobile phones.
[0114] It can be understood that the first input information is input into the question-answering model to obtain the answer information A, the result information T of the word segmentation of the question-answering model, and the attention matrix Atten_m, including:
[0115] The first input information is segmented to obtain second input information I with a length of Li.
[0116] It can be understood that the first input information is input into the question-answering model to obtain the answer information A, the result information T of the word segmentation of the question-answering model, and the attention matrix Atten_m, including:
[0117] Input the second input information I into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after word segmentation of the first input information, and the suffix is the lexicalized sequence of the answer information A, with a length of La, and the length of T is the sum of Li and La.
[0118] It can be understood that the first input information is input into the question-answering model to obtain the answer information A, the result information T of the word segmentation of the question-answering model, and the attention matrix Atten_m, including:
[0119] The attention mechanism in the question-answering model includes N layers and H heads. The dimension of the attention matrix is [N, H, Lt, Lt]. The elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m. The dimension of the attention matrix Atten_m is [Lt, Lt].
[0120] It can be understood that, given a focus range, calculating the attention matrix within the focus range includes:
[0121] The text range [Li+1, Lt] of the answer part or the text range [Ls, Le] of the custom selected part is used as the focus range, wherein the answer part is the content of the result information T;
[0122] Define a focus vector V, whose length is Lt, wherein the elements within the focus range are 1, and the elements in other ranges are 0;
[0123] The focus vector is multiplied by the attention matrix to obtain the attention matrix within the focus range.
[0124] It can be understood that the heat map is calculated based on the attention matrix within the focus range, and the result information T after the word segmentation of the question-answering model is visualized, including:
[0125] The attention matrix based on the focus range is matched with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
[0126] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0127] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A visualization processing method based on attention mechanism, characterized in that: Applied in an electronic device, the method comprises: The original text C is combined with the question Q to obtain the first input information; Input the first input information into the question-answering model to obtain answer information A, result information T output by the question-answering model, and attention matrix Atten_m; Given a focus range, calculating an attention matrix within the focus range; Based on the attention matrix within the focus range, a heat map is calculated and visualized on the result information T output by the question-answering model.
2. The method according to claim 1, characterized in that: The first input information is input into the question-answering model to obtain answer information A, result information T output by the question-answering model, and attention matrix Atten_m, including: The first input information is segmented to obtain second input information I with a length of Li.
3. The method according to claim 2, characterized in that The first input information is input into the question-answering model to obtain answer information A, result information T output by the question-answering model, and attention matrix Atten_m, including: Input the second input information I into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after word segmentation of the first input information, and the suffix is the lexicalized sequence of the answer information A, with a length of La, and the length of T is the sum of Li and La.
4. The method according to claim 3, characterized in that The first input information is input into the question-answering model to obtain answer information A, result information T of the word segmentation answered by the question-answering model, and attention matrix Atten_m, including: The attention mechanism in the question-answering model includes N layers and H heads. The dimension of the attention matrix is [N, H, Lt, Lt]. The elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m. The dimension of the attention matrix Atten_m is [Lt, Lt].
5. The method according to claim 4, characterized in that Given a focus range, calculating an attention matrix within the focus range includes: The text range [Li+1, Lt] of the answer part or the text range [Ls, Le] of the custom selected part is used as the focus range, wherein the answer part is the content of the result information T; Define a focus vector V, whose length is Lt, wherein the elements within the focus range are 1, and the elements in other ranges are 0; The focus vector is multiplied by the attention matrix to obtain the attention matrix within the focus range.
6. The method according to claim 1, characterized in that The heat map is calculated based on the attention matrix within the focus range, and visualization is performed on the result information T after the word segmentation of the question-answering model answer, including: The attention matrix based on the focus range is matched with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
7. A visualization processing device based on an attention mechanism, characterized in that: Applied in electronic equipment, the device comprises: A concatenation module, used for concatenating the original text C with the question Q to obtain first input information; A first processing module is used to input the first input information into the question-answering model to obtain answer information A, result information T of the word segmentation answered by the question-answering model, and an attention matrix Atten_m; A focusing module, configured to calculate an attention matrix within a given focusing range; The second processing module is used to calculate a heat map based on the attention matrix within the focus range, and perform visualization processing on the result information T after the question-answering model answers the word segmentation.
8. The device according to claim 7, characterized in that The first processing module is specifically used to perform word segmentation processing on the first input information to obtain second input information I with a length of Li.
9. The device according to claim 8, characterized in that The first processing module is specifically used to input the second input information I into the question-answering model to obtain the result information T and the attention matrix Atten_m; wherein the prefix of T is the lexicalized sequence I after word segmentation of the first input information, and the suffix is the lexicalized sequence of the answer information A, with a length of La, and the length of T is the sum of Li and La.
10. The device according to claim 9, characterized in that The first processing module, which is used for the attention mechanism in the question-answering model, includes N layers and H heads, and the dimension of the attention matrix is [N, H, Lt, Lt]. The elements of the N layers and H heads of the attention matrix are averaged to obtain the attention matrix Atten_m, and the dimension of the attention matrix Atten_m is [Lt, Lt].
11. The device according to claim 10, characterized in that The focusing module is specifically used to use the text range [Li+1, Lt] of the answer part, or the text range [Ls, Le] of the custom selected part as the focusing range, wherein the answer part is the content of the result information T; define a focusing vector V with a length of Lt, wherein the elements within the focusing range are 1, and the elements in other ranges are 0; multiply the focusing vector by the attention matrix to obtain the attention matrix within the focusing range.
12. The device according to claim 7, characterized in that The second processing module is specifically used to correspond the attention matrix based on the focus range with the question-answering model output result information T, wherein the higher the element value of the matrix, the darker the color of the word block corresponding to the element.
13. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a computer, enables the computer to perform the visualization processing method based on the attention mechanism as described in any one of claims 1 to 6.