Method for generating answer template according to user keywords under inquiry condition

Abstract

The invention relates to the field of natural language processing, and provides a method for generating an answer template according to user keywords under an inquiry condition. The invention aims to solve the problem that a common template matching method cannot provide an accurate and personalized template reply corresponding to inquiry information input by a user. According to the main scheme, the method comprises the following steps: generating a keyword positive-order text, encoding an inquiry statement keyword text input by a user, and inputting the encoded inquiry statement keyword text into a rightward text generator to obtain a rightward text; generating a keyword reverse order text, performing reverse order arrangement on the inquiry statement keyword text input by the user, and inputting the inquiry statement keyword text into a left text generator after encoding to obtain a left text; text fusion: splicing and fusing the right text and the left text to obtain a fused text; and generating a dynamic template, encoding the fused text and inputting the encoded fused text into a template generator for fine tuning to obtain a generated dynamic inquiry template.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, and provides a method for generating an answer template according to a user keyword under the condition of medical inquiry. BACKGROUND

[0002] Technical background (NLP algorithm relied on by the current work)

[0003] The current work needs to input the extracted medical inquiry keywords into the trained natural language model for processing, and generate a corresponding pre-medical inquiry answer for the medical inquiry question raised by the user, so the main technical direction is template generation. The purpose of template generation is to extract features from the extracted medical inquiry keywords, transfer the features to the decoder to generate corresponding answers that meet the conditions, including generating right and left templates starting from the keyword, and inputting the generated sequence into a new model for correction. The technology used in this process is transformer.

[0004] Background (solution strategy of existing work)

[0005] The general solution to this problem is template matching. The process of template matching can be described as follows: first, template definition is performed, one or more templates need to be defined, and the templates are generally represented by regular expressions, syntax trees or other forms. A template can be a word, a phrase, a sentence or a longer text segment. Then, template matching is performed. After the defined template is obtained, it can be applied to a text to find a text segment that meets the template definition. Template matching usually uses string matching, syntax analysis or other forms for matching.

[0006] The focus of this solution is on the definition and matching strategy of the template, that is, which method is used to define the template, and which method is used to match the defined template according to the keyword.

[0007] Technical problems to be solved by the present application (main innovation of the current work)

[0008] The current work mainly solves the problem of dynamic template generation under the condition of medical inquiry. Dynamic template generation here refers to generating an answer template according to the medical keywords raised by the user, rather than mechanically matching the existing template. The current work mainly solves the problem that the common template matching method cannot provide accurate and personalized template replies corresponding to the user input medical inquiry information. By training the medical inquiry keywords extracted from the user data and the corresponding doctor answers, the model can learn the generation method from keywords to sentences. For the medical inquiry field, the generation method is to generate a question that simulates the doctor's questioning of the patient. SUMMARY

[0009] The purpose of this invention is to solve the problem that common template matching methods cannot provide accurate and personalized template responses that correspond to the user's input consultation information.

[0010] To achieve the above objectives, the present invention employs the following technical means:

[0011] A technology for generating answer templates based on user keywords includes the following steps:

[0012] Step 1: Generating keyword-ordered text

[0013] The keywords in the user's consultation statement are encoded and input into the right-direction text generator to obtain the right-direction text.

[0014] Step 2: Keyword Reverse Text Generation

[0015] The keywords of the user's consultation statement are sorted in reverse order, encoded, and then input into the left-direction text generator to obtain the left-direction text.

[0016] Step 3: Text Fusion

[0017] By concatenating and merging the right-hand text and the left-hand text, the merged text is obtained.

[0018] Step 4: Dynamic Template Generation

[0019] The merged text is encoded and input into a template generator for fine-tuning to obtain a generated dynamic consultation template.

[0020] Step 1 above specifically includes the following steps:

[0021] Step 1.1: Perform word embedding encoding on keywords in the consultation statements.

[0022] For the characteristic keywords of the consultation statement entered by the user, first perform an embedding encoding operation to obtain the embedding encoding X. r The encoding process extracts user-defined keywords from the user's input and feeds them into the model. These keywords are then segmented into individual characters to obtain segmented keywords, allowing for the extraction of relevant features between them after inputting into the encoder. The segmented keywords are then bidirectionally mapped to unique numeric IDs using an index dictionary. These numeric IDs are then mapped to 512-dimensional high-dimensional vectors for subsequent input into the model. In other words, the segmented n keywords become:

[0023] [x1, x2, x3, ..., x i ], where x i Let i be the 512-dimensional vector mapped to the i-th keyword;

[0024] Since the transformer performs parallel operations on keywords in a sentence, in order to preserve the sequential structure of words in the sentence, it is necessary to calculate the positional encoding corresponding to each segmented keyword. The positional encoding can extract positional information in the sentence structure. Then, the positional encoding is fused with the 512-dimensional vector corresponding to the keyword. While preserving the semantics, sentence structure information is added to the encoding. For the pos-th character, the even-numbered and odd-numbered elements of the pos-th character are encoded as follows:

[0025]

[0026] In the above formula, i represents the position code of the i-th element corresponding to the pos-th keyword, and dmodel represents the total number of dimensions;

[0027] Location encoding PE r Compared with the previously constructed 512-dimensional Embedding encoding X r The word embedding vector V is obtained by performing addition and fusion. r This refers to the vector encoding input into the model:

[0028] V r =X r +PE r

[0029] Step 1.2: Input the obtained features into the right encoder to extract features;

[0030] After word embedding is completed, the word embedding vector V is... r The input is fed into the right-hand template encoding generator, which generates encoding from the keyword to the right, embedding the word into the vector V. r The input is fed into the encoder of the right-hand template generation model, which uses a transformer model.

[0031] The model's encoder performs matrix transformations through a neural network, converting the input word embedding vector V into a matrix. r These are mapped to three matrices, Q, K, and V, respectively. Since matrices Q, K, and V are derived from V... r Since they are mapped, all three matrices contain the semantic and structural information of the original sentence. This process can be viewed as mapping the original vector V. r Perform feature extraction;

[0032] By multiplying matrices Q and K, words at corresponding positions can be linked to words at other positions. Specifically, when matrices Q and K are multiplied, each element in matrix Q is multiplied with an element in matrix K, and the result is the similarity between the two multiplied elements. Therefore, multiplying matrices Q and K is actually calculating the similarity between all words in the sentence and other words in parallel, resulting in a structural information matrix that contains the structural information of the sentence itself.

[0033] The structural information matrix is ​​scaled and a softmax operation is performed to map each element to the interval [0, 1]. Then, the V matrix is ​​multiplied, which means that the structural information between sentences is appended to the original word positions. The result is subjected to residual operation and passed to the feedforward neural network to obtain the feature vector, which is sent to the right decoder for text generation in step 1.3.

[0034]

[0035] Step 1.3: Input the feature vector obtained in Step 1.2 into the right-hand decoder for template generation;

[0036] The feature vectors after SelfAttention are sent to the right-hand decoder for template generation. In the decoder, the data is first generated by MaskedAttention to obtain the generated data information. Similar to SelfAttention, this allows the sentence to be generated from front to back. The difference is that in order to avoid reading the information after the generation, an upper triangular matrix is ​​used to mask the data after the generated characters.

[0037] The structural information obtained in step 1.2 is then input together with the generated data information after MaskedAttention into a new SelfAttention to decode the data. The generated vector is then processed as a residual, input into a feedforward neural network, and then into softmax for output, resulting in right-hand text.

[0038] S r = softmax(W(SelfAttention(Q)) D K E V E )+X)+b)

[0039] This formula represents the combination of the vector after SelfAttention and the original word embedding vector V. rThe residue is processed by addition, and then the output vector is input into the neural network, where W is the training weight of the neural network and b is the bias value of the neural network. The output of the neural network is then activated by softmax to obtain the final right-directed text sequence S. r .

[0040] Step 2 above specifically includes the following steps:

[0041] Step 2.1: Sort the characteristic keywords of the user-input statement in reverse order.

[0042] Reversing the order of user-input keywords preserves the original structure and relative positions while facilitating subsequent reverse text generation.

[0043] Step 2.2: Perform word embedding encoding on the reverse-order keywords;

[0044] Similar to step 1.1, this process performs embedding encoding and positional encoding on the reversed keywords, and finally adds the two to complete the word embedding operation, obtaining the word embedding vector V. l .

[0045] V l =X l +PE l

[0046] Step 2.3: Input the obtained features into the left encoder to extract features;

[0047] The information is input into the left-hand template generation encoding, which maps the encoding vector V into three matrices Q, K, and V. By performing operations on the Q and K matrices, the words at the corresponding positions can be connected with other positions. Then, the structure is scaled and a softmax operation is performed, and then multiplied with the V matrix to append the structural information between sentences to the original word positions. The residual operation is then performed and passed to the feedforward neural network, which is then sent to the decoder for subsequent text generation.

[0048]

[0049] Step 2.4: Input the obtained features into the left-hand decoder for template generation.

[0050] The feature vector is fed into the right-hand decoder for template generation. In the decoder, the data is first generated through MaskedAttention, similar to SelfAttention, except that the process is generated from front to back. To avoid reading subsequent information, an upper triangular matrix is ​​used to mask the data after the generated characters.

[0051] The encoded structural information and the generated data information after MaskedAttention are then input into a new SelfAttention to decode the data. The generated vector is then processed as a residual, input into a feedforward neural network, and then into a softmax for output, resulting in the left-directed text.

[0052] S l = softmax(W(SelfAttention(Q)) D K E V E )+X)+b)

[0053] This formula represents the combination of the vector after SelfAttention and the reverse word embedding vector V. l The residues are added together and processed, then the resulting vector is input into a neural network, where W is the training weight of the neural network and b is the bias value of the neural network. The output of the neural network is then activated by softmax to obtain the final left-directed text sequence S. l .

[0054] Step 3 above specifically includes the following steps:

[0055] Step 3.1: In order for the model to have the ability to generate in reverse, the order of the sentences generated from the left needs to be reversed, which is the result of the model's left generation.

[0056] S l =[S n S n-1 S n-2 S1]

[0057] Step 3.2: Concatenate the sentences generated from the left and the sentences generated from the right.

[0058] S = S l +S r

[0059] Step 4 above specifically includes the following steps:

[0060] Step 4.1: Perform word embedding processing on the fused text S

[0061] This process is consistent with step 1.1, and requires processing the text S = [S1, S2, S3, ..., S...]. n Perform embedding encoding and location encoding (PE) corresponding to the new location information. new (pos), and finally add the two together to complete the word embedding operation, and obtain the word embedding vector V.

[0062] V = S + PE new (pos)

[0063] Step 4.2: Input the obtained word embedding vector V into the fine-tuning encoder to extract features;

[0064] Similar to step 1.2, the information is input into the left-directed template generation encoding, and the encoding vector V is mapped into three matrices Q, K, and V. By performing operations on matrices Q and K, the words at the corresponding positions can be connected with other positions. Then, the structure is scaled and a softmax operation is performed, and then multiplied with matrix V to append the structural information between sentences to the original word positions. The residual operation is then performed and passed to the feedforward neural network, and finally sent to the decoder for subsequent text generation.

[0065]

[0066] Step 4.3: Input the obtained features into the left-hand decoder for template generation.

[0067] The feature vector is fed into the right-hand decoder for template generation. In the decoder, the data is first generated through MaskedAttention, similar to SelfAttention, except that the process is generated from front to back. To avoid reading subsequent information, an upper triangular matrix is ​​used to mask the data after the generated characters.

[0068] The encoded structural information and the generated data information after MaskedAttention are then input into a new SelfAttention to decode the data. The generated vector is then processed as a residual and input into a feedforward neural network before being fed into softmax for output.

[0069] S = softmax(W(SelfAttention(Q)) D K E V E )+X)+b)

[0070] At this point, the model outputs the final, finely tuned dynamic template.

[0071] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0072] 1. This invention solves the problem that template matching methods cannot fully extract keyword structural information by means of step 1 (generating keyword text in ascending order) and step 2 (generating keyword text in descending order), and achieves the effect of generating more accurate templates by utilizing bidirectional syntactic structure.

[0073] In step 1, the keywords input by the user are encoded in forward order (including word embedding and positional encoding) and input into a right-hand text generator (based on a transformer model). The similarity between words is calculated in parallel using the SelfAttention mechanism to extract positive syntactic structure information (such as semantic association and positional dependency). In step 2, the keywords are encoded in reverse order and input into a left-hand text generator to extract negative syntactic structure information. Traditional template matching relies solely on predefined rules and cannot dynamically capture the contextual relationships between keywords, resulting in inaccurate response templates. This invention, however, uses bidirectional generation, where the model simultaneously learns both positive and negative structures (e.g., positive generation captures "start-end" relationships, and negative generation captures "end-start" relationships), thereby extracting keyword features more comprehensively (such as the symptom sequence logic in medical consultations). This solves the problem of inaccurate responses caused by template matching ignoring structural information, making the generated templates more closely match user input and improving the accuracy of personalized responses.

[0074] 2. This invention solves the problem that the one-way generated template may miss some semantic information by means of the technical means of step 3 (text fusion), and achieves the effect of fusion of two-way text to retain the complete response content.

[0075] In step 3, the right-directed text (forward structure) generated in step 1 and the left-directed text (reverse structure) generated in step 2 are reversed and merged (e.g., the left-directed text is reversed and then connected to the right-directed text). Traditional methods only generate templates in one direction (e.g., only right-directed), which easily overlooks the complete semantic chain of keywords due to directional limitations (e.g., the reverse logic of "symptom-cause" in a medical consultation). This invention combines forward syntax (emphasizing sequential development) and reverse syntax (emphasizing causal backtracking) through a fusion operation, so that the fused text contains both types of structural information simultaneously (e.g., forward text generates "headache -> fever", reverse text generates "fever <- headache", and the fusion strengthens the symptom association). This solves the problem of missing information in unidirectional generation, making the template content richer and improving the comprehensiveness and information density of personalized responses.

[0076] 3. The present invention solves the problem that the generated text syntax does not conform to natural language habits through the technical means of step 4 (dynamic template generation), and achieves the effect of optimizing the fluency and accuracy of the template through fine-tuning.

[0077] In step 4, after word embedding and positional encoding of the fused text from step 3, it is input into the template generator for fine-tuning (based on transformer-based SelfAttention and MaskedAttention mechanisms). Traditional template matching or simple generation models lack grammatical correction mechanisms, easily producing stiff or incoherent responses. This invention, through a fine-tuning process (such as residual processing and feedforward neural networks), recalculates the structural information of the fused text (e.g., adjusting word order to conform to subject-verb-object grammar) and applies softmax output to optimize grammar. This resolves potential grammatical conflicts in bidirectional fused text (such as word order disorder in reversed text), making the final template conform to natural language habits (such as fluent questioning sentences in a medical consultation response), thereby improving template accuracy and user comprehensibility.

[0078] 4. By combining the technical means of steps 1-4, this invention solves the problem that static templates cannot dynamically adapt to user input, and achieves the effect of generating flexible and personalized consultation templates.

[0079] Steps 1-2 involve bidirectional generation to extract multidimensional structural information, step 3 involves information fusion and integration, and step 4 involves fine-tuning and optimizing the output. Traditional template matching relies on predefined rules and cannot dynamically create templates based on user keywords (e.g., new symptoms cannot match existing templates). This invention utilizes dynamic processing of a transformer model (e.g., an encoder-decoder architecture), where the model learns the features of input keywords in real time (e.g., preserving semantic position through positional encoding) and generates customized templates (rather than mechanical matching). This solves the rigidity problem of static methods, enabling templates to adapt to diverse consultation scenarios (e.g., different combinations of medical keywords), improving the personalization and relevance of responses. Attached Figure Description

[0080] Figure 1 This is a simplified flowchart of the present invention. Detailed Implementation

[0081] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0082] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0083] A technology for generating answer templates based on user keywords includes the following steps:

[0084] Step 1: Generating keyword-ordered text

[0085] The keywords in the user's consultation statement are encoded and input into the right-direction text generator to obtain the right-direction text.

[0086] Step 2: Keyword Reverse Text Generation

[0087] The keywords of the user's consultation statement are sorted in reverse order, encoded, and then input into the left-direction text generator to obtain the left-direction text.

[0088] Step 3: Text Fusion

[0089] By concatenating and merging the right-hand text and the left-hand text, the merged text is obtained.

[0090] Step 4: Dynamic Template Generation

[0091] The merged text is encoded and input into a template generator for fine-tuning to obtain a generated dynamic consultation template.

[0092] Step 1 above specifically includes the following steps:

[0093] Step 1.1: Perform word embedding encoding on keywords in the consultation statements.

[0094] For the characteristic keywords of the consultation statement entered by the user, first perform an embedding encoding operation to obtain the embedding encoding X. r The encoding process extracts user-defined keywords from the user's input and feeds them into the model. First, the keywords are segmented into individual characters to obtain segmented keywords, allowing for the extraction of relevant features between keywords after input to the encoder. Then, the segmented keywords are bidirectionally mapped to unique numeric IDs using an index dictionary. These numeric IDs are then mapped to 512-dimensional high-dimensional vectors for subsequent input into the model. In other words, the segmented n keywords become:

[0095] [x1, x2, x3, ..., x i ], where x i Let i be the 512-dimensional vector mapped to the i-th keyword;

[0096] Since the transformer performs parallel operations on keywords in a sentence, in order to preserve the sequential structure of words in the sentence, it is necessary to calculate the positional encoding corresponding to each segmented keyword. The positional encoding can extract positional information in the sentence structure. Then, the positional encoding is fused with the 512-dimensional vector corresponding to the keyword. While preserving semantics, sentence structure information is added to the encoding. For the pos-th character, the positional encoding is performed for even-numbered and odd-numbered positions of the pos-th character as follows:

[0097]

[0098] In the above formula, i represents the position code of the i-th element corresponding to the pos-th keyword, and dmodel represents the total number of dimensions;

[0099] Location encoding PE r Compared with the previously constructed 512-dimensional Embedding encoding X r The word embedding vector V is obtained by performing addition and fusion. r This refers to the vector encoding input into the model:

[0100] V r =X r +PE r

[0101] Step 1.2: Input the obtained features into the right encoder to extract features;

[0102] After word embedding is completed, the word embedding vector V is... r The input is fed into the right-hand template encoding generator, which generates encoding from the keyword to the right, embedding the word into the vector V. r The input is fed into the encoder of the right-hand template generation model, which uses a transformer model.

[0103] The model's encoder performs matrix transformations through a neural network, converting the input word embedding vector V into a matrix. r These are mapped to three matrices, Q, K, and V, respectively. Since matrices Q, K, and V are derived from V... r Since they are mapped, all three matrices contain the semantic and structural information of the original sentence. This process can be viewed as mapping the original vector V. r Perform feature extraction;

[0104] By multiplying matrices Q and K, words at corresponding positions can be linked to words at other positions. Specifically, when matrices Q and K are multiplied, each element in matrix Q is multiplied with an element in matrix K, and the result is the similarity between the two multiplied elements. Therefore, multiplying matrices Q and K is actually calculating the similarity between all words in the sentence and other words in parallel, resulting in a structural information matrix that contains the structural information of the sentence itself.

[0105] The structural information matrix is ​​scaled and a softmax operation is performed to map each element to the interval [0, 1]. Then, the V matrix is ​​multiplied, which means that the structural information between sentences is appended to the original word positions. The result is subjected to residual operation and passed to the feedforward neural network to obtain the feature vector, which is sent to the right decoder for text generation in step 1.3.

[0106]

[0107] Step 1.3: Input the feature vector obtained in Step 1.2 into the right-hand decoder for template generation;

[0108] The feature vectors after SelfAttention are sent to the right-hand decoder for template generation. In the decoder, the data is first generated by MaskedAttention to obtain the generated data information. Similar to SelfAttention, this allows the sentence to be generated from front to back. The difference is that in order to avoid reading the information after the generation, an upper triangular matrix is ​​used to mask the data after the generated characters.

[0109] The structural information obtained in step 1.2 is then input together with the generated data information after MaskedAttention into a new SelfAttention to decode the data. The generated vector is then processed as a residual, input into a feedforward neural network, and then into softmax for output, resulting in right-hand text.

[0110] S r = softmax(W(SelfAttention(Q)) D K E V E )+X)+b)

[0111] Step 2 above specifically includes the following steps:

[0112] Step 2.1: Sort the characteristic keywords of the user-input statement in reverse order.

[0113] Reversing the order of user-input keywords preserves the original structure and relative positions while facilitating subsequent reverse text generation.

[0114] Step 2.2: Perform word embedding encoding on the reverse-order keywords;

[0115] Similar to step 1.1, this process performs embedding encoding and positional encoding on the reversed keywords, and finally adds the two to complete the word embedding operation, obtaining the word embedding vector V. l .

[0116] V l =X l +PE l

[0117] Step 2.3: Input the obtained features into the left encoder to extract features;

[0118] The information is input into the left-hand template generation encoding, which maps the encoding vector V into three matrices Q, K, and V. By performing operations on the Q and K matrices, the words at the corresponding positions can be connected with other positions. Then, the structure is scaled and a softmax operation is performed, and then multiplied with the V matrix to append the structural information between sentences to the original word positions. The residual operation is then performed and passed to the feedforward neural network, which is then sent to the decoder for subsequent text generation.

[0119]

[0120] Step 2.4: Input the obtained features into the left-hand decoder for template generation.

[0121] The feature vector is fed into the right-hand decoder for template generation. In the decoder, the data is first generated through MaskedAttention, similar to SelfAttention, except that the process is generated from front to back. To avoid reading subsequent information, an upper triangular matrix is ​​used to mask the data after the generated characters.

[0122] The encoded structural information and the generated data information after MaskedAttention are then input into a new SelfAttention to decode the data. The generated vector is then processed as a residual, input into a feedforward neural network, and then into a softmax for output, resulting in the left-directed text.

[0123] S l = softmax(W(SelfAttention(Q)) D K E V E )+X)+b)

[0124] Step 3 above specifically includes the following steps:

[0125] Step 3.1: In order for the model to have the ability to generate in reverse, the order of the sentences generated from the left needs to be reversed, which is the result of the model's left generation.

[0126] S l =[S n S n-1 S n-2 S1]

[0127] Step 3.2: Concatenate the sentences generated from the left and the sentences generated from the right.

[0128] S = S l +S r

[0129] Step 4 above specifically includes the following steps:

[0130] Step 4.1: Perform word embedding processing on the fused text S

[0131] This process is consistent with step 1.1, and requires processing the text S = [S1, S2, S3, ..., S...]. n Perform embedding encoding and location encoding (PE) corresponding to the new location information. new (pos), and finally add the two together to complete the word embedding operation, and obtain the word embedding vector V.

[0132] V = S + PE new (pos)

[0133] Step 4.2: Input the obtained word embedding vector V into the fine-tuning encoder to extract features;

[0134] Similar to step 1.2, the information is input into the left-directed template generation encoding, and the encoding vector V is mapped into three matrices Q, K, and V. By performing operations on matrices Q and K, the words at the corresponding positions can be connected with other positions. Then, the structure is scaled and a softmax operation is performed, and then multiplied with matrix V to append the structural information between sentences to the original word positions. The residual operation is then performed and passed to the feedforward neural network, and finally sent to the decoder for subsequent text generation.

[0135]

[0136] Step 4.3: Input the obtained features into the left-hand decoder for template generation.

[0137] The feature vector is fed into the right-hand decoder for template generation. In the decoder, the data is first generated through MaskedAttention, similar to SelfAttention, except that the process is generated from front to back. To avoid reading subsequent information, an upper triangular matrix is ​​used to mask the data after the generated characters.

[0138] The encoded structural information and the generated data information after MaskedAttention are then input into a new SelfAttention to decode the data. The generated vector is then processed as a residual and input into a feedforward neural network before being fed into softmax for output.

[0139] S = softmax(W(SelfAttention(Q)) D K E V E )+X)+b)

[0140] At this point, the model outputs the final, finely tuned dynamic template.

[0141] Current technology integrates different data features, minimizing randomness and effectively improving the fluency and accuracy of responses through multiple decoding. The categories after data featureization are more easily recognized by natural language processing encoders. It achieves multi-encoded information fusion while preventing the influence of more cluttered raw data on the results, something previous methods overlooked. Furthermore, the multi-decoder approach is a solution that simultaneously improves accuracy and fluency, something previous methods lacked.

[0142] In summary, the present invention has the following characteristics:

[0143] 1. This proposal adopts a dynamic template generation method, using the transformer algorithm to generate dynamic templates based on user-input keywords, providing users with more flexible template generation.

[0144] 2. The present invention uses steps 1 and 2 to achieve bidirectional keyword generation, so that the generated sentences simultaneously use the forward and reverse structural information of the keywords. Under the premise of keyword length, more structural information of the words themselves is obtained and the generated results make the maximum use of forward and reverse syntactic structures.

[0145] 3. In step 3 of this invention, the bidirectional generated text is fused to make the positive and negative structures of keywords connected in an orderly manner, and the positive and negative generation results of sentences are fused together. This process retains the bidirectional generation information of words, thereby obtaining a more accurate generation template containing more response information than traditional methods.

[0146] 4. In step 4 of this invention, the template of the fused text is fine-tuned to make the text syntax containing bidirectional generation information more in line with natural language habits, and the generated template is more accurate through syntax fine-tuning.

[0147] 5. Bidirectional Generation and Syntax Fine-Tuning in Dynamic Template Generation. Bidirectional generation enables the model to extract more keyword information, helping to generate better sentences from the text. Syntax fine-tuning adjusts the generated text containing bidirectional information into a generated template that conforms to natural language grammar, making the output text information more accurate.

Claims

1. A method for multimodal alignment processing of large medical models based on image channel grouping, characterized in that, Includes the following steps: Step S1: Input the full-scale medical image into the ResNet backbone network, extract basic features through layers conv1 to conv4, and insert parallel multi-scale convolutional modules after the basic features to generate multi-scale features; Step S2: Perform global pooling and recalibration on the multi-scale features to generate a channel attention heatmap, and cluster the information channels into multiple channel groups based on the peak response region to obtain a fine-grained image feature vector; Step S3: The input text is segmented, embedded, and positionally encoded using a Transformer encoder to generate a text feature matrix; Step S4: Map the text feature matrix and the fine-grained image feature vector to a unified dimensional space, and then concatenate and fuse them into a multimodal feature vector.

2. The method according to claim 1, characterized in that, Step 1 includes the following steps: Step S1.1: Input the full-scale image into the conv1 to conv4 layers of ResNet50 to extract the 1024-channel basic features F. base : F base =ResNet50 conv1→conv4 (X) X represents the input full-scale image; Step S1.2: Insert a multi-scale module into the conv4 layer, the multi-scale module containing three parallel branches: First branch: Processing F using a standard convolutional layer with a 1×1 kernel. base The feature F1 = Conv is obtained. 1*1 (F base ); The second branch: uses a convolutional layer with a 3×3 kernel and a dilation of 2 to process F. base The feature F2 = Conv is obtained. 3*3，dilation＝2 (F base ); The third branch: uses a convolutional layer with a 5×5 kernel and a dilation of 4 to obtain feature F3 = Conv. 5*5，dilation＝4 (F base ); Step S1.3: Concatenate the output features F1, F2, and F3 of the three branches to obtain the concatenated feature F. concat =Concat(F1, F2, F3); Step S1.4: For the splicing feature F concat Convolution operations are performed using 1×1 kernels to achieve dimensionality reduction and obtain multi-scale features F. multi-scale =Conv 1*1 (F concat ); Step S1.5: Input features F of the multi-scale module base With the output feature F of the multi-scale module multi-scale The addition operation is performed to form the residual structure, and the final multi-scale feature output is obtained.

3. The method according to claim 1, characterized in that, Step 2 includes the following steps: Step S2.1: Perform global pooling on the multi-scale features, and recalibrate the features through a fully connected layer, a ReLU layer, and a Sigmoid layer to obtain the recalibrated features F. recalibrated ; Step S2.2: F recalibrated Input a 1×1 convolutional layer to generate a feature heatmap H: H=Conv 1*1 (F recalibrated ) Step S2.3: Set the hyperparameter k, extract the k regions with the strongest thermodynamic response from H, and obtain H. top-k : H top-k =TopK(H,k) Where TopK(·,·) represents the operation of selecting the peak response region; Step S2.4: Use the k-means clustering algorithm to cluster H top-k Information channels with similar response regions are clustered into n groups {G1, G2, ..., G...} n }:{G1,G2,...,G n }=Kmeans(H top-k (n) Kmeans represents the k-means clustering algorithm, where n represents the preset number of channel groups. Step S2.5: Merge the clustered channel groups to generate a fine-grained image feature vector F grouped :F grouped =Concat(G1, G2, ..., G n ) Concat(·) represents the tensor concatenation operation.

4. The method according to claim 1, characterized in that, Step 3 includes the following steps: Step S3.1: The input text T is processed by word segmentation and vectorization using a 12-layer Transformer encoder to generate text embedding features T. embed : T embed =Transformer(T) Step S3.2: For T embed Add positional encoding to preserve the contextual information of the text sequence and generate a text feature matrix [T1, T2…T]. n ].

5. The method according to claim 1, characterized in that, Step S4 includes: Step S4.1: Convert the text feature matrix [T1, T2…T…] n The text vector T is obtained by mapping the vector to a vector space of dimension d = 512 through a linear transformation layer. mapped : T mapped =W T *T embed W T Represents the text linear transformation weight matrix. Step S4.2: Convert the fine-grained image feature vector F grouped The image vector F is obtained by mapping to a vector space of dimension d = 512 through a linear transformation layer. mapped =W F *T grouped W F Represents the image linear transformation weight matrix. Step S4.3: Place T mapped and F mapped Perform a concatenation operation to generate a multimodal feature vector F. fused : F fused =Concat(T mapped ,F mapped ) Concat(·,·) represents the concatenation operation.

Images