Chinese neck vessel ultrasound prompt generation method and system
By introducing multiple attention mechanisms into the LSTM network, the Chinese cervical vascular ultrasound cue generation method is improved, and the existing methods are solved, and more efficient and accurate cue generation is achieved to meet the needs of clinical auxiliary diagnosis.
Patent Information
- Application Number
- CN202510649621.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-20
- Publication Date
- 2025-09-02
AI Technical Summary
The existing Chinese cervical vascular ultrasound tip generation methods are inefficient and the accuracy is limited by the doctor's professional knowledge. Traditional methods require a lot of manual annotation. Deep learning methods are not effective in Chinese cervical vascular ultrasound and cannot effectively capture the association between long-term dependencies and prompts.
Multiple attention mechanisms are introduced to improve the LSTM network, and by examining the observed attention mechanism, attention mechanism between prompts and attention mechanism within prompts, the accuracy of the generation of Chinese cervical vascular ultrasound cues is improved. The Bi-LSTM encoder is used to extract long-term dependencies, combine the attention mechanism to capture semantic relationships, and generate high-quality cues through the splicing output module.
It improves the accuracy and efficiency of Chinese cervical vascular ultrasound prompt generation, can better capture long-term dependence and associations between prompts, generate higher-quality results, and meet the needs of clinical auxiliary diagnosis.
Smart Images
Figure CN120579516A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of text data processing, and in particular to a method and system for generating Chinese-language neck vascular ultrasound prompts. Background Art
[0002] Neck ultrasound is an important tool for the prevention and early screening of stroke. A typical Chinese neck ultrasound report typically includes five sections: examination type, clinical diagnosis, examination items, examination findings, and ultrasound indications. The examination findings section provides a detailed description of the ultrasound examination, while the ultrasound indication section contains at least one sub-item summarizing the key findings. Promoting the widespread use of neck ultrasound for stroke screening would be highly beneficial in reducing morbidity and mortality. However, current clinical practice requires recording numerous and complex objective measurements and subjective descriptions of findings during neck ultrasound examinations. Some of these findings are mutually inferential and corroborative, while others are irrelevant and serve only as supplementary information. These complex diagnostic criteria place high demands on physicians, hindering the widespread adoption of neck ultrasound for stroke screening.
[0003] With the development of artificial intelligence technology, data from Chinese neck vascular ultrasound are expected to be used to develop an intelligent model that uses text generation technology to automatically generate Chinese neck vascular ultrasound prompts, thereby providing doctors with real-time prompt references, saving clinical reporting time, reducing low-level errors such as content contradictions, and standardizing report content. This has very important practical significance for promoting the use of neck vascular ultrasound and thus improving the prevention effect and medical quality of heart and nervous system diseases such as stroke.
[0004] The automatic generation of Chinese-language neck vascular ultrasound prompts is a newly proposed task. Currently, the exploration of end-to-end methods based on deep learning is still relatively blank. Existing methods are mainly based on traditional entity recognition, entity extraction, and rule-based template text generation. Not only does this require a large amount of manual data annotation, but its efficiency and accuracy are also limited by the professional knowledge and energy input of medical personnel, which is time-consuming and labor-intensive.
[0005] Since the rise of deep learning, natural language processing technology has developed rapidly. Deep networks, represented by recurrent neural networks, have achieved outstanding performance in text processing. LSTM, a commonly used text generation network, introduces a gating mechanism to address the vanishing gradient problem of traditional recurrent neural networks when processing long sequences of data, enabling it to effectively capture long-term dependencies in the sequence.
[0006] Ultrasound cervical vascular prompts are a type of medical imaging prompt. Current research on deep learning-based prompt generation for medical imaging mainly focuses on generating English radiology prompts. However, due to differences in word segmentation standards, report content composition, and prompt generation methods between English and Chinese radiology reports, LSTM-based methods for generating English radiology prompts cannot be directly applied to Chinese cervical vascular ultrasound. Traditional rule-based machine learning methods have multiple processing steps and poor generalization capabilities. Furthermore, ultrasound examination findings often contain objective numerical text and subjective descriptions, resulting in prompt generation that does not meet clinical diagnostic needs. LSTM ultrasound prompt generation networks, however, are ineffective for generating prompts for Chinese cervical vascular ultrasound due to the fact that each sub-prompt within a Chinese cervical vascular ultrasound prompt is associated with different diseases, with varying degrees of association. Summary of the Invention
[0007] The purpose of the present invention is to provide a method and system for generating Chinese-language neck vascular ultrasound prompts, introduce a multi-attention mechanism to improve the LSTM network, and allow the model to pay more attention to the potential associations between prompts and the feature representations within the prompts during the prompt generation process, so as to improve the accuracy of Chinese-language neck vascular ultrasound prompt generation and solve at least one technical problem existing in the above-mentioned background technology.
[0008] In order to achieve the above object, the present invention adopts the following technical solutions:
[0009] In a first aspect, the present invention provides a method for generating Chinese-language neck vascular ultrasound prompts, comprising:
[0010] Obtain data on neck vascular examination findings in Chinese;
[0011] The acquired examination findings data are processed using a pre-trained ultrasound prompt generation model to generate corresponding Chinese neck vascular ultrasound prompts; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination findings attention mechanism module, an inter-prompt attention mechanism module, an intra-prompt attention mechanism module, and a splicing output module;
[0012] The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence;
[0013] The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector;
[0014] The observed attention mechanism module is used to calculate the observed context vector of the decoder hidden state at the current time step and the encoder hidden state at each previous time step;
[0015] The inter-cue attention mechanism module is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the current generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the current generated prompt;
[0016] The intra-cue attention mechanism module is used to calculate the intra-cue context vector, which contains the most relevant information of the generated words in the generated prompt to the current generated word;
[0017] The splicing output module is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
[0018] As a further limitation of the first aspect of the present invention, in the encoder module, for the input inspection word vector sequence, the input inspection word vector sequence is processed in sequence according to the time step order to calculate the forward hidden state, and the input inspection word vector sequence is processed in reverse order starting from the last time step to calculate the reverse hidden state. The forward and reverse hidden states are spliced to obtain the encoder hidden state, and finally the encoder hidden state sequence is obtained.
[0019] As a further limitation of the first aspect of the present invention, in the decoder module, the hidden state of the encoder in the last time step contains the forward and reverse information of the entire inspection input. The hidden state of the encoder in the last time step is used as the initial hidden state of the decoder, and the input ultrasonic prompt word vector is processed in sequence according to the time step order. The hidden state of the decoder in the current time step is calculated based on the decoder hidden state of the previous time step and the input ultrasonic prompt word vector.
[0020] As a further limitation of the first aspect of the present invention, it is checked that in the attention mechanism module, the hidden state s of the LSTM decoder at the current time step t t and the hidden state h of the encoder at each time step before i , calculate s t For h i Attention score Get the attention score of all H, and then get the attention weight a after normalization by softmax t , reflects the importance of each word seen in the inspection to the word currently generating the prompt; t The context vector seen by the checker is calculated by weighted summation of H, which contains the most relevant information about each word seen by the checker to the word currently generating the prompt.
[0021] As a further limitation of the first aspect of the present invention, the inter-cue attention mechanism module includes a first-level inter-cue attention mechanism unit and a second-level inter-cue attention mechanism unit; in the first-level inter-cue attention mechanism unit, for the hidden state s of the decoder at the current time step t and the hidden state of the decoder at each time step of the sub-prompt sequence Calculate s t right Attention score Get all Attention score Attention weights are obtained after softmax normalization right and The weighted sum is used to calculate the context vector between the first-level prompts; in the attention mechanism unit between the second-level prompts, the hidden state s of the decoder at the current time step t is t and the sub-cue context vector sc at each time step of the context vector sequence between the first-level cues i , calculate s t sc i Attention score Attention weights are obtained after softmax normalization right The second-level inter-cue context vector is calculated by weighted summation of SC.
[0022] As a further limitation of the first aspect of the present invention, in the attention mechanism module, for the hidden state s of the decoder at the current time step, t And the decoder hidden state of each time step of the sub-prompt sequence Calculate s t right Attention score Attention weights are obtained after softmax normalization right and S intra The weighted sum is used to calculate the intra-cue context vector.
[0023] In a second aspect, the present invention provides a Chinese neck vascular ultrasound prompt generation system, comprising:
[0024] Acquisition module, used to obtain the data of Chinese neck vascular examination;
[0025] a processing module, configured to process the acquired examination finding data using a pre-trained ultrasound prompt generation model to obtain corresponding neck vascular ultrasound prompts in Chinese; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination finding attention mechanism unit, an inter-prompt attention mechanism unit, an intra-prompt attention mechanism unit, and a splicing output unit;
[0026] The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence;
[0027] The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector;
[0028] The observed attention mechanism unit is used to calculate the observed context vector of the dimension based on the decoder hidden state of the current time step and the encoder hidden state of each previous time step;
[0029] The inter-cue attention mechanism unit is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the currently generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the currently generated prompt;
[0030] The intra-cue attention mechanism unit is used to calculate the intra-cue context vector, which contains the most relevant information about the generated words in the generated prompt to the current generated word;
[0031] The splicing output unit is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
[0032] In a third aspect, the present invention provides a non-transitory computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed by a processor, the method for generating Chinese neck vascular ultrasound prompts as described in the first aspect is implemented.
[0033] In a fourth aspect, the present invention provides a computer device comprising a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the Chinese neck vascular ultrasound prompt generation method as described in the first aspect.
[0034] In a fifth aspect, the present invention provides an electronic device comprising: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the Chinese neck vascular ultrasound prompt generation method as described in the first aspect.
[0035] The beneficial effects of the present invention are as follows: the encoder adopts Bi-LSTM to effectively extract long-term dependencies in the text, and combines the inspection observation attention mechanism to extract the deep association between the inspection observation and the ultrasound prompt; the LSTM decoder combines the inter-prompt attention mechanism and the intra-prompt attention mechanism to capture the potential deep semantic relationship and feature representation between each sub-prompt and within a single sub-prompt, and finally calculates the cross-entropy loss based on the gold standard ultrasound prompt; the beam search technology is used to expand the candidate range of the prompt generation result, so that the model has a better chance of finding the optimal prompt and obtaining higher quality results.
[0036] Additional advantages of the present invention will be more clearly given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0038] Figure 1 This is a schematic diagram of the structure of the multi-attention mechanism LSTM network model described in an embodiment of the present invention. DETAILED DESCRIPTION
[0039] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.
[0040] Those skilled in the art will understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.
[0041] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.
[0042] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.
[0043] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise contradictory.
[0044] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings. However, the specific embodiments do not constitute a limitation on the embodiments of the present invention.
[0045] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.
[0046] Example 1
[0047] In this first embodiment, a system for generating Chinese-language neck vascular ultrasound prompts is provided, comprising: an acquisition module for acquiring Chinese-language neck vascular examination finding data; and a processing module for processing the acquired examination finding data using a pre-trained ultrasound prompt generation model to obtain corresponding Chinese-language neck vascular ultrasound prompts, i.e., word distributions.
[0048] In this embodiment, the aforementioned system is used to implement a method for generating Chinese-language neck vascular ultrasound prompts, including: using an acquisition module to acquire Chinese-language neck vascular examination data; then using a processing module to process the acquired examination data using a pre-trained ultrasound prompt generation model to generate corresponding Chinese-language neck vascular ultrasound prompts.
[0049] In this embodiment, the ultrasound prompt generation model includes an encoder, a decoder, an inspection finding attention mechanism module, an inter-prompt attention mechanism module, an intra-prompt attention mechanism module, and a splicing output module. Among them, the encoder is used to encode the inspection word vector sequence into a hidden state sequence; the decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step based on the decoder hidden state of the previous time step and the input ultrasonic prompt word vector; the inspection attention mechanism module is used to calculate the dimension inspection context vector based on the decoder hidden state of the current time step and the hidden state of the encoder of each previous time step; the inter-prompt attention mechanism module is used to calculate the first-level inter-prompt context vector, which contains the most relevant information of each word of each generated sub-prompt to the word of the currently generated prompt; the second-level inter-prompt context vector is calculated based on the first-level inter-prompt context vector, which contains the most relevant information of each generated sub-prompt context to the word of the currently generated prompt; the intra-prompt attention mechanism module is used to calculate the intra-prompt context vector, which contains the most relevant information of the words generated in the prompt being generated to the currently generated word; the splicing output module is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-prompt attention mechanism context vector and the intra-prompt attention mechanism context vector to calculate the predicted word distribution.
[0050] Among them, in the encoder module, for the input inspection word vector sequence, the input inspection word vector sequence is processed in sequence according to the time step order to calculate the forward hidden state, and the input inspection word vector sequence is processed in reverse order starting from the last time step to calculate the reverse hidden state. The forward and reverse hidden states are concatenated to obtain the encoder hidden state, and finally the encoder hidden state sequence is obtained.
[0051] Specifically, for the Bi-LSTM encoder part, the input inspection word vector sequence X={x1,x2,…,x N}(where x i To check the word vector of the i-th word in the word sequence, the dimension is 300, the value range of i is [1, N], N is the number of words in the word sequence, and the final encoding result is obtained by Bi-LSTM at each time step. The forward LSTM processes the input word vector sequence of the check word in the order of the time step to calculate the forward hidden state with a dimension of 100. The reverse LSTM processes the input word vector sequence of the check word in reverse order starting from the last time step and calculates the reverse hidden state with a dimension of 100. The forward and reverse hidden states are concatenated to obtain the encoder hidden state with a dimension of 200, and finally the encoder hidden state sequence H = {h1,h2,…,h N}(where h iTo check the encoder hidden state vector of the i-th word in the word sequence, the dimension is 200, the value range of i is [1, N], N is the number of words in the word sequence), as shown in formula (1). The above is the Bi-LSTM encoder part.
[0052] H=Bi-LSTM(X) (1)
[0053] In the decoder module, the hidden state of the encoder at the last time step contains the forward and reverse information of the entire inspection input. The hidden state of the encoder at the last time step is used as the initial hidden state of the decoder. The input ultrasonic prompt word vector is processed in sequence according to the time step order. The decoder hidden state of the current time step is calculated based on the decoder hidden state of the previous time step and the input ultrasonic prompt word vector.
[0054] Specifically, the LSTM decoder part. The hidden state h of the last time step of the Bi-LSTM encoder is N The forward and reverse information of the entire inspection input is included and used as the initial hidden state s0 of the LSTM decoder. The encoding results of each time step are obtained through LSTM. LSTM processes the input ultrasound prompt word vector in sequence according to the time step order. The decoder hidden state s with a dimension of 200 at the previous time step t-1 before the current time step t is used. t-1 And the input ultrasound prompt word vector y with a dimension of 300 t-1 Calculate the decoder hidden state s of the current time step with a dimension of 200 t , as shown in formula (2).
[0055] s t =LSTM(s t-1 ,y t-1 ) (2)
[0056] Check the hidden state s of the LSTM decoder at the current time step t in the attention mechanism module. t and the hidden state h of the encoder at each time step before i , calculate s t For h i Attention score Get the attention score of all H, and then get the attention weight a after normalization by softmax t , reflects the importance of each word seen in the inspection to the word currently generating the prompt; t The context vector seen by the checker is calculated by weighted summation of H, which contains the most relevant information about each word seen by the checker to the word currently generating the prompt.
[0057] Specifically, check the attention mechanism section. For the current time step t, the LSTM decoder has a hidden state s of dimension 200. t and the hidden state h of the Bi-LSTM encoder dimension of 200 at each time step before i , compute s by inspecting the attention mechanism seen t For h i Attention score Get the attention score for all H (See formula (3)), and then the attention weight is obtained by softmax normalization It reflects the importance of each word seen in the inspection to the word currently generating the prompt (see formula (4)). t The weighted sum of H and the inspection context vector with a dimension of 200 is calculated It contains the information about each word seen in the inspection that is most relevant to the word currently generating the prompt, as shown in formula (5).
[0058]
[0059] a t =softmax(e t ) (4)
[0060]
[0061] Among them, the value range of i is {1,2,…,N}, W h 、W s and v are learnable parameters of the observed attention mechanism, which are randomly initialized and optimized and updated through backpropagation and gradient descent algorithms during model training.
[0062] The inter-cue attention mechanism module includes a first-level inter-cue attention mechanism unit and a second-level inter-cue attention mechanism unit; in the first-level inter-cue attention mechanism unit, for the hidden state s of the current time step decoder t and the hidden state of the decoder at each time step of the sub-prompt sequence Calculate s t right Attention score Get all Attention score Attention weights are obtained after softmax normalization right and The weighted sum is used to calculate the context vector between the first-level prompts; in the attention mechanism unit between the second-level prompts, the hidden state s of the decoder at the current time step t is tand the sub-cue context vector sc at each time step of the context vector sequence between the first-level cues i , calculate s t sc i Attention score Attention weights are obtained after softmax normalization right The second-level inter-cue context vector is calculated by weighted summation of SC.
[0063] Specifically, for the inter-cue attention mechanism, the first is the first-level inter-cue attention mechanism, the decoder hidden state sequence of each sub-cue generated before the sub-cue at the current time step t is (in is the encoder hidden state vector of the jth word in the generated i-th sub-prompt word sequence, with a dimension of 200, i ranging from [1, n], where n is the number of sub-prompts in the prompt, and j ranging from [1, m], where m is the number of words in the sub-prompt). For the current time step, the LSTM decoder has a hidden state s of dimension 200. t And the hidden state of the LSTM decoder with dimension 200 at each time step of the sub-prompt sequence Calculate s through the first-level inter-cue attention mechanism t right Attention score Get all Attention score (See formula (8)), and then the attention weight is obtained after softmax normalization It reflects the importance of each word of each generated sub-prompt to the word of the current generated prompt (see formula (9)). and The weighted summation calculates the first-level inter-cue context vector sc with a dimension of 200 i , which contains the most relevant information of each word of each generated sub-prompt to the word of the current generated prompt, as shown in formula (10). Then, a two-level inter-prompt attention mechanism is constructed, and the first-level inter-prompt context vector sequence is SC = {sc1, sc2…, sc n}(where sc i is the first-level inter-prompt context vector of the generated i-th sub-prompt word sequence, with a dimension of 200, and the value range of i is [1,n], where n is the number of sub-prompts in the prompt). For the hidden state s of the LSTM decoder with a dimension of 200 at the current time step t t The sub-prompt context vector sc with a dimension of 200 at each time step of the context vector sequence between the first-level prompts i , calculate s through the secondary inter-cue attention mechanism t sci Attention score Get the attention score of all SCs (See formula (11)), and then the attention weight is obtained after softmax normalization It reflects the importance of each generated sub-prompt context to the word of the current generated prompt (see formula (12)). The weighted sum of SC and Calculate the secondary inter-cue context vector with a dimension of 200 It contains the most relevant information of each generated sub-prompt context to the word of the currently generated prompt, as shown in formula (13).
[0064]
[0065]
[0066] Among them, the value range of j is {1,2,…,m}, the value range of i is {1,2,…,n}, and W inter1 、W sinter1 、v inter1 、W inter2 、W sinter2 and v inter2 It is a learnable parameter of the inter-cue attention mechanism, which is randomly initialized and optimized and updated by backpropagation and gradient descent algorithms during model training.
[0067] In the hint attention mechanism module, the hidden state s of the decoder at the current time step is t And the decoder hidden state of each time step of the sub-prompt sequence Calculate s t right Attention score Attention weights are obtained after softmax normalization right and S intra The weighted sum is used to calculate the intra-cue context vector.
[0068] Specifically, for the part of the attention mechanism within the prompt, the decoder hidden state sequence before the current time step of the sub-prompt being generated at the current time step t is (in is the encoder hidden state vector of the i-th word in the sub-prompt word sequence being generated, with a dimension of 200, the value range of i is [1, tp-1], and p is the position of the first word in the sub-prompt currently being generated). For the current time step, the LSTM decoder has a hidden state s with a dimension of 200. t And the hidden state of the LSTM decoder with dimension 200 at each time step of the sub-prompt sequence Compute s via the intra-cue attention mechanism t right Attention score Get all S intra Attention score (See formula (14)), and then the attention weight is obtained after softmax normalization reflects the importance of the words generated in the prompt being generated to the current word (see formula (15)). As shown in formula (16), and S intra The weighted sum calculates the context vector of the prompt with a dimension of 200 It contains the most relevant information about the currently generated word among the previously generated words in the prompt being generated.
[0069]
[0070]
[0071] Among them, the value range of i is {1,2,…,tp-1}, W intra 、W sintra and v intra It is a learnable parameter of the intra-cue attention mechanism, which is randomly initialized and optimized and updated by backpropagation and gradient descent algorithms during model training.
[0072] Finally, in the concatenated output unit, the concatenated LSTM decoder hidden state s of the current time step is t , check the attention mechanism context vector seen Inter-cue attention mechanism context vector and the context vector of the intra-cue attention mechanism After concatenation, the predicted word distribution is calculated through the output layer, as shown in formula (6):
[0073]
[0074] Among them, X represents the word sequence seen in the inspection, Y <t represents the ultrasound prompt word sequence generated before the current time step t, V and V′ are learnable output layer parameters, which are randomly initialized and optimized and updated through back propagation and gradient descent algorithms during the model training process.
[0075] Example 2
[0076] In this embodiment 2, a method for generating Chinese-language neck vascular ultrasound prompts is provided. The method first obtains Chinese-language neck vascular examination data to be processed. Then, a pre-trained ultrasound prompt generation model is used to process the obtained examination data to obtain corresponding Chinese-language neck vascular ultrasound prompt results.
[0077] like Figure 1 As shown, in this embodiment, the construction and training of the above-mentioned pre-trained ultrasound prompt generation model includes the following specific steps:
[0078] Step 1: Preprocess the text;
[0079] Step 1.1: Perform Chinese word segmentation on the training text including examination findings and ultrasound prompts and convert them into word sequences, with words separated by ",".
[0080] Step 1.2: Add start and end markers to the ultrasound prompt word sequence, including adding a " <sos>" (start of sentence) marks the beginning of the sequence and adds a " <eos>"(end of sentence)" marks the end of the sequence to clearly mark the beginning and end of the generated prompt.
[0081] Step 1.3: Replace the original separators "|:" between each sub-prompt in the ultrasound prompt sequence with " <sep>” (separator) mark to determine the starting and ending positions of each sub-prompt in the overall prompt sequence.
[0082] Step 1.4: Create a guide sequence based on the ultrasound prompt word sequence. In the guide sequence, the value 0 or 1 represents whether the corresponding position of the ultrasound prompt word sequence is " <sep>" mark to distinguish the position ranges of different sub-cues in the complete cue sequence.
[0083] Step 2: Build a multi-attention mechanism LSTM network model;
[0084] Step 2.1: Initialize word embeddings. After removing duplicate words from the examination findings, we obtained a vocabulary containing 33,326 words without prior information, which completely covers all words found in ultrasound examination findings. Based on this vocabulary, we used randomly initialized 300-dimensional word vectors as the initial values for the model's word embeddings.
[0085] Step 2.2: Construct the Bi-LSTM encoder part. For the input inspection, the word vector sequence X={x1,x2,…,x N }(where x i To check the word vector of the i-th word in the word sequence, the dimension is 300, the value range of i is [1, N], N is the number of words in the word sequence, and the final encoding result is obtained by Bi-LSTM at each time step. The forward LSTM processes the input word vector sequence of the check word in the order of the time step to calculate the forward hidden state with a dimension of 100. The reverse LSTM processes the input word vector sequence of the check word in reverse order starting from the last time step and calculates the reverse hidden state with a dimension of 100. The forward and reverse hidden states are concatenated to obtain the encoder hidden state with a dimension of 200, and finally the encoder hidden state sequence H = {h1,h2,…,h N }(where h i To check the encoder hidden state vector of the i-th word in the word sequence, the dimension is 200, the value range of i is [1, N], N is the number of words in the word sequence), as shown in formula (1).
[0086] H=Bi-LSTM(X)#(1)
[0087] Step 2.3: Construct the LSTM decoder part. The hidden state h of the last time step of the Bi-LSTM encoder is N The forward and reverse information of the entire inspection input is included and used as the initial hidden state s0 of the LSTM decoder. The encoding results of each time step are obtained through LSTM. LSTM processes the input ultrasound prompt word vector in sequence according to the time step order. The decoder hidden state s with a dimension of 200 at the previous time step t-1 before the current time step t is used. t-1 And the input ultrasound prompt word vector y with a dimension of 300 t-1 Calculate the decoder hidden state s of the current time step with a dimension of 200 t , as shown in formula (2).
[0088] s t =LSTM(s t-1 ,y t-1 )#(2)
[0089] Step 2.4: Construct the attention mechanism part of the inspection. For the current time step t, the LSTM decoder has a hidden state s of dimension 200. t and the hidden state h of the Bi-LSTM encoder dimension of 200 at each time step before i , compute s by inspecting the attention mechanism seen t For h i Attention score Get the attention score for all H (See formula (3)), and then the attention weight is obtained after softmax normalization It reflects the importance of each word seen in the inspection to the word currently generating the prompt (see formula (4)). t The weighted sum of H and the inspection context vector with a dimension of 200 is calculated It contains the information about each word seen in the inspection that is most relevant to the word currently generating the prompt, as shown in formula (5).
[0090]
[0091] a t =softmax(e t )#(4)
[0092]
[0093] Among them, the value range of i is {1,2,…,N}, W h 、W s and v are learnable parameters of the observed attention mechanism, which are randomly initialized and optimized and updated through backpropagation and gradient descent algorithms during model training.
[0094] Step 2.5: Construct the inter-cue attention mechanism. First, construct the first-level inter-cue attention mechanism. The decoder hidden state sequence of each sub-cue generated before the sub-cue at the current time step t is (in is the encoder hidden state vector of the jth word in the generated i-th sub-prompt word sequence, with a dimension of 200, i ranging from [1, n], where n is the number of sub-prompts in the prompt, and j ranging from [1, m], where m is the number of words in the sub-prompt). For the current time step, the LSTM decoder has a hidden state s of dimension 200. t And the hidden state of the LSTM decoder with dimension 200 at each time step of the sub-prompt sequence Calculate s through the first-level inter-cue attention mechanism t right Attention score Get all Attention score (See formula (8)), and then the attention weight is obtained after softmax normalization It reflects the importance of each word of each generated sub-prompt to the word of the current generated prompt (see formula (9)). and The weighted summation calculates the first-level inter-cue context vector sc with a dimension of 200 i , which contains the most relevant information of each word of each generated sub-prompt to the word of the current generated prompt, as shown in formula (10). Then, a two-level inter-prompt attention mechanism is constructed, and the first-level inter-prompt context vector sequence is SC = {sc1, sc2…, sc n }(where sc i is the first-level inter-prompt context vector of the generated i-th sub-prompt word sequence, with a dimension of 200, and the value range of i is [1,n], where n is the number of sub-prompts in the prompt). For the hidden state s of the LSTM decoder with a dimension of 200 at the current time step t t The sub-prompt context vector sc with a dimension of 200 at each time step of the context vector sequence between the first-level prompts i , calculate s through the secondary inter-cue attention mechanism t sc i Attention score Get the attention score of all SCs (See formula (11)), and then the attention weight is obtained after softmax normalization It reflects the importance of each generated sub-prompt context to the word of the current generated prompt (see formula (12)). The weighted sum of SC and Calculate the secondary inter-cue context vector with a dimension of 200 It contains the most relevant information of each generated sub-prompt context to the word of the currently generated prompt, as shown in formula (13).
[0095]
[0096] Among them, the value range of j is {1,2,…,m}, the value range of i is {1,2,…,n}, and W inter1 、W sinter1 、v inter1 、W inter2 、W sinter2 and v inter2 It is a learnable parameter of the inter-cue attention mechanism, which is randomly initialized and optimized and updated by backpropagation and gradient descent algorithms during model training.
[0097] Step 2.6: Construct the intra-cue attention mechanism. The decoder hidden state sequence before the current time step of the sub-cue being generated at the current time step t is (in is the encoder hidden state vector of the i-th word in the sub-prompt word sequence being generated, with a dimension of 200, the value range of i is [1, tp-1], and p is the position of the first word in the sub-prompt currently being generated). For the current time step, the LSTM decoder has a hidden state s with a dimension of 200. t And the hidden state of the LSTM decoder with dimension 200 at each time step of the sub-prompt sequence Compute s via the intra-cue attention mechanism t right Attention score Get all S intra Attention score (See formula (14)), and then the attention weight is obtained after softmax normalization reflects the importance of the words generated in the prompt being generated to the current word (see formula (15)). As shown in formula (16), and S intra The weighted sum calculates the context vector of the prompt with a dimension of 200 It contains the most relevant information about the currently generated word among the previously generated words in the prompt being generated.
[0098]
[0099] Among them, the value range of i is {1,2,…,tp-1}, W intra 、W sintra and v intra It is a learnable parameter of the intra-cue attention mechanism, which is randomly initialized and optimized and updated by backpropagation and gradient descent algorithms during model training.
[0100] Step 2.7: The concatenated LSTM decoder hidden state s of the current time step t , check the attention mechanism context vector seen Inter-cue attention mechanism context vector and the context vector of the intra-cue attention mechanism After concatenation, the predicted word distribution is calculated through the output layer, as shown in formula (6):
[0101]
[0102] Among them, X represents the word sequence seen in the inspection, Y <t represents the ultrasound prompt word sequence generated before the current time step t, V and V′ are learnable output layer parameters, which are randomly initialized and optimized and updated through back propagation and gradient descent algorithms during the model training process.
[0103] During model training, the negative log-likelihood between the generated ultrasound prompts and the gold standard ultrasound prompts written by doctors is used as the loss function, as shown in formula (7):
[0104]
[0105] Among them, g t represents the word of the gold standard prompt at time step t.
[0106] The Adam optimizer was used for training, with an initial learning rate of 0.0001 and a dropout rate of 0.5, updated every epoch. A teacher-forcing model was used during training, using the gold-standard ultrasound cue sequence to guide the model output rather than the cue sequence generated by the model itself. Beam search was used for prediction, retaining a certain number of candidate cue sequences with the highest probability at each step of cue generation as the search space, with a beam width of 5.
[0107] Example 3
[0108] This embodiment 3 provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the method for generating a Chinese-language neck vascular ultrasound prompt is implemented. The method includes:
[0109] Obtain data on neck vascular examination findings in Chinese;
[0110] The acquired examination findings data are processed using a pre-trained ultrasound prompt generation model to generate corresponding Chinese neck vascular ultrasound prompts; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination findings attention mechanism module, an inter-prompt attention mechanism module, an intra-prompt attention mechanism module, and a splicing output module;
[0111] The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence;
[0112] The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector;
[0113] The observed attention mechanism module is used to calculate the observed context vector of the decoder hidden state at the current time step and the encoder hidden state at each previous time step;
[0114] The inter-cue attention mechanism module is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the current generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the current generated prompt;
[0115] The intra-cue attention mechanism module is used to calculate the intra-cue context vector, which contains the most relevant information of the generated words in the generated prompt to the current generated word;
[0116] The splicing output module is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
[0117] Example 4
[0118] This embodiment 4 provides a computer device, including a memory and a processor, wherein the processor and the memory communicate with each other, the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the above-mentioned method for generating a Chinese-language neck vascular ultrasound prompt, the method comprising:
[0119] Obtain data on neck vascular examination findings in Chinese;
[0120] The acquired examination findings data are processed using a pre-trained ultrasound prompt generation model to generate corresponding Chinese neck vascular ultrasound prompts; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination findings attention mechanism module, an inter-prompt attention mechanism module, an intra-prompt attention mechanism module, and a splicing output module;
[0121] The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence;
[0122] The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector;
[0123] The observed attention mechanism module is used to calculate the observed context vector of the decoder hidden state at the current time step and the encoder hidden state at each previous time step;
[0124] The inter-cue attention mechanism module is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the current generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the current generated prompt;
[0125] The intra-cue attention mechanism module is used to calculate the intra-cue context vector, which contains the most relevant information of the generated words in the generated prompt to the current generated word;
[0126] The splicing output module is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
[0127] Example 5
[0128] This embodiment 5 provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing the above-mentioned method for generating a Chinese-language neck vascular ultrasound prompt, the method comprising:
[0129] Obtain data on neck vascular examination findings in Chinese;
[0130] The acquired examination findings data are processed using a pre-trained ultrasound prompt generation model to generate corresponding Chinese neck vascular ultrasound prompts; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination findings attention mechanism module, an inter-prompt attention mechanism module, an intra-prompt attention mechanism module, and a splicing output module;
[0131] The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence;
[0132] The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector;
[0133] The observed attention mechanism module is used to calculate the observed context vector of the decoder hidden state at the current time step and the encoder hidden state at each previous time step;
[0134] The inter-cue attention mechanism module is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the current generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the current generated prompt;
[0135] The intra-cue attention mechanism module is used to calculate the intra-cue context vector, which contains the most relevant information of the generated words in the generated prompt to the current generated word;
[0136] The splicing output module is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
[0137] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0138] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0139] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0141] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.< / sep> < / sep> < / eos> < / sos>
Claims
1. A method for generating Chinese neck vascular ultrasound prompts, characterized in that: include: Obtain data on neck vascular examination findings in Chinese; The acquired examination findings data are processed using a pre-trained ultrasound prompt generation model to generate corresponding Chinese neck vascular ultrasound prompts; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination findings attention mechanism module, an inter-prompt attention mechanism module, an intra-prompt attention mechanism module, and a splicing output module; The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence; The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector; The observed attention mechanism module is used to calculate the observed context vector of the decoder hidden state at the current time step and the encoder hidden state at each previous time step; The inter-cue attention mechanism module is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the current generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the current generated prompt; The intra-cue attention mechanism module is used to calculate the intra-cue context vector, which contains the most relevant information of the generated words in the generated prompt to the current generated word; The splicing output module is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
2. The method for generating Chinese neck blood vessel ultrasound prompts according to claim 1, characterized in that: In the encoder module, for the input word vector sequence seen by inspection, the input word vector sequence seen by inspection is processed in sequence according to the time step order to calculate the forward hidden state, and the input word vector sequence seen by inspection is processed in reverse order starting from the last time step to calculate the reverse hidden state. The forward and reverse hidden states are concatenated to obtain the encoder hidden state, and finally the encoder hidden state sequence is obtained.
3. The method for generating Chinese neck blood vessel ultrasound prompts according to claim 1, characterized in that: In the decoder module, the hidden state of the encoder at the last time step contains the forward and reverse information of the entire inspection input. The hidden state of the encoder at the last time step is used as the initial hidden state of the decoder. The input ultrasonic prompt word vector is processed in sequence according to the time step order. The decoder hidden state of the current time step is calculated based on the decoder hidden state of the previous time step and the input ultrasonic prompt word vector.
4. The method for generating Chinese neck blood vessel ultrasound prompts according to claim 1, characterized in that: Check the hidden state s of the LSTM decoder at the current time step t in the attention mechanism module. t and the hidden state h of the encoder at each time step before i , calculate s t For h i Attention score Get the attention score of all H, and then get the attention weight a after softmax normalization t , reflects the importance of each word seen in the inspection to the word currently generating the prompt; t The context vector seen by the checker is calculated by weighted summation of ∑i=1,i=2,i,i,n,H, which contains the most relevant information about each word seen by the checker for the word currently generating the prompt.
5. The method for generating Chinese neck blood vessel ultrasound prompts according to claim 1, characterized in that: The inter-cue attention mechanism module includes a first-level inter-cue attention mechanism unit and a second-level inter-cue attention mechanism unit; in the first-level inter-cue attention mechanism unit, for the hidden state s of the current time step decoder t and the hidden state of the decoder at each time step of the sub-prompt sequence Calculate s t right Attention score Get all Attention score Attention weights are obtained after softmax normalization right and The weighted sum is used to calculate the context vector between the first-level prompts; in the attention mechanism unit between the second-level prompts, the hidden state s of the decoder at the current time step t is t and the sub-cue context vector sc at each time step of the context vector sequence between the first-level cues i , calculate s t sc i Attention score Attention weights are obtained after softmax normalization right The second-level inter-cue context vector is calculated by weighted summation of SC.
6. The method for generating Chinese neck blood vessel ultrasound prompts according to claim 1, characterized in that: In the hint attention mechanism module, the hidden state s of the decoder at the current time step is t And the decoder hidden state of each time step of the sub-prompt sequence Calculate s t right Attention score Attention weights are obtained after softmax normalization right and S intra The weighted sum is used to calculate the intra-cue context vector.
7. A Chinese neck vascular ultrasound prompt generation system, characterized in that: include: Acquisition module, used to obtain the data of Chinese neck vascular examination; a processing module, configured to process the acquired examination finding data using a pre-trained ultrasound prompt generation model to obtain corresponding neck vascular ultrasound prompts in Chinese; wherein the ultrasound prompt generation model includes an encoder, a decoder, an examination finding attention mechanism unit, an inter-prompt attention mechanism unit, an intra-prompt attention mechanism unit, and a splicing output unit; The encoder is used to encode the word vector sequence seen by the inspection into a hidden state sequence; The decoder is used to process the input ultrasonic prompt word vector in sequence according to the time step order, and calculate the decoder hidden state of the current time step according to the decoder hidden state of the previous time step and the input ultrasonic prompt word vector; The observed attention mechanism unit is used to calculate the observed context vector of the dimension based on the decoder hidden state of the current time step and the encoder hidden state of each previous time step; The inter-cue attention mechanism unit is used to calculate a first-level inter-cue context vector, which contains the most relevant information of each word of each generated sub-cue to the word of the currently generated prompt; and calculate a second-level inter-cue context vector based on the first-level inter-cue context vector, which contains the most relevant information of each generated sub-cue context to the word of the currently generated prompt; The intra-cue attention mechanism unit is used to calculate the intra-cue context vector, which contains the most relevant information about the generated words in the generated prompt to the current generated word; The splicing output unit is used to splice the decoder hidden state of the current time step, the inspection attention mechanism context vector, the inter-cue attention mechanism context vector and the intra-cue attention mechanism context vector to calculate the predicted word distribution.
8. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for generating Chinese neck blood vessel ultrasound prompts according to any one of claims 1 to 6 is implemented.
9. A computer device, characterized in that: It includes a memory and a processor, the processor and the memory communicate with each other, the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the Chinese neck blood vessel ultrasound prompt generation method according to any one of claims 1-6.
10. An electronic device, characterized in that: include: A processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the Chinese neck vascular ultrasound prompt generation method as described in any one of claims 1-6.