Renal artery stenosis ultrasonic diagnosis report automatic generation method

Abstract

The invention provides a renal artery stenosis ultrasonic diagnosis report automatic generation method, and relates to the technical field of image data processing, and the method comprises the steps: inputting a plurality of renal artery ultrasonic images, carrying out the redundancy processing, calculating the semantic correlation between each image and a diagnosis text, and screening out a key image; performing feature extraction and enhancement on the screened key images, constructing a static medical knowledge memory matrix, and aligning enhanced visual features with the static medical knowledge memory matrix to generate knowledge alignment vectors; maintaining a dynamic memory network, and injecting the knowledge alignment vector into the dynamic memory network; and fusing the external knowledge token into a code, and finally outputting a structured renal artery ultrasonic diagnosis report based on the enhanced visual features, the external knowledge token and the dynamic memory network state. According to the method, redundant information processing and fine-grained feature extraction of the renal artery ultrasonic image are realized, and external medical knowledge is guided to generate a long ultrasonic diagnosis report which is consistent in logic and has clinical interpretability.

Description

Technical Field

[0001] This invention relates to the field of image data processing technology, and in particular to a method for automatically generating ultrasound diagnostic reports of renal artery stenosis. Background Technology

[0002] Renal artery stenosis (RAS) is the most common cause of hypertension. The renal arteries play a central role in maintaining renal perfusion, regulating fluid balance, and maintaining blood pressure homeostasis. Stenosis and hemodynamic changes in the renal arteries are closely associated with renovascular hypertension, ischemic nephropathy, and various cardiorenal syndromes. As a treatable condition, the screening and diagnosis of renal artery stenosis are of great clinical concern.

[0003] Renal artery ultrasound is an important diagnostic tool for assessing renal hemodynamics and screening for renal artery stenosis, and its diagnostic reports play a crucial role in clinical decision-making. Renal artery ultrasound not only offers advantages such as being non-invasive, convenient, radiation-free, and real-time, but also provides hemodynamic information about the renal arteries. However, the clinical application of renal artery ultrasound still faces several limitations. On the one hand, due to the complex anatomical features and hemodynamics of the renal arteries, as well as the influence of the physical properties of ultrasound, ultrasound image acquisition and Doppler parameter measurement are somewhat operator-dependent. Differences in scanning angle, sampling location, and spectral quality can easily lead to unstable diagnostic conclusions. On the other hand, the deep location of the renal arteries, the limited acoustic window, and the non-uniform image quality make the diagnosis of renal artery stenosis inherently unique and complex.

[0004] The writing of renal artery ultrasound reports is highly dependent on the examiner's experience and expertise, and thus inherently subjective. Ultrasound reports have complex text structures and require precise descriptions of multiple anatomical regions, blood flow parameters, and lesion characteristics. Beginners and junior physicians often omit information, produce semantic ambiguities, or exhibit logical inconsistencies, impacting diagnostic quality and reproducibility. However, under the dual pressures of continuously increasing examination volumes and a chronic shortage of ultrasound physicians, traditional manual writing methods struggle to balance efficiency and quality, necessitating the use of intelligent technology to improve report consistency, accuracy, and interpretability. With the continued growth in healthcare services, these issues make the automated generation of high-quality, standardized renal artery ultrasound reports increasingly important.

[0005] In recent years, deep learning has been widely applied and has achieved significant breakthroughs in the automatic generation of medical reports. One paper proposed AMAnet, an automatic ultrasound image report generation method based on an adaptive multimodal attention mechanism. Its goal is to automatically generate Chinese ultrasound reports that closely resemble clinical writing conventions from a single breast ultrasound image, thereby alleviating the problems of reliance on manual report writing, low efficiency, and inconsistent quality. This method designs a visual encoding and multi-label attribute prediction module, a report generation module based on multimodal attention fusion, and an adaptive attention mechanism with sentinel gating. This improves the accuracy of descriptions of key local features such as "boundaries, morphology, and blood flow" and the consistency of templated expressions, effectively improving the quality of the generated report. However, this method only utilizes a single static image, making it difficult to reflect the complete diagnostic process in real-world scenarios where doctors select multiple ultrasound images to generate accurate diagnostic text. Another paper proposed an automatic ultrasound report generation framework, UG-CMFA, combining unsupervised and supervised learning. This framework mainly includes a knowledge distillation module, a knowledge matching visual feature extraction module, and a report generation module. This method first uses unsupervised knowledge distillation to automatically extract prior knowledge from ultrasound reports. Then, it designs a visual feature extraction and cross-modal alignment mechanism based on knowledge pseudo-labels. Finally, it integrates a Transformer report generation module with global semantic similarity constraints. This achieves the mitigation of feature inconsistencies between ultrasound images and text reports without the need for additional disease labels, and generates ultrasound examination reports that conform to clinical semantics. However, this method still has shortcomings. First, it does not fully consider scenarios with multiple ultrasound images. Second, it does not introduce external medical knowledge but instead uses unsupervised clustering to generate knowledge topics, resulting in noisy pseudo-labels. These noisy pseudo-labels are used as "real categories" during training, which can mislead the visual encoder and the cross-modal alignment process. The discrepancy between the knowledge topics and real clinical concepts affects the clinical interpretability of the model-generated text.

[0006] In general, the automated generation of ultrasound reports for renal artery stenosis lacks systematic research and faces challenges such as redundant / irrelevant images interfering with report accuracy, difficulty in effectively incorporating medical knowledge into the text generation process, and unstable logic in generating long texts. To address these issues, there is an urgent need for an automated method for generating ultrasound diagnostic reports for renal artery stenosis. This method should perform high-precision modeling of fine-grained features in renal artery ultrasound images, enabling the generation of logically consistent and clinically interpretable long ultrasound diagnostic reports guided by external medical knowledge. This would improve the efficiency of ultrasound report generation and alleviate the contradiction between the shortage of ultrasound physicians and the continuously increasing volume of examinations. Summary of the Invention

[0007] To address the problems in the background technology, this invention provides an automatic generation method for ultrasound diagnostic reports of renal artery stenosis. By introducing a redundancy adaptation mechanism, a dual memory network, and an external medical knowledge fusion mechanism, it achieves high-precision modeling of fine-grained features in renal artery ultrasound images and generates accurate, logically consistent, and clinically interpretable standardized ultrasound reports, thus assisting in the manual writing process of ultrasound reports and improving report generation efficiency.

[0008] To achieve the above objectives, the present invention provides a method for automatically generating ultrasound diagnostic reports for renal artery stenosis, comprising: Input multiple renal artery ultrasound images, which are derived from a single renal artery ultrasound examination; The image redundancy adaptation module processes the multiple images, calculates the semantic relevance of each image to the diagnostic text, and filters out key images based on the relevance score to suppress interference from redundant images. The key images after screening are processed by a visual feature enhancement module. This includes applying position attention and channel attention mechanisms to enhance the fine-grained feature expression of the renal artery stenosis region and obtain enhanced visual features. A static medical knowledge memory matrix is ​​constructed, which encodes medical knowledge vectors related to renal artery stenosis. The knowledge vectors include anatomical structure descriptions, hemodynamic parameters, and stenosis grading criteria. The enhanced visual features are aligned with the static medical knowledge memory matrix using a memory alignment module to generate a knowledge alignment vector. The dynamic memory network is maintained by a dual memory network module, and the knowledge alignment vector is injected into the dynamic memory network to perform cross-sentence context modeling, ensuring the logical consistency of the generated report. An external knowledge token is integrated into the encoding process through an external knowledge injection module. The external knowledge token includes a static knowledge token and a dynamic knowledge token. The static knowledge token originates from the textual semantics of the diagnostic text, and the dynamic knowledge token originates from the output of the dynamic memory network. The report generation module outputs a structured renal artery ultrasound diagnostic report based on the enhanced visual features, external knowledge tokens, and dynamic memory network state.

[0009] As a further improvement of the present invention, the step of processing the multiple images through the image redundancy adaptation module, calculating the semantic relevance of each image to the diagnostic text, and filtering out key images based on the relevance score to suppress redundant image interference includes: The global semantic vector for each image is extracted using a pre-trained CLIP visual encoder. The diagnostic text is encoded into a text semantic vector using the CLIP text encoder. Calculate the cosine similarity between the global semantic vector and the text semantic vector of each image to obtain a semantic relevance score; The first m key images are selected based on the semantic relevance scores. The weights of the first m key images are normalized by softmax and then weighted and fused.

[0010] As a further improvement of the present invention, the cosine similarity between the global semantic vector of each image and the text semantic vector is calculated using the following formula: in, Let represent the semantic relevance score of the i-th image, gi represent the global semantic vector of the i-th image, f represent the text semantic vector, and τ represent the learnable temperature coefficient, which is used to control the score range. The learnable temperature coefficient τ is a learnable parameter during model training, used to optimize the distribution of similarity scores and enhance the stability and adaptability of key image screening.

[0011] As a further improvement of the present invention, a visual feature enhancement module is used to extract and enhance features from the selected key images, including applying positional attention and channel attention mechanisms to strengthen the fine-grained feature expression of the renal artery stenosis region, thereby obtaining enhanced visual features; including: The location attention mechanism is used to model the dependencies between different spatial locations in the image, thereby enhancing the spatial contrast of vascular structures and stenotic regions. The channel attention mechanism is used to redistribute the weights of feature channels to highlight blood flow velocity and key features at the high-brightness boundaries. The positional attention mechanism and channel attention mechanism are applied sequentially to output the enhanced visual features with fine-grained enhancement.

[0012] As a further improvement of the present invention, a static medical knowledge memory matrix is ​​constructed, wherein medical knowledge vectors related to renal artery stenosis are encoded, and the knowledge vectors include anatomical structure descriptions, hemodynamic parameters, and stenosis grading criteria; including: A static medical knowledge memory matrix is ​​generated using a sentence encoder. The sentence encoder embeds medical knowledge text, generating a dimension of d. model The knowledge vector, where d model For model dimensions; The medical knowledge text includes at least one of the following: normal value of renal artery peak systolic velocity, stenosis grading threshold, and description of typical signs.

[0013] As a further improvement of the present invention, the step of aligning the visual features with the static medical knowledge memory matrix through the memory alignment module to generate a knowledge alignment vector includes: The enhanced visual features are input into a multi-head attention mechanism, with the enhanced visual features serving as the query; the static medical knowledge memory matrix serves as the key and value, and a knowledge alignment vector is generated through attention-weighted calculation and multi-head concatenation. The knowledge alignment vector represents the mapping of visual features in the medical knowledge space.

[0014] As a further improvement of the present invention, a dual-memory network module is used to maintain the dynamic memory state, and combined with the knowledge alignment vector, cross-sentence context modeling is performed to ensure the logical consistency of the generated report; including: The dual-memory network module includes a dynamic memory matrix and a knowledge-enhancing writing mechanism; The dynamic memory matrix is ​​used to store the semantic context state during the generation process; The knowledge-enhanced writing mechanism uses gating to control the retention of old information and the writing of new information, and combines the knowledge alignment vector to update the memory state, so as to ensure the consistency of terminology and logical coherence between report paragraphs.

[0015] As a further improvement of the present invention, an external knowledge token is integrated into the encoding process through an external knowledge injection module, including: The static knowledge token and dynamic knowledge token are concatenated into the input of the visual feature encoder.

[0016] As a further improvement of the present invention, the report generation module is implemented using a Transformer decoder, which accesses the enhanced visual features, external knowledge tokens, and dynamic memory network states through a cross-attention mechanism; and gradually generates a renal artery ultrasound diagnostic report text, including a multi-segment structure of examination findings, diagnostic opinions, and conclusions. As a further improvement to the present invention, this method also includes a training phase: End-to-end training was performed using a dataset consisting of renal artery ultrasound images and corresponding diagnostic reports; The optimization objectives include maximizing at least one of the BLEU, CIDEr, and ROUGE-L metrics generated in the report.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention focuses on the automatic generation of renal artery ultrasound diagnostic reports. It proposes several innovations in areas such as multi-image redundant information processing, visual feature enhancement, and external medical knowledge integration. Compared with existing technologies, this invention effectively improves the accuracy of generated text, the logical consistency of long texts, and clinical effectiveness, thereby assisting ultrasound physicians in providing accurate ultrasound reports more quickly, reducing the burden on doctors, and improving clinical efficiency. Specifically: (1) This invention provides a multi-image redundancy adaptation mechanism driven by image-text consistency. By matching the correlation between image semantics and medical description semantics, multiple frames of renal artery ultrasound images are weighted and filtered, which can effectively remove blurry frames and artifact frames, so that the model can focus on key images related to stenosis and blood flow acceleration, thereby improving the quality of visual input. (2) The present invention designs a dual attention visual enhancement module, which strengthens the spatial dependence between different segments of the renal artery through position attention, and highlights key features such as blood flow velocity and vessel edge through channel attention, so that the model can more accurately capture the fine-grained lesion manifestations in the stenotic area; (3) This invention designs a dual memory network and introduces external medical knowledge to guide the text generation process. First, a static medical knowledge memory matrix is ​​constructed, which explicitly organizes the renal artery anatomy, stenosis threshold and typical signs into a knowledge matrix in a vectorized form, so that the model can refer to clinical standards, use standardized terminology and maintain the integrity of diagnostic basis when generating reports. Then, a cross-modal memory alignment module is designed, which dynamically aligns visual features with the medical knowledge matrix through multi-head attention, so that the model can "call" the medical knowledge that best matches the image content on demand, thereby reducing diagnostic illusion and improving the correspondence between report content and image. Finally, the generated text is continuously tracked and key diagnostic basis is retained through dynamic memory matrix and knowledge enhancement writing mechanism, so that long text reports maintain high stability in paragraph logic, terminology consistency and content integrity. Attached Figure Description

[0018] Figure 1 This is a model architecture diagram of an automatic generation method for ultrasound diagnostic reports of renal artery stenosis disclosed in one embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The present invention will now be described in further detail with reference to the accompanying drawings: like Figure 1 As shown, the present invention provides an automatic generation method for ultrasound diagnostic reports of renal artery stenosis, which overcomes the following problems: (1) Redundant / irrelevant images interfere with the accuracy of the report: During the renal artery ultrasound examination, a case is often accompanied by multiple dynamically captured or two-dimensional images acquired from different angles. Traditional multi-image fusion methods usually input all images into the model with "equal weight" or "random sampling". This method is prone to introducing artifacts, redundant frames and low-quality tokens, inducing hallucinatory terms, and diluting the weight of lesions. (2) Inconsistent logic in long text reports: When generating long texts with multi-segment structure for diagnostic opinions, existing methods are prone to logical errors such as inconsistent terminology, omission of key conclusions, and mismatch between parameters and diagnosis, which affect clinical usability. (3) Difficulty in effectively injecting medical knowledge into the generation process: Existing ultrasound report generation models mostly rely on pure data-driven methods and lack explicit use of medical knowledge such as renal artery stenosis grading standards, hemodynamic thresholds, and typical sign description standards, resulting in non-standard terminology, insufficient diagnostic basis, and lack of clinical credibility in the generated content. This ultimately achieved high-precision modeling of fine-grained lesion features in renal artery ultrasound images, generating standardized ultrasound reports with accurate terminology, logical consistency, and strong clinical interpretability. This significantly improved report generation efficiency and medical consistency, alleviating the contradiction between the shortage of ultrasound physicians and the continuously increasing volume of examinations. The specific process includes: S1. Input multiple renal artery ultrasound images, which are derived from a single renal artery ultrasound examination. in, A single patient examination typically includes k ultrasound images: { , ,…, The system automatically generates an ultrasound diagnostic report for renal artery stenosis using multiple ultrasound images of the patient.

[0021] S2. The image redundancy adaptation module processes multiple images, calculates the semantic relevance of each image to the diagnostic text, and selects key images based on the relevance score to suppress redundant image interference. in, The global semantic vector for each image is extracted using a pre-trained CLIP visual encoder. The diagnostic text is encoded into a text semantic vector using the CLIP text encoder. Calculate the cosine similarity between the global semantic vector of each image and the text semantic vector to obtain a semantic relevance score; The first m key images are selected based on semantic relevance scores. The weights of the first m key images are normalized by softmax and then weighted and fused to obtain the fused image features.

[0022] Specifically, (1) Perform basic visual encoding Each image, after being size-normalized, is input into a pre-trained ResNet-101 network to extract local patch pixel features. In the formula, N represents the number of patches into which the image is divided. This represents the convolutional feature dimension (2048 in this invention).

[0023] To unify the dimensions for subsequent Transformers, features are mapped to model dimensions: (2) Construct a global semantic description To determine the importance of each image for diagnosis, this invention calculates a global semantic vector for each image, using the following formula: In the formula, This vector represents the feature of the p-th patch in the i-th image. This vector can be regarded as the "global representative feature" of the entire image.

[0024] (3) Image-text correlation calculation based on CLIP To determine which images are most relevant to the diagnostic description, this invention uses the CLIP text encoder to encode the "Findings" physician descriptions in the report into semantic vectors, using the following formula: The cosine similarity between the global semantic vector of each image and the text semantic vector is calculated using the following formula: In the formula, Let represent the semantic relevance score of the i-th image, gi represent the global semantic vector of the i-th image, f represent the text semantic vector, and τ represent the learnable temperature coefficient, which is used to control the score range. The learnable temperature coefficient τ is a parameter that can be learned during model training. It is used to optimize the distribution of similarity scores and enhance the stability and adaptability of key image selection.

[0025] (4) Select the first m images with the highest similarity and perform softmax weight normalization on their scores. The formula is as follows: Weighted fusion of image features: The model assigns higher weights to images it deems more relevant, and in the method of this invention, images with the clearest, narrowest visualizations are typically given high weights.

[0026] S3. The key images after screening are subjected to feature extraction and enhancement through the visual feature enhancement module, including the application of position attention mechanism and channel attention mechanism to enhance the fine-grained feature expression of the renal artery stenosis area and obtain enhanced visual features. In order to further improve the regional expression effect of key lesions, this invention will fuse image features. Input into the dual attention structure: By modeling the dependencies between different spatial locations in an image through a position attention mechanism, the spatial contrast of vascular structures and stenotic regions can be enhanced. The weights of feature channels are reallocated through a channel attention mechanism to highlight blood flow velocity and key features at the boundary. By sequentially applying positional attention and channel attention mechanisms, fine-grained enhanced visual features are output, represented as follows: S4. Construct a static medical knowledge memory matrix, which encodes medical knowledge vectors related to renal artery stenosis. The knowledge vectors include anatomical structure descriptions, hemodynamic parameters, and stenosis grading criteria. in, A static medical knowledge memory matrix is ​​generated using a sentence encoder. The sentence encoder embeds medical knowledge text, generating a dimension of d. model The knowledge vector, where d model For model dimensions; The medical knowledge text includes at least one of the following: normal value of renal artery peak systolic velocity, threshold for stenosis classification, and description of typical signs.

[0027] Specifically, This invention designs a fixed medical knowledge base matrix for renal artery scenarios, used to record knowledge in areas such as standard descriptions of the renal artery, hemodynamic parameters, and stenosis assessment rules. This medical knowledge base originates from patient examination data and specifically includes: (1) Constructing the medical knowledge matrix M Construct an L× Static medical knowledge matrix: In the formula, L represents the number of knowledge vectors. This represents the vector representation of the j-th piece of medical knowledge. (For example...) Description of peak systolic velocity (PSV) in the renal artery. Description of normal right renal artery diameter. The knowledge vector is generated by the sentence encoder: In the formula, This represents the t-th word in the j-th sentence, and the memory matrix remains fixed during the reasoning phase.

[0028] S5. The enhanced visual features are aligned with the static medical knowledge memory matrix through the memory alignment module to generate a knowledge alignment vector; in, The enhanced visual features are input into a multi-head attention mechanism, with the enhanced visual features serving as the query. The static medical knowledge memory matrix serves as the key and value. A knowledge alignment vector is generated through attention-weighted calculation and multi-head concatenation. The knowledge alignment vector represents the mapping of visual features in the medical knowledge space.

[0029] Specifically, visual-knowledge alignment (MAM) includes: Inputting visual features into the Transformer Encoder yields: Define Query: Static medical knowledge matrix M serves as Key and Value: The knowledge alignment vector is obtained through attention weighting: The global knowledge alignment vector is formed by concatenating multiple heads. : This indicates the location of the image content from this examination within the medical knowledge base.

[0030] S6. Maintain the dynamic memory network through the dual memory network module, inject the knowledge alignment vector into the dynamic memory network, and perform cross-sentence context modeling. in, The dual-memory network module includes a dynamic memory matrix and a knowledge-enhancing writing mechanism; The dynamic memory matrix is ​​used to store the semantic context state during the generation process; The knowledge-enhanced writing mechanism uses gating to control the retention of old information and the writing of new information, and combines knowledge alignment vectors to update the memory state, so as to ensure the consistency of terminology and logical coherence between report paragraphs.

[0031] Specifically, the Dynamic Memory Network (DMN) is used to store the semantic context that gradually forms during the generation process, ensuring the structural consistency of long text generation, including: (1) Constructing a dynamic memory matrix: Where S represents the number of dynamic memory vectors, used to store semantic states at different levels. This represents the memory state at time step t-1.

[0032] The decoder output word embedding at the previous time step Composition of Query, Key, Value: Through attention, we obtain: Then update using a gated method: in, Represents the Gate of Oblivion These represent input gates, which control the retention of old information and the writing of new information, respectively.

[0033] (2) Injecting aligned medical knowledge The global knowledge vector obtained from MAM Injecting DMN: in, These are learnable parameters.

[0034] S7. External knowledge tokens are integrated into the coding process through the external knowledge injection module. External knowledge tokens include static knowledge tokens and dynamic knowledge tokens. Static knowledge tokens are derived from the textual semantics of the diagnostic text, and dynamic knowledge tokens are derived from the output of the dynamic memory network. in, The static knowledge token and the dynamic knowledge token are concatenated into the input of the visual feature encoder.

[0035] Specifically, the External Knowledge Injection Module (KIM) constructs two special knowledge tokens: (1) Static token: derived from CLIP text semantics, i.e.: text semantics of diagnostic text. (2) Dynamic token: from The output of the dynamic memory network after injecting global knowledge into the aligned vector; The two are concatenated to the encoder input, represented as: This ensures that the decoder can directly access prior medical knowledge, generating more coherent text.

[0036] S8. The report generation module outputs structured renal artery ultrasound diagnostic reports based on enhanced visual features, external knowledge tokens, and dynamic memory network states.

[0037] in, The report generation module is implemented using a Transformer decoder. The Transformer decoder accesses enhanced visual features, external knowledge tokens, and dynamic memory network states through a cross-attention mechanism; it then progressively generates the renal artery ultrasound diagnostic report text, including a multi-segment structure of examination findings, diagnostic opinions, and conclusions. Specifically, Employing a Transformer decoder, each step accesses visual information, static / dynamic medical knowledge, and semantic memory states through cross-attention, ultimately generating a sequence: The method of the present invention also includes a training phase: End-to-end training was performed using a dataset consisting of renal artery ultrasound images and corresponding diagnostic reports; The optimization objectives include maximizing at least one of the BLEU, CIDEr, and ROUGE-L metrics generated in the report.

[0038] Example The following section presents a typical research and development test scheme and experimental results of this invention, using a renal artery ultrasound report generation task as an example. All three examples of this invention utilize real clinical renal artery multi-frame ultrasound images and structured report data, and are completed under unified hardware and training configurations to accurately demonstrate the technical effects of this invention.

[0039] Example 1: Multi-frame selection experiment based on redundancy adaptation mechanism In this example, to evaluate the effectiveness of the redundancy adaptation mechanism in a multi-frame input scenario for renal artery ultrasound, this invention first extracts CLIP features from all ultrasound cross-sectional images of each case and sorts them according to their cosine similarity to the original text report. Then, the 1st, 2nd, 3rd, 4th, and 5th images with the highest similarity are selected as visual inputs, maintaining complete consistency in model structure and hyperparameters (learning rate 1e-4, batch size 16, training epochs 100) to compare the impact of different frame numbers on report generation performance. Experimental results show that the performance is optimal when selecting the first two images, with a BLEU-4 of 0.631, higher than the 0.622 of a single-frame input, and significantly better than the performance drop that occurs when selecting 3 to 5 frames (e.g., BLEU-4 is 0.626 and CIDEr is only 5.596 for 5 frames). This result demonstrates that the redundancy adaptation mechanism proposed in this invention can effectively identify the most diagnostically valuable key frames and eliminate low-quality or irrelevant images, thereby avoiding noise interference caused by multi-frame stacking, maintaining the characteristics of efficient, compact, and information-focused image input, and further improving the accuracy and stability of report generation.

[0040] Example 2: Report Generation Experiment Incorporating Structured Medicine Knowledge In this example, to evaluate the improvement effect of the knowledge-guided mechanism based on the examination findings text vector on the report generation quality, this invention, based on the image input after redundancy adaptation, encodes the complete text of the "examination findings" of each ultrasound case into a 512-dimensional knowledge vector through a pre-trained Chinese medical text encoder. This knowledge vector is then used as an additional knowledge token at the decoder, participating in attention calculation along with image features, enabling the decoder to simultaneously reference visual and structured semantic information during generation. The training parameters in this example are kept consistent (batch size 16, learning rate 1e–4, training for 100 epochs), and the generation performance of the "no knowledge vector used" and "knowledge vector used" models are compared. Experimental results show that after adding this knowledge token, the model's BLEU-4, ROUGE-L, and CIDEr are all steadily improved (CIDEr improvement of approximately 0.12–0.18). This result demonstrates that although this knowledge vector structure is simple, it can provide a stable medical semantic reference for the decoder, thereby effectively improving the quality of the final generated report.

[0041] Example 3: Cross-modal consistency enhancement experiment with the addition of a dual-memory network In this example, to verify the improvement effect of the dual-memory network on the quality of report generation, this invention, based on redundant adaptive frame selection and knowledge vector guidance, incorporates a dual-memory structure consisting of image short-term memory and text long-term memory. A gating mechanism dynamically balances local visual information and cross-sentence semantic information during the generation process. The experimental conditions were uniformly set as batch size 16, learning rate 1e–4, and training for 100 epochs. The results show that adding the dual-memory network improved BLEU-4 by approximately 0.015–0.02 and CIDEr by approximately 0.18–0.25. This experiment demonstrates that the dual-memory network can effectively integrate visual details and long-term semantics, enabling the model to maintain logical coherence when generating sentences involving multi-stage reasoning, significantly reducing medical hallucinations and improving the stability and reliability of diagnostic expressions.

[0042] Advantages of this invention: This invention addresses the issue of numerous redundant, blurred, and non-critical frames in renal artery ultrasound examinations by proposing a redundancy adaptation method based on image-text consistency. The model first extracts the global semantic representation of each ultrasound image. And encode text semantic vectors from a given medical description template. Then, the semantic similarity between the two is calculated. The importance weights of the images are generated by normalizing using softmax. Finally, a weighted strategy is used to fuse multiple images, giving key frames higher weights while automatically weakening frames with insufficient content or noise. This solves the problem that traditional ultrasound report generation methods often use fixed averaging or random selection of multiple images as input, which easily introduces artifact frames, incomplete frames, and images without diagnostic value into the model, leading to diluted visual features and further inducing terminology hallucinations or diagnostic errors. The image-text consistency screening mechanism of this invention can significantly reduce the interference of redundant information, making the input of multiple images more focused on key images with stenosis and abnormal blood flow, thus improving the overall stability and diagnostic relevance of visual coding.

[0043] This invention constructs a dual attention mechanism, including positional attention and channel attention, to enhance the expression of renal artery stenosis regions and hemodynamic features. Positional attention is used to capture the interdependence of different spatial regions, enhancing the spatial contrast of vascular structure and local stenosis. Channel attention, on the other hand, reallocates the weights of different channels based on the global response, highlighting key channels such as blood flow velocity and boundary brightness during feature learning. This addresses the problem that renal artery stenosis diagnosis relies on multi-dimensional fine-grained information, but traditional convolution or simple Transformer methods struggle to simultaneously capture spatial dependence and channel weight differences. The dual attention module of this invention effectively enhances key diagnostic features in ultrasound images, enabling the model to more accurately identify stenosis signs and blood flow abnormalities, providing higher-quality visual representations for report generation.

[0044] This invention constructs a structured medical knowledge matrix M, which encodes medical knowledge such as renal artery anatomy, ultrasound stenosis grading thresholds, and typical pathological signs. This knowledge is represented as a computable vector space through learnable embeddings; subsequently, the model utilizes a multi-head attention mechanism to integrate visual features. With medical knowledge matrix Perform association to generate a knowledge alignment vector R = Attention( This invention enables precise retrieval of knowledge within the visual semantic space. It addresses the shortcomings of existing ultrasound report generation models, which primarily rely on data-driven approaches, struggle to understand clinical standards, diagnostic thresholds, and professional terminology, and are prone to outputting content lacking medical basis or with non-standard terminology. Through explicit medical knowledge injection, this invention allows the model to make judgments based on medical standards when interpreting ultrasound images, significantly enhancing diagnostic interpretability, terminology standardization, and clinical credibility.

[0045] This invention proposes a cross-modal memory alignment module that performs deep matching between visual features and medical knowledge. Using visual features as the query and a medical knowledge matrix as the key / value pair, feature relevance is calculated through multi-head attention. This enables the model to automatically retrieve the most relevant medical knowledge elements from the knowledge base and write them into the cross-modal memory vector, forming a knowledge-enhanced input that guides diagnostic content. This addresses the problem that traditional report generation models, while potentially incorporating medical knowledge, fail to select appropriate knowledge based on the current image content, leading to improper knowledge use or knowledge failing to guide visual understanding. Through cross-modal alignment, this invention achieves precise coupling between visual semantics and knowledge semantics, ensuring a high degree of consistency between diagnostic criteria and image semantics, significantly reducing diagnostic illusions, logical biases, and knowledge misuse.

[0046] This invention designs a dual-memory network, including a dynamic memory matrix. Knowledge-enhanced writing vectors The dynamic memory matrix captures contextual history across sentences and paragraphs, enabling the model to maintain terminological consistency and logical coherence when generating long texts. The knowledge-enhancing writing mechanism jointly writes visual features, cross-modal knowledge vectors, and the current generation state into memory, ensuring that key diagnostic evidence is continuously retained and influences the generation of subsequent sentences, resulting in a more complete report text structure and more standardized content. This addresses the problems of content duplication, omission of key conclusions, inconsistencies, or logical inconsistencies that easily occur in existing models when generating long medical reports. Through the dual memory mechanism, this invention systematically solves the "global consistency" problem in long text generation, ensuring that the generated renal artery ultrasound report has a correct structure, clear paragraph logic, stable terminology, and consistency with clinical expression habits.

[0047] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatically generating ultrasound diagnostic reports for renal artery stenosis, characterized in that, include: Input multiple renal artery ultrasound images, which are derived from a single renal artery ultrasound examination; The image redundancy adaptation module processes the multiple images, calculates the semantic relevance of each image to the diagnostic text, and filters out key images based on the relevance score to suppress interference from redundant images. The key images after screening are processed by a visual feature enhancement module. This includes applying position attention and channel attention mechanisms to enhance the fine-grained feature expression of the renal artery stenosis region and obtain enhanced visual features. A static medical knowledge memory matrix is ​​constructed, which encodes medical knowledge vectors related to renal artery stenosis. The knowledge vectors include anatomical structure descriptions, hemodynamic parameters, and stenosis grading criteria. The enhanced visual features are aligned with the static medical knowledge memory matrix using a memory alignment module to generate a knowledge alignment vector. The dynamic memory network is maintained by a dual memory network module, and the knowledge alignment vector is injected into the dynamic memory network to perform cross-sentence context modeling, ensuring the logical consistency of the generated report. An external knowledge token is integrated into the encoding process through an external knowledge injection module. The external knowledge token includes a static knowledge token and a dynamic knowledge token. The static knowledge token originates from the textual semantics of the diagnostic text, and the dynamic knowledge token originates from the output of the dynamic memory network. The report generation module outputs a structured renal artery ultrasound diagnostic report based on the enhanced visual features, external knowledge tokens, and dynamic memory network state.

2. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, The process involves using an image redundancy adaptation module to process the multiple images, calculate the semantic relevance of each image to the diagnostic text, and filter out key images based on the relevance score to suppress redundant image interference; including: The global semantic vector for each image is extracted using a pre-trained CLIP visual encoder. The diagnostic text is encoded into a text semantic vector using the CLIP text encoder. Calculate the cosine similarity between the global semantic vector and the text semantic vector of each image to obtain a semantic relevance score; The first m key images are selected based on the semantic relevance scores. The weights of the first m key images are normalized by softmax and then weighted and fused.

3. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 2, characterized in that, The formula for calculating the cosine similarity between the global semantic vector of each image and the text semantic vector is as follows: in, Let represent the semantic relevance score of the i-th image, gi represent the global semantic vector of the i-th image, f represent the text semantic vector, and τ represent the learnable temperature coefficient, which is used to control the score range. The learnable temperature coefficient τ is a learnable parameter during model training, used to optimize the distribution of similarity scores and enhance the stability and adaptability of key image screening.

4. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, The visual feature enhancement module extracts and enhances features from the selected key images, including applying positional attention and channel attention mechanisms to strengthen the fine-grained feature expression of the renal artery stenosis region, resulting in enhanced visual features; including: The location attention mechanism is used to model the dependencies between different spatial locations in the image, thereby enhancing the spatial contrast of vascular structures and stenotic regions. The channel attention mechanism is used to redistribute the weights of feature channels to highlight blood flow velocity and key features at the high-brightness boundaries. The positional attention mechanism and channel attention mechanism are applied sequentially to output the enhanced visual features with fine-grained enhancement.

5. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, A static medical knowledge memory matrix is ​​constructed, which encodes medical knowledge vectors related to renal artery stenosis. These knowledge vectors include anatomical descriptions, hemodynamic parameters, and stenosis grading criteria. A static medical knowledge memory matrix is ​​generated using a sentence encoder. The sentence encoder embeds medical knowledge text, generating a dimension of d. model The knowledge vector, where d model For model dimensions; The medical knowledge text includes at least one of the following: normal value of renal artery peak systolic velocity, stenosis grading threshold, and description of typical signs.

6. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, The step of aligning the visual features with the static medical knowledge memory matrix through a memory alignment module to generate a knowledge alignment vector includes: The enhanced visual features are input into a multi-head attention mechanism, with the enhanced visual features serving as the query; the static medical knowledge memory matrix serves as the key and value, and a knowledge alignment vector is generated through attention-weighted calculation and multi-head concatenation. The knowledge alignment vector represents the mapping of visual features in the medical knowledge space.

7. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, The dynamic memory state is maintained through a dual-memory network module, and cross-sentence context modeling is performed in conjunction with the knowledge alignment vector to ensure the logical consistency of the generated report; including: The dual-memory network module includes a dynamic memory matrix and a knowledge-enhancing writing mechanism; The dynamic memory matrix is ​​used to store the semantic context state during the generation process; The knowledge-enhanced writing mechanism uses gating to control the retention of old information and the writing of new information, and combines the knowledge alignment vector to update the memory state, so as to ensure the consistency of terminology and logical coherence between report paragraphs.

8. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that: External knowledge tokens are integrated into the coding process through an external knowledge injection module, including: The static knowledge token and dynamic knowledge token are concatenated into the input of the visual feature encoder.

9. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, The report generation module is implemented using a Transformer decoder, which accesses the enhanced visual features, external knowledge tokens, and dynamic memory network states through a cross-attention mechanism; and gradually generates the renal artery ultrasound diagnostic report text.

10. The method for automatically generating ultrasound diagnostic reports for renal artery stenosis according to claim 1, characterized in that, It also includes the training phase: End-to-end training was performed using a dataset consisting of renal artery ultrasound images and corresponding diagnostic reports; The optimization objectives include maximizing at least one of the BLEU, CIDEr, and ROUGE-L metrics generated in the report.

Images