Image report generation method and device, electronic equipment and storage medium

Abstract

The invention provides an image report generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining the detection information of a user, and the detection information comprises a medical image and clinical information; obtaining abnormal sign information of each region based on the medical image and the clinical information, and obtaining description information of each focus based on the medical image; under the condition that the historical examination information of the user does not exist, screening description information of a key focus based on the description information of each focus and the clinical information; the image report is generated based on the clinical information, the abnormal sign information and the description information of the key focus, the abnormal sign information and the description information of the focus in each area are obtained by integrating the medical image and the clinical information, key details of the physical condition of a patient can be comprehensively and accurately captured, omission or misjudgment caused by a single information source is avoided, and the accuracy of the patient is improved. And the content of the image report is more complete and accurate.

Description

Technical Field

[0001] This application belongs to the field of image report generation, and in particular relates to an image report generation method, apparatus, electronic device and storage medium. Background Technology

[0002] Medical imaging reports are crucial for clinical diagnosis and treatment decisions, and their quality directly impacts the accuracy of disease diagnosis, staging, and treatment strategies. The technical approach in related technologies follows a step-by-step process of "lesion detection - structured report generation." After detecting and locating lesions using image recognition algorithms, report content is generated based on extracted features. However, these methods generally suffer from weak integration with clinical information, focusing primarily on the feature analysis of the images themselves without fully integrating key clinical data such as patient history and laboratory test results. This leads to discrepancies between the generated reports and the actual needs of clinical diagnostic decisions. Furthermore, when dealing with multiple lesions throughout the body, these methods lack intelligent screening mechanisms, potentially leading to the overlooking or underestimation of critical lesions, thus reducing the efficiency of imaging diagnosis. Summary of the Invention

[0003] In view of this, embodiments of this application provide an image report generation method, apparatus, electronic device, and storage medium, which can improve report quality and thus improve diagnostic efficiency.

[0004] In a first aspect, embodiments of this application provide an image report generation method, including: Obtain the user's detection information, which includes medical images and clinical information; Based on the medical images and clinical information, abnormal signs in each region are obtained, and descriptive information of each lesion is obtained based on the medical images. In the absence of the user's historical examination information, the description information of key lesions is screened based on the description information of each lesion and the clinical information; An imaging report is generated based on the clinical information, the abnormal signs, and the description of the key lesions.

[0005] In some embodiments, obtaining abnormal sign information based on the medical images and clinical information, and obtaining descriptive information of each lesion based on the medical images, includes: The medical images and clinical information are input into the anomaly recognition model to obtain the abnormal sign information; Extract local images of each lesion from the medical images; Extract the lesion image features from the local image; The similarity between the lesion image features and the anatomical location descriptions in the pre-constructed anatomical location dictionary is calculated to obtain the first similarity calculation result, and the anatomical location descriptions of each lesion are selected based on the first similarity calculation result. The similarity between the lesion image features and the image manifestation descriptions in the pre-constructed image manifestation lexicon is calculated to obtain a second similarity calculation result, and the image manifestation descriptions of each lesion are selected based on the second similarity calculation result. Based on the local images, the lesion parameters of each lesion are obtained; The descriptive information of each lesion is obtained based on the anatomical location description, the imaging manifestation description, and the lesion parameters.

[0006] In some embodiments, the step of filtering key lesion description information based on the description information of each lesion and the clinical information includes: Image features are extracted from local images of each lesion. Text features are extracted from the descriptive information of each lesion. The text features and image features of each lesion are fused to obtain the first fused feature; The first fusion feature is input into the clustering model to obtain the lesion grouping results; Based on the lesion grouping results and the clinical information, key lesions are screened to obtain descriptive information of the key lesions.

[0007] In some embodiments, the step of screening key lesions based on the lesion grouping results and clinical information to obtain descriptive information of the key lesions includes: The lesion groups and the clinical information are input into the first ranking model to obtain the importance ranking of each lesion group; The lesion groups and the clinical information are input into the second ranking model to obtain the importance ranking of the lesions in each lesion group; Key lesions were selected based on the importance ranking of each lesion group and the importance ranking of lesions within each lesion group, in order to obtain descriptive information about the key lesions.

[0008] In some embodiments, the method further includes: If the user's historical examination information exists, the medical image is registered with the medical image in the historical examination information; The registered medical images and the medical images in the historical detection information are matched for lesions to obtain matched lesion groups and unmatched lesion groups. Metabolic analysis of the matched lesion group and the unmatched lesion group was performed to obtain metabolic progression information of each lesion; The second fusion feature is obtained based on the descriptive information of each lesion and the metabolic progression information of each lesion; The second fusion feature is input into the clustering model to obtain the lesion grouping results; Based on the lesion grouping results and the clinical information, key lesions are screened to obtain descriptive information of the key lesions; An imaging report is generated based on the clinical information, the abnormal signs, the description of the key lesions, and the metabolic progression information of the lesions.

[0009] In some embodiments, generating an image report based on the clinical information, the abnormal sign information, and the descriptive information of the key lesions includes: Based on the medical images, the clinical information, and the descriptive information of the key lesions, similar image reports are selected from the knowledge base; Using the similar image reports as a reference, an image report is generated based on the clinical information, the abnormal signs information, and the descriptive information of the key lesions.

[0010] In some embodiments, the step of filtering similar image reports from a knowledge base based on the medical image, the clinical information, and the descriptive information of the key lesions includes: Calculate the first similarity between the medical image and the medical image corresponding to the image report in the knowledge base; Calculate the second similarity between the description information of the key lesion and the description information of the key lesion corresponding to the image report in the knowledge base; Calculate the third similarity between the clinical information and the clinical information corresponding to the image reports in the knowledge base; The first similarity, the second similarity, and the third similarity are weighted and summed to obtain the weighted similarity results for each image report; Similar image reports are selected from the image reports based on the weighted similarity.

[0011] Secondly, embodiments of this application provide an image report generation apparatus, comprising: The first acquisition module is used to acquire the user's detection information, which includes medical images and clinical information. The first determining module is used to obtain abnormal sign information of each region based on the medical images and clinical information, and to obtain descriptive information of each lesion based on the medical images. The filtering module is used to filter the description information of key lesions based on the description information of each lesion and the clinical information when the user's historical examination information is not available. The generation module is used to generate an image report based on the clinical information, the abnormal signs information, and the description information of the key lesions.

[0012] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement any of the methods described above.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the methods described above.

[0014] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes an electronic device to execute any of the methods described above.

[0015] The beneficial effects of the embodiments in this application compared with the prior art are: This application provides an image report generation method that acquires a user's detection information, including medical images and clinical information; obtains abnormal sign information for each region based on the medical images and clinical information, and obtains descriptive information for each lesion based on the medical images; in the absence of the user's historical examination information, filters descriptive information of key lesions based on the descriptive information of each lesion and the clinical information; and generates an image report based on the clinical information, the abnormal sign information, and the descriptive information of key lesions. By comprehensively acquiring abnormal sign information and lesion descriptive information for each region from medical images and clinical information, this method can comprehensively and accurately capture key details of the patient's physical condition, avoiding omissions or misjudgments caused by a single information source, making the image report more complete and accurate. It automatically filters key lesions based on lesion descriptions and clinical information, eliminating the need for doctors to manually analyze and judge each lesion, saving significant time and effort, allowing doctors to focus on key lesions more quickly, and improving diagnostic efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1A schematic diagram illustrating the implementation flow of an image report generation method provided in this application embodiment; Figure 2 A schematic diagram illustrating the implementation process of step S102 provided in an embodiment of this application; Figure 3 A schematic diagram illustrating the implementation process of step S103 provided in an embodiment of this application; Figure 4 A schematic diagram illustrating the implementation flow of an image report generation method provided in this application embodiment; Figure 5 A schematic diagram illustrating the inputs and outputs of various modules in an image report generation system provided in this application embodiment; Figure 6 A schematic diagram illustrating the implementation process of a lesion description generation module provided in an embodiment of this application; Figure 7 A schematic diagram of the analysis process of a metabolic dynamics analysis module provided in an embodiment of this application; Figure 8 A schematic diagram of the screening process of a key lesion screening module provided in an embodiment of this application; Figure 9 A schematic diagram illustrating the implementation process of the report generation module provided in this application embodiment; Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0019] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0020] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0021] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected," or "in response to detection."

[0022] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0023] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0024] Based on the problems in related technologies, this application provides an image report generation method that can be applied to electronic devices, including: mobile phones, tablets, wearable devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. Figure 1 This is a schematic diagram illustrating the implementation flow of an image report generation method provided in an embodiment of this application, as shown below. Figure 1 As shown, the image report generation method includes: Step S101: Obtain the user's detection information, which includes medical images and clinical information.

[0025] In this application embodiment, medical imaging refers to the technology and processing procedures for obtaining images of internal tissues of the human body or a part of the human body in a non-invasive manner for medical treatment or medical research. Common medical images include X-rays, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound images, and positron emission tomography (PET) images. These images can present information such as the morphology and structure of internal organs and tissues of the human body, helping doctors to detect lesions and determine the condition. Clinical information refers to various information related to the patient's clinical diagnosis and treatment process, covering the patient's basic information (such as age, gender, medical history, etc.), symptoms, laboratory test results (such as blood tests, urine tests, and other indicators), and previous treatment history.

[0026] In this embodiment, the patient's medical imaging data can be obtained through platforms such as the hospital's information system, image archiving, and communication system. Simultaneously, the patient's clinical information, including basic information, symptom descriptions, and laboratory test results, can be extracted from the hospital's electronic medical record system. Alternatively, medical images and clinical information can be directly transmitted to electronic devices in real time via specialized testing equipment or data acquisition terminals.

[0027] Step S102: Obtain abnormal sign information for each region based on the medical images and clinical information, and obtain descriptive information for each lesion based on the medical images.

[0028] In this embodiment, abnormal sign information refers to the characteristic information presented in medical images of areas that differ from the imaging appearance of normal tissues or organs. These abnormalities may indicate the presence of lesions, such as abnormal density, morphological changes, or blurred boundaries in the images. Analysis of abnormal sign information can provide a preliminary assessment of the nature and location of the lesion. Lesion description information is detailed information describing each lesion found in medical images, including its location, size, shape, density, edge features, and internal structure. Accurate lesion description information helps doctors make more precise diagnoses and assessments of lesions.

[0029] In this embodiment, image processing and machine learning algorithms can be used to analyze medical images to obtain abnormal sign information. A neural network model can be pre-trained, and medical images and clinical information can be input into the neural network model to output abnormal sign information.

[0030] In this embodiment, image segmentation technology can be used to separate each lesion in a medical image from normal tissue, and then detailed feature extraction and description can be performed on the segmented lesions. For example, by measuring the size of the lesion, calculating the density value of the lesion, and analyzing the edge features (such as smoothness, roughness, etc.) and internal structure (such as uniformity, non-uniformity, etc.) of the lesion, comprehensive descriptive information of each lesion can be formed.

[0031] Step S103: In the absence of the user's historical examination information, the description information of key lesions is screened based on the description information of each lesion and the clinical information.

[0032] In this embodiment, the descriptive information of key lesions refers to detailed descriptions of lesions selected from all detected lesions that are of significant importance to disease diagnosis, treatment, and prognostic assessment. Key lesions are typically those that have a significant impact on the patient's health and require focused attention and treatment.

[0033] In this embodiment, a key lesion screening model can be established, which comprehensively considers the descriptive and clinical information of the lesions. For example, based on factors such as the size and location of the lesion (e.g., whether it is located in an important organ or near a critical blood vessel or nerve), morphological characteristics (e.g., whether there are malignant signs), the degree of impact on the patient's symptoms, and the patient's age and underlying diseases, a corresponding weight value is assigned to each lesion. By setting a certain threshold or using a ranking algorithm, lesions with higher weight values ​​are selected as key lesions, and their descriptive information is extracted.

[0034] In some embodiments, a neural network model can be used to screen key lesions, thereby obtaining a description of the key lesions.

[0035] Step S104: Generate an image report based on the clinical information, the abnormal sign information, and the description information of the key lesions.

[0036] In this embodiment, natural language generation technology can be used to integrate and transform clinical information, abnormal sign information, and descriptive information of key lesions to generate an imaging report that conforms to medical standards and clinical practices. Simultaneously, natural language processing technology is used to perform grammatical checks and semantic optimization on the generated report to ensure its accuracy and fluency. In some embodiments, imaging reports similar to clinical information, medical images, and key lesions can be searched, and an imaging report can be generated based on these reports.

[0037] The method provided in this application acquires the user's detection information, including medical images and clinical information; obtains abnormal sign information for each region based on the medical images and clinical information, and obtains descriptive information for each lesion based on the medical images; in the absence of the user's historical examination information, filters the descriptive information of key lesions based on the descriptive information of each lesion and the clinical information; and generates an image report based on the clinical information, the abnormal sign information, and the descriptive information of key lesions. By comprehensively acquiring abnormal sign information and lesion descriptive information for each region from medical images and clinical information, the method can comprehensively and accurately capture key details of the patient's physical condition, avoiding omissions or misjudgments caused by a single information source, making the image report more complete and accurate. The automatic filtering of key lesions based on lesion descriptions and clinical information eliminates the need for doctors to manually analyze and judge each lesion, saving significant time and effort, allowing doctors to focus on core issues more quickly and accelerating the diagnostic process. By automating the acquisition of detection information, analysis of image features, and generation of reports, the time and workload of manual operations are greatly reduced. Doctors no longer need to manually organize and analyze large amounts of imaging data and clinical information, and can obtain imaging reports more quickly, thereby shortening patients' diagnostic waiting time and improving medical efficiency.

[0038] In some embodiments, Figure 2 This application provides a schematic diagram of the implementation process of step S102, as shown in the embodiment. Figure 2 Step S102 can be achieved through the following steps: Step S1021: Input the medical images and clinical information into the abnormality recognition model to obtain the abnormal sign information.

[0039] In this embodiment, the anomaly recognition model is a model built based on machine learning or deep learning algorithms. After being trained on a large amount of labeled normal and abnormal medical image data, it can automatically analyze and judge the input medical images and clinical information, identify abnormal areas in the images, and output abnormal sign information.

[0040] In this embodiment, the medical images are first preprocessed, such as through image enhancement and normalization, to improve image quality and standardize data format. Simultaneously, clinical information is organized and encoded, converting it into a numerical form that the model can process. Then, the processed medical images and clinical information are input into a pre-trained anomaly recognition model. The model extracts and analyzes features from the input data using its internal neural network structure. Based on learned normal and abnormal patterns, it determines whether abnormal regions exist in the images and outputs abnormal sign information, such as the location of the abnormal region and possible abnormal types.

[0041] Step S1022: Extract local images of each lesion from the medical images.

[0042] In this embodiment, the local image is a segmented image from a complete medical image, containing only a single lesion and a certain range of surrounding tissue. By extracting the local image, the analysis of the single lesion can be more focused, reducing the influence of other irrelevant areas.

[0043] In this embodiment, image segmentation algorithms, such as threshold-based segmentation, region-based segmentation, edge-based segmentation, or deep learning-based segmentation, can be used to process medical images. These algorithms separate individual lesions from normal tissue and extract local images containing the lesions and a certain range of surrounding tissue for subsequent detailed analysis of individual lesions.

[0044] Step S1023: Extract the lesion image features of the local image.

[0045] In this embodiment, the lesion image features are information extracted from the local image of the lesion that reflects the essential characteristics of the lesion, such as image texture features (e.g., roughness, contrast), shape features (e.g., roundness, rectangularity), and grayscale features (e.g., average grayscale value, grayscale distribution). These features help to accurately describe and classify the lesions.

[0046] In this embodiment, feature extraction algorithms can be used to process local images. For example, for texture feature extraction, methods such as gray-level co-occurrence matrix and local binary pattern can be used; for shape feature extraction, the shape of a lesion can be described by calculating parameters such as its contour, geometric center, major axis, and minor axis; for gray-level feature extraction, the average gray value and gray-level standard deviation of the local image can be calculated. Furthermore, convolutional neural networks in deep learning can automatically learn and extract high-level features from local images, which are more representative and discriminative.

[0047] Step S1024: The similarity between the lesion image features and the anatomical location descriptions in the pre-constructed anatomical location lexicon is calculated to obtain the first similarity calculation result, and the anatomical location descriptions of each lesion are selected based on the first similarity calculation result.

[0048] In this embodiment, the anatomical location lexicon is a pre-constructed vocabulary set containing accurate descriptions of various anatomical locations in the human body. The vocabulary in this lexicon covers the names and detailed location descriptions of all organs, tissues, and parts of the human body from head to toe, and is used to determine the specific anatomical location of lesions within the human body.

[0049] In this embodiment, a similarity calculation algorithm, such as cosine similarity or Euclidean distance, can be used to calculate the similarity between the lesion image features and the feature vector corresponding to each anatomical location description in the localization anatomical location dictionary. For each lesion, a set of similarity values ​​with each anatomical location description in the dictionary is obtained, i.e., the first similarity calculation result. Then, a similarity threshold is set, or the top few descriptions with the highest similarity are selected to filter out the description information that best matches the anatomical location of the lesion.

[0050] In some embodiments, lesion image features can be input into a contrastive learning network model to obtain an anatomical location description.

[0051] In this embodiment, the contrastive learning network model is based on the idea of ​​contrastive learning and designs a dual-path contrastive learning mechanism. The image encoder and text encoder respectively process the sequence of local lesion image blocks {I} i {i=1,2,3 … n} and the descriptive text sequence {D} i Let , i=1,2,3 … m} be mapped to the feature {IF} respectively. i {i=1,2,3 … n} and {DF} i The model, using an asymmetric contrastive loss, brings matching image-text pairs closer together while pushing away unmatched pairs. It also specifically masks the K most similar negative samples to avoid forced differentiation of clinically similar descriptions, enhancing the model's robustness in multi-label scenarios. Furthermore, it simultaneously performs contrastive learning between lesion description texts as auxiliary supervision. By bringing semantically similar text features closer together, it explicitly constructs semantic cluster structures of lesion descriptions in the feature space, aiming to improve the quality of text features. When encountering new vocabulary or rare descriptions in the application scenario, the text cluster associations constructed during the learning process can serve as prior knowledge to assist in image-text alignment. Thus, anatomical location descriptions can be obtained through a contrastive learning network model.

[0052] Step S1025: Calculate the similarity between the lesion image features and the image performance descriptions in the pre-constructed image performance lexicon to obtain a second similarity calculation result, and then filter out the image performance descriptions of each lesion based on the second similarity calculation result.

[0053] In this embodiment, the image manifestation lexicon is a pre-collected and organized vocabulary used to describe various lesion manifestations in medical images. It includes various characteristic descriptions of different types of lesions presented on images, such as density, regularity of shape, and clarity of boundaries, to accurately describe the appearance of lesions on images.

[0054] In this embodiment, similar to the screening of anatomical location descriptions, the similarity between the lesion image features and the feature vectors corresponding to each image manifestation description in the image manifestation vocabulary is calculated to obtain a second similarity calculation result. Based on a set threshold or by selecting descriptions with higher similarity, terms that can accurately describe the lesion's appearance on the image, such as lesion density, morphology, and boundary features, are selected.

[0055] In some embodiments, lesion image features can be input into a contrastive learning network to obtain an image representation description.

[0056] Step S1026: Obtain lesion parameters for each lesion based on the local image.

[0057] In this embodiment, lesion parameters are indicators that quantify the lesion, such as the size (e.g., length, width, area, volume), density value, and enhancement degree of the lesion. These parameters can more accurately reflect the physical characteristics of the lesion and provide important basis for the diagnosis and treatment of the disease.

[0058] In this embodiment, the size parameter of the lesion can be obtained by measuring the contour length, area, or volume of the lesion in a local image. For example, in a two-dimensional image, the pixel area of ​​the lesion can be calculated using image processing algorithms, and then the actual area of ​​the lesion can be obtained based on the conversion relationship between the pixel size of the image and the actual physical size. In a three-dimensional image, the volume of the lesion can be calculated using methods such as voxel counting. The density parameter of the lesion can be obtained by analyzing the gray values ​​of the lesion region in the local image, such as calculating the average gray value or the pixel ratio within a specific gray range. Parameters such as the degree of enhancement can be determined by comparing local images at different time points or under different imaging sequences and analyzing the changes in the gray values ​​of the lesion.

[0059] Step S1027: Obtain descriptive information for each lesion based on the anatomical location description, the imaging manifestation description, and the lesion parameters.

[0060] In this embodiment, the anatomical location description, imaging appearance description, and calculated lesion parameters of each selected lesion can be integrated and organized into a complete lesion description information according to a certain format and logic. For example, a clear and accurate lesion description can be formed by using expressions such as "the lesion is located at [anatomical location description], appears as [imaging appearance description], is approximately [size in lesion parameters], and has a density of [density in lesion parameters]".

[0061] The method provided in this application, by inputting medical images and clinical information into an anomaly recognition model and analyzing multiple aspects of information, can fully utilize the auxiliary role of clinical information in image interpretation, reduce misjudgments caused by single image information, and improve the accuracy and reliability of abnormal sign recognition. By using a similarity calculation method to match lesion image features with a localized anatomical location dictionary, the anatomical location of the lesion in the human body can be determined more accurately, providing clinicians with more precise lesion location information, which helps in formulating more reasonable treatment plans and surgical plans. Through similarity calculation with an image manifestation dictionary, accurate image manifestation description terms are selected, which can more detailed and accurately describe various features of the lesion on the image, such as density, shape, and boundary, providing richer evidence for disease diagnosis and differential diagnosis. Obtaining quantitative parameters such as lesion size and density can more objectively reflect the physical characteristics of the lesion, facilitating doctors to perform quantitative analysis and evaluation of the lesion, track changes in the lesion, and evaluate treatment effects. By integrating anatomical location descriptions, imaging findings, and lesion parameters, the generated lesion description information is more comprehensive, accurate, and detailed, providing clinicians with richer and more valuable information. This helps improve the accuracy of disease diagnosis and the effectiveness of treatment, thereby enhancing the overall quality of medical care.

[0062] In some embodiments, Figure 3 This application provides a schematic diagram of the implementation process of step S103, as shown in the embodiment. Figure 3 As shown, step S103 can be achieved through the following steps: Step S1031: Extract image features based on local images of each lesion to obtain image features.

[0063] In this embodiment, image features are information extracted from local images of the lesion that reflects the essential characteristics of the image. These features can be low-level visual features, such as color, texture, and shape, or high-level semantic features, which are automatically learned through deep learning models and used to describe various attributes of the lesion in the image.

[0064] In this embodiment, a pre-trained convolutional neural network, such as VGG or ResNet, is used to input a local image into the model and extract feature vectors from the intermediate or output layers of the model as image features. These features contain high-level semantic information about the image and can better represent the characteristics of the lesions.

[0065] Step S1032: Extract text features from the description information of each lesion to obtain text features.

[0066] In this embodiment, text features are key information extracted from the descriptive information of the lesions (usually natural language text) that can represent the semantics and structure of the text. These features can be word frequency features or TF-IDF features in a bag-of-words model, or word vectors (such as vectors generated by Word2Vec, GloVe, etc.) or semantic features extracted through more complex natural language processing models (such as BERT, etc.). In this embodiment of the application, text features can be extracted using methods such as bag-of-words model, word vector methods, and deep learning natural language processing models.

[0067] Step S1033: The text features and image features of each lesion are fused to obtain the first fused feature.

[0068] In this embodiment, the first fusion feature is a new feature formed by combining the textual and image features of the lesion using a specific fusion method. This fusion feature integrates information from two different modalities, text and image, and can more comprehensively describe the characteristics of the lesion.

[0069] In this embodiment, text feature vectors and image feature vectors can be directly concatenated to form a new, longer-dimensional feature vector as the first fused feature. In some embodiments, a weight can be assigned to each of the text and image features, and then the weighted feature vectors are summed to obtain the first fused feature. The weight assignment can be set according to the actual application scenario and experience, or the optimal weights can be learned by training a simple model. In some embodiments, an attention mechanism is introduced, allowing the model to automatically learn the correlation between text and image features, and to perform weighted fusion of features from different modalities based on the correlation. This method can better capture the interaction information between features from different modalities and improve the quality of the fused feature.

[0070] Step S1034: Input the first fusion feature into the clustering model to obtain the lesion grouping results.

[0071] In this embodiment, the clustering model is an unsupervised learning model whose purpose is to group samples in the dataset according to similarity, so that samples within the same group have high similarity, while samples between different groups have low similarity. The clustering model is used to divide lesions into different groups based on a first fusion feature.

[0072] In this embodiment, the first fused feature can be appropriately preprocessed, such as normalized or standardized, to ensure that different features have the same scale, facilitating clustering model processing. Commonly used clustering algorithms include K-Means, hierarchical clustering, and DBSCAN. For example, when using the K-Means algorithm, the number of clusters K needs to be specified in advance, and then samples are assigned to different clusters through iterative optimization, minimizing the distance between samples within the same cluster and maximizing the distance between samples in different clusters. Hierarchical clustering constructs a hierarchical structure by continuously merging or splitting clusters, ultimately obtaining clustering results at different levels. DBSCAN is a density-based clustering algorithm that can discover clusters of arbitrary shapes and identify noise points. The preprocessed first fused feature is used to generate a new feature vector, which is then input into the selected clustering model, and the clustering model obtains the grouping results of the lesions.

[0073] Step S1035: Based on the lesion grouping results and clinical information, key lesions are screened to obtain descriptive information of the key lesions.

[0074] In this embodiment, the key lesion is the lesion that has a more significant impact on the diagnosis, treatment, or prognostic assessment of the disease among numerous lesions. Screening for key lesions helps physicians focus their attention on the most important lesions, improving the efficiency and accuracy of diagnosis and treatment.

[0075] In this embodiment, lesions in each group are evaluated and screened according to set rules to determine key lesions. Then, the descriptive information corresponding to the key lesions is extracted from the original lesion description information.

[0076] The method provided in this application extracts and fuses image and text features, comprehensively utilizing both visual and semantic information of the lesions. This allows for a more comprehensive and accurate description of the lesion's characteristics, avoiding the limitations of single-modal information. Inputting the fused features into a clustering model for lesion grouping ensures that the grouping results consider the inherent characteristics of the lesions, improving the rationality and accuracy of the grouping. Based on reasonable lesion grouping results and clinical information, key lesions can be screened more accurately, identifying those with significant impact on disease diagnosis and treatment. This helps doctors focus on critical lesions, improving diagnostic and treatment efficiency, reducing unnecessary examinations and treatments, and enhancing the utilization efficiency of medical resources.

[0077] In some embodiments, step S1035 can be implemented by the following steps: Step S351: Input the lesion grouping and clinical information into the first ranking model to obtain the importance ranking of each lesion group.

[0078] In this embodiment, the first ranking model is a model for assessing and ranking the importance of lesion groups. It receives lesion group-related information and clinical information as input, and outputs the ranking of the importance of each lesion group in the whole through a specific algorithm and training mechanism.

[0079] Step S352: Input each lesion group and the clinical information into the second ranking model to obtain the importance ranking of the lesions in each lesion group.

[0080] In this embodiment, the second ranking model is specifically designed to assess and rank the importance of lesions within each lesion group. It takes each lesion and clinical information within a single lesion group as input, analyzes the relative importance of each lesion within that group, and provides the ranking results.

[0081] Step S353: Based on the importance ranking of each lesion group and the importance ranking of lesions in each lesion group, key lesions are selected to obtain descriptive information of key lesions.

[0082] In this embodiment of the application, the group with the highest importance in the lesion group can be selected, and the most important lesion in each lesion group can be selected to obtain the key lesion.

[0083] The method provided in this application sorts lesion groups by importance using a first sorting model, which can identify lesion groups with a significant impact on the overall disease from a macroscopic perspective. Then, a second sorting model is used to finely sort lesions within each group, identifying the most critical lesions within each group from a microscopic perspective. This hierarchical screening method comprehensively considers the importance of lesions between and within groups, making it more accurate than screening methods based on a single feature or simple rules, and can more accurately identify lesions with a key impact on clinical decisions. When faced with a large amount of lesion information, doctors often need to spend a lot of time and energy analyzing and judging. This method automatically filters out key lesions and provides detailed descriptive information, providing doctors with a focus area, enabling them to quickly understand the core pathological conditions of the disease, thereby saving diagnostic time and improving the efficiency of clinical decision-making. Accurate screening of key lesions helps doctors analyze the essential characteristics of the disease more comprehensively and deeply, avoiding misdiagnosis or missed diagnosis due to neglecting important lesions. At the same time, detailed descriptive information provides doctors with rich diagnostic evidence, helping them make more accurate diagnoses, formulate more reasonable treatment plans, and improve the treatment effect and patient prognosis.

[0084] In some embodiments, this application further provides a method for generating an image report. Figure 4 This application provides a schematic flowchart of an image report generation method, which includes the following steps after step S102: Step S106: If the user's historical examination information exists, register the medical image with the medical image in the historical examination information.

[0085] In this embodiment, the user's historical examination information refers to a series of relevant data and information generated during past medical examinations, including historical medical images (such as X-ray, CT, MRI, etc.), examination reports (recording the diagnosis results, abnormal signs, etc.), and clinical information (such as the patient's age, gender, medical history, symptoms, etc.). Medical image registration is the process of spatially aligning two or more medical images acquired at different times, with different devices, or from different perspectives, so that they are geometrically consistent for subsequent comparative analysis. For example, registering the current CT image with historical CT images ensures that the same anatomical structures are in the same position in both images.

[0086] In this embodiment, feature points (such as corner points, edge points, etc.) can be extracted from current medical images and historical medical images. Then, feature matching algorithms (such as SIFT, SURF, etc.) are used to find corresponding feature point pairs in the two images. Finally, spatial transformation parameters (such as affine transformation parameters) are calculated based on these point pairs to achieve image registration.

[0087] Step S107: Perform lesion matching on the registered medical images and the medical images in the historical detection information to obtain matched lesion groups and unmatched lesion groups.

[0088] In this embodiment, the matched lesion group and the unmatched lesion group are two sets of lesions obtained after performing lesion matching analysis on the registered current medical image and historical medical images. The matched lesion group refers to the set of lesions in which corresponding positions and similar features can be found in the two images; the unmatched lesion group is the set of lesions that appear in the current image but cannot be found in the historical image, or that exist in the historical image but have disappeared in the current image.

[0089] In this embodiment, an image segmentation algorithm can be used to detect and segment lesion regions from the registered current image and historical images, respectively, to obtain a binary mask image of the lesion. Morphological features of the lesion (such as area, perimeter, roundness, major axis, minor axis, etc.) are extracted, and the feature similarity between lesions in the current image and historical images is calculated. A set threshold is used to determine whether a match exists; matched lesions are grouped into a matched lesion group, and unmatched lesions are grouped into an unmatched lesion group. Considering the spatial location information of the lesion in the image, and combining the transformation relationship after registration, it is determined whether the spatial locations of the lesions in the current image and historical images correspond, thereby performing matching and grouping.

[0090] Step S108: Perform lesion metabolic analysis on the matched lesion group and the unmatched lesion group to obtain metabolic progression information of each lesion.

[0091] In this embodiment, lesion metabolic analysis is a process of assessing changes in the physiological and pathological state of lesions by analyzing metabolic-related indicators (such as the concentration of metabolites and metabolic rates). For example, the uptake of radioactive tracers in PET-CT images can be used to analyze the metabolic activity of lesions. Metabolic progression information describes the changes in the metabolic state of lesions at different time points, reflecting the development trend of lesions, such as increased, decreased, or stable metabolic activity, and can be used to determine the malignancy of lesions and the effectiveness of treatment. Metabolic progression information may include: newly emerging lesion group, emerging lesion group, changing lesion group, stable lesion group, etc. In some embodiments, metabolic progression information may also include: metabolic-time trajectory curves.

[0092] In this embodiment, for matched lesion groups, the metabolic indicators of corresponding lesions in the current image and historical images are compared, the rate of change of metabolic indicators is calculated, and the metabolic progression of the lesions is judged based on the magnitude and trend of the rate of change, such as increased, decreased, or stable metabolic activity. For unmatched lesion groups, the metabolic activity of newly appearing lesions in the current image can be preliminarily judged based on their metabolic indicators; lesions that existed in historical images but disappeared in the current image can be considered to have decreased or disappeared metabolic activity.

[0093] Step S109: A second fusion feature is obtained based on the description information of each lesion and the metabolic progression information of each lesion.

[0094] In this embodiment of the application, the second fusion feature is a comprehensive feature vector obtained by fusing the descriptive information of each lesion (such as the location, size, and morphology of the lesion) and the metabolic progression information of each lesion. It integrates multiple aspects of information such as the morphology and metabolism of the lesion.

[0095] In this embodiment, morphological features (such as area and perimeter as mentioned above) and texture features (such as contrast and correlation extracted from the gray-level co-occurrence matrix) are extracted from the lesion description information; features such as metabolic change rate are extracted from metabolic progression information. The extracted morphological features, texture features, and metabolic features are concatenated or weighted fusion is used to obtain a second fused feature vector. For example, the features are arranged into a long vector in a certain order, or different weights are assigned to the features according to their importance and then added together to obtain the fused feature.

[0096] Step S110: Input the second fusion feature into the clustering model to obtain the lesion grouping results.

[0097] In this embodiment, the clustering model is an unsupervised learning algorithm used to group samples in a dataset according to similarity, resulting in high similarity among samples within the same group and low similarity among samples in different groups. In medical image analysis, lesions can be divided into different groups based on the second fusion feature.

[0098] In this embodiment, the second fusion feature (which can be appropriately encoded, such as one-hot encoding of the categorical variable) can be used to generate a feature vector, which is then input into the selected clustering model to finally obtain the lesion grouping result.

[0099] Step S111: Based on the lesion grouping results and clinical information, key lesions are screened to obtain descriptive information of the key lesions.

[0100] In this embodiment, lesions with high importance can be selected as key lesions based on the importance of lesion groups and the importance of lesions within each group. Similarly, the importance ranking of each lesion group and the importance ranking of lesions within each lesion group can be obtained using a first ranking model and a second ranking model.

[0101] Step S112: Generate an image report based on the clinical information, the abnormal sign information, the description information of the key lesions, and the metabolic progression information of the lesions.

[0102] In this embodiment, the extracted clinical information, abnormal signs, descriptions of key lesions, and metabolic progression information are input into a large model, and a complete imaging report is generated according to the report template format. Natural language processing technology can be used to appropriately organize and refine the information, making the report more standardized and readable.

[0103] The method provided in this application, by registering and matching current medical images with historical images, can more accurately observe changes in lesions. Combined with metabolic progression information obtained from metabolic analysis, it provides doctors with more comprehensive dynamic information about lesions, helping them to more accurately determine the nature (e.g., benign or malignant), development stage, and treatment effect of lesions, thereby improving diagnostic accuracy. Considering the user's historical examination information and individual differences, lesions are analyzed and grouped in detail, key lesions are selected, and personalized imaging reports are generated. Doctors can develop treatment plans more suited to the patient's condition based on the detailed information in the report, achieving personalized medicine. Metabolic analysis can detect early metabolic changes in lesions, even when morphological changes are not obvious. Combining historical image comparison can identify disease progression trends earlier, providing doctors with a basis for early intervention, which helps improve the cure rate and patient survival rate. The automated image registration, lesion matching, metabolic analysis, and report generation process reduces the workload of doctors manually analyzing and writing reports, saving time and energy, allowing doctors to devote more time to patient diagnosis and treatment decisions, thus improving medical efficiency.

[0104] In some embodiments, step S105 can be implemented by the following steps: Step S1051: Based on the medical images, the clinical information, and the description information of the key lesions, similar image reports are selected from the knowledge base.

[0105] In this embodiment, the knowledge base is a database storing a large number of medical imaging reports and related medical knowledge. These imaging reports contain information such as image descriptions, diagnostic conclusions, and treatment recommendations for various diseases and conditions, and may also be associated with relevant clinical information, imaging features, and other knowledge to assist doctors in diagnosis and report generation. Similar imaging reports are stored imaging reports in the knowledge base that have similar characteristics to the case for which a report is to be generated in terms of clinical information, key lesion descriptions, etc. These similar reports can provide reference and guidance for the generation of the imaging report for the current case.

[0106] In this embodiment, a first similarity can be calculated between the medical image and the corresponding medical image in the image report in the knowledge base; a second similarity can be calculated between the description information of the key lesion and the description information of the key lesion in the image report in the knowledge base; a third similarity can be calculated between the clinical information and the clinical information in the image report in the knowledge base; a weighted sum is performed based on the first similarity, the second similarity, and the third similarity to obtain a weighted similarity result for each image report; and similar image reports are selected from the image reports based on the weighted similarity.

[0107] In this embodiment of the application, the image report with the highest similarity weight can be selected as the similar image report.

[0108] Step S1052: Using the similar image report as a reference, generate an image report based on the clinical information, the abnormal sign information, and the description information of the key lesions.

[0109] In this embodiment, a common report template structure can be extracted from the selected similar image reports, including the format and content organization of each part of the report (such as patient information, examination methods, imaging manifestations, diagnostic opinions, treatment recommendations, etc.). A large model can be invoked, which extracts relevant text based on clinical information, the abnormal sign information, and the description information of the key lesions, and generates an image report based on the style of similar image reports.

[0110] The method provided in this application avoids doctors having to write reports from scratch by selecting similar imaging reports from a knowledge base, saving significant time and effort. Doctors can quickly obtain report frameworks and common expressions for similar cases, and modify and improve them accordingly, greatly shortening report generation time and improving work efficiency. The imaging reports in the knowledge base are usually written and reviewed by professional doctors, possessing a certain degree of standardization and accuracy. Reports generated using these similar reports as references can adhere to certain standards in terms of format, content, and diagnostic reasoning, reducing errors caused by doctors' lack of experience or negligence, and ensuring report quality.

[0111] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0112] Based on the foregoing embodiments, this application provides an image report generation system. The system's modules and units can be implemented using a processor in a computer device; alternatively, they can be implemented using specific logic circuits. During implementation, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc. This application also provides an image report generation method applied to an image report generation system. This system includes: an abnormal sign recognition mode, a lesion detection module, a historical examination retrieval mode, a similar case retrieval model, a lesion description generation module, a dynamic metabolic analysis module, a key lesion screening module, and a report generation module. Figure 5 This application provides a schematic diagram illustrating the inputs and outputs of various modules in an image report generation system, as shown in the embodiments of this application. Figure 5 As shown, the currently examined PET / CT image data and clinical information are processed along several pathways: on one hand, they are input into the abnormal sign recognition module to generate abnormal sign information; on the other hand, they are input into the lesion detection module to obtain systemic lesions. These systemic lesions are further input into the lesion description generation module to generate detailed lesion information, including location and uptake information. Subsequently, they are input into the key lesion screening module to obtain the lesion criticality ranking and generate an ordered description of systemic lesions. Simultaneously, the current examination information triggers the historical examination retrieval module and the similar case retrieval module in the multi-disease knowledge base to obtain multi-time point examination information and similar case reports, respectively. If multi-time point examination information is obtained, the corresponding lesion information will be input into the dynamic metabolic analysis module along with the currently examined lesion detailed information. The generated lesion metabolic progression information will be sent back to the key lesion screening module to generate a dynamic progression description of the lesions. Ultimately, the description of systemic lesions (or the dynamic progression description of existing lesions) and abnormal signs information are input into the report generation module to generate an imaging report that conforms to clinical norms and narrative style, with reference to similar case reports; at the same time, the abnormal signs information and the description of systemic lesions directly generate a structured report, providing a clear record of lesions.

[0113] The following is a detailed explanation of each module: Figure 6 This is a schematic diagram illustrating the implementation process of a lesion description generation module provided in an embodiment of this application, as shown below. Figure 6 As shown, the execution flow of the lesion description generation module is as follows: The lesion detection module is used to input imaging data and clinical information to detect individual lesions. These lesions are then input into the lesion description module to generate lesion description information.

[0114] The lesion description generation module takes as input PET image blocks, CT image blocks, and lesion segmentation result image blocks. Through a contrastive learning algorithm, it calculates the similarity between the extracted lesion image features and a pre-built anatomical location vocabulary, selecting the n anatomical location descriptions with the highest matching degree. From a pre-built image manifestation vocabulary (covering dimensions such as density and morphology, e.g., "nodular," "low-density shadow," "rough edges"), it matches the m image manifestation descriptions that best fit the lesion image features. The feature matching process ensures accuracy through sorting and setting similarity thresholds. Finally, combined with lesion measurement parameters, it generates lesion description text (same as the lesion information in the image).

[0115] The similar case retrieval module and the historical examination module perform searches based on a pre-built knowledge base storing multi-disease image-report-clinical information. The similar case retrieval module performs joint searches based on multimodal information, including images, reports, and clinical information. It constructs a multimodal feature-weighted similarity calculation, including calculating image cosine similarity for image information, semantic similarity for text information, and feature similarity based on key lesion structured information. The results are sorted according to the weighted similarity, and reports of similar cases that meet the similarity threshold are used as reference text. When the similarity of all image reports in the knowledge base does not meet the requirements, a negative template is used as the reference text. The historical examination module directly searches based on identifiable identifiers such as examination numbers or patient IDs.

[0116] The metabolic dynamics analysis module generates metabolic progress information. Figure 7 This is a schematic diagram of the analysis flow of a metabolic dynamics analysis module provided in an embodiment of this application, as shown below. Figure 7 As shown, based on current examination information and historical examination image data, the system first performs lesion matching between the examination information and historical examination information from multiple time points to obtain matched lesion groups and unmatched lesion groups. Subsequently, metabolic analysis of the matched and unmatched lesion groups is carried out. Based on the differences in metabolic status and following clinical analysis practices, the system automatically completes the grouping and outputs lesion metabolic progress information including categories such as newly emerging lesion groups, disappearing lesion groups, changing lesion groups, and stable lesion groups. At the same time, a "metabolism-time" trajectory curve is generated by combining the characteristics of lesion metabolic changes, realizing a systematic analysis and visualization of the dynamic changes in lesion metabolism.

[0117] The key lesion screening module performs key lesion screening. Figure 8 This is a schematic diagram of the screening process of a key lesion screening module provided in an embodiment of this application, as shown below. Figure 8As shown in the figure, the screening process is as follows: Each lesion image block and its corresponding lesion description (or lesion metabolic progression description) is used to extract features of each lesion; these features are then combined with clinical information and fed into an adaptive clustering system to obtain lesion grouping results; the grouping results are processed by a ranking network to obtain the group importance ranking results and the lesion importance ranking results within each group; based on the ranking results, the lesion group with the highest importance ranking and the lesions with the highest importance ranking in each of the remaining groups are designated as key lesions, thereby obtaining descriptive information for the key lesions.

[0118] The whole-body abnormal signs module classifies abnormal signs in major anatomical regions based on PET / CT data to obtain the status and description of abnormal signs in each region. These abnormal signs include, but are not limited to, "calcification," "proliferation," "edema," and "no abnormalities." The abnormal sign information is integrated with detailed information on lesions throughout the body to generate a structured report.

[0119] Figure 9 This is a schematic diagram illustrating the implementation process of the report generation module provided in an embodiment of this application, as shown below. Figure 10 As shown, the report generation module will use similar case reports (similar imaging reports in the above embodiments) as the basis for report writing style, integrate the structured information and clinical information of the current examination data, and output the final imaging report. Where historical examination data exists, the report generation style is optional; it can generate an independent imaging report or a follow-up-style imaging report with efficacy evaluation.

[0120] The image report generation system provided in this application can realize the intelligent generation of image reports. It can not only obtain reasonable structured lesion information, but also generate natural language reports that meet clinical requirements and styles. It can also realize intelligent report generation and efficacy evaluation for follow-up data.

[0121] The system provided in this application addresses the issues of accuracy and universality in lesion description by constructing a lesion description prediction network based on contrastive learning to achieve accurate characterization of lesions throughout the body. To address the problem of insufficient flexibility in lesion description grouping (e.g., existing schemes often group based on location, making it difficult to classify high-risk lesions according to distribution characteristics or metastasis paths), an adaptive clustering scheme is adopted for lesion grouping, providing reliable support for the screening of key lesions. To address the problems of manual comparison of images at multiple time points, insufficient metabolic quantification, inefficient reporting, and weak clinical relevance, an automated dynamic metabolic analysis module is designed to intelligently generate dynamic metabolic progress information of lesions. Finally, to address the issue of reporting styles being difficult to adapt to different medical centers and the need for follow-up comparison, a report generation method integrating diverse information is proposed.

[0122] Based on the foregoing embodiments, this application further provides an image report generation apparatus, including: The first acquisition module is used to acquire the user's detection information, which includes medical images and clinical information. The first determining module is used to obtain abnormal sign information of each region based on the medical images and clinical information, and to obtain descriptive information of each lesion based on the medical images. The filtering module is used to filter the description information of key lesions based on the description information of each lesion and the clinical information when the user's historical examination information is not available. The generation module is used to generate an image report based on the clinical information, the abnormal signs information, and the description information of the key lesions.

[0123] In addition, the image report generation device described above can be a software unit, a hardware unit, or a combination of software and hardware. It can also be integrated into an electronic device as a separate component, or exist as a standalone terminal device.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0125] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 As shown, the electronic device of this embodiment may include: at least one processor 30 ( Figure 10 Only one processor 30, memory 31, and computer program 32 stored in memory 31 and executable on at least one processor 30 are shown. When the processor 30 executes the computer program 32, it implements the steps in any of the above method embodiments, or the processor 30 executes the computer program 32 to implement the functions of each module / unit in the above device or system embodiments.

[0126] For example, computer program 32 may be divided into one or more modules / units, one or more of which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units may be a series of computer program 32 instruction segments capable of performing a specific function, which describe the execution process of computer program 32 in an electronic device.

[0127] This application also provides a computer-readable storage medium storing a computer program 32, which, when executed by a processor 30, implements the steps described in the above-described method embodiments.

[0128] This application provides a computer program product that, when run on an electronic device, enables the electronic device to perform the steps described in the various method embodiments above.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program 32 instructing related hardware. The computer program 32 can be stored in a computer-readable storage medium, and when executed by the processor 30, it can implement the steps of the various method embodiments described above. The computer program 32 includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a terminal, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An image report generation method characterized by comprising: include: Obtain the user's detection information, which includes medical images and clinical information; Based on the medical images and clinical information, abnormal signs in each region are obtained, and descriptive information of each lesion is obtained based on the medical images. In the absence of the user's historical examination information, the description information of key lesions is screened based on the description information of each lesion and the clinical information; An imaging report is generated based on the clinical information, the abnormal signs, and the description of the key lesions.

2. The method according to claim 1, characterized in that, The process of obtaining abnormal sign information based on the medical images and clinical information, and obtaining descriptive information of each lesion based on the medical images, includes: The medical images and clinical information are input into the anomaly recognition model to obtain the abnormal sign information; Extract local images of each lesion from the medical images; Extract the lesion image features from the local image; The similarity between the lesion image features and the anatomical location descriptions in the pre-constructed anatomical location dictionary is calculated to obtain the first similarity calculation result, and the anatomical location descriptions of each lesion are selected based on the first similarity calculation result. The similarity between the lesion image features and the image manifestation descriptions in the pre-constructed image manifestation lexicon is calculated to obtain a second similarity calculation result, and the image manifestation descriptions of each lesion are selected based on the second similarity calculation result. Based on the local images, the lesion parameters of each lesion are obtained; The descriptive information of each lesion is obtained based on the anatomical location description, the imaging manifestation description, and the lesion parameters.

3. The method according to claim 1, characterized in that, The descriptive information for screening key lesions based on the descriptive information of each lesion and the clinical information includes: Image features are extracted from local images of each lesion. Text features are extracted from the descriptive information of each lesion. The text features and image features of each lesion are fused to obtain the first fused feature; The first fusion feature is input into the clustering model to obtain the lesion grouping results; Based on the lesion grouping results and clinical information, key lesions are screened to obtain descriptive information of the key lesions.

4. The method according to claim 3, characterized in that, The step of screening key lesions based on the lesion grouping results and clinical information to obtain descriptive information of the key lesions includes: The lesion groups and the clinical information are input into the first ranking model to obtain the importance ranking of each lesion group; The lesion groups and the clinical information are input into the second ranking model to obtain the importance ranking of the lesions in each lesion group; Key lesions were selected based on the importance ranking of each lesion group and the importance ranking of lesions within each lesion group, in order to obtain descriptive information about the key lesions.

5. The method according to claim 1, characterized in that, The method further includes: If the user's historical examination information exists, the medical image is registered with the medical image in the historical examination information; The registered medical images and the medical images in the historical detection information are matched for lesions to obtain matched lesion groups and unmatched lesion groups. Metabolic analysis of the matched lesion group and the unmatched lesion group was performed to obtain metabolic progression information of each lesion; The second fusion feature is obtained based on the descriptive information of each lesion and the metabolic progression information of each lesion; The second fusion feature is input into the clustering model to obtain the lesion grouping results; Based on the lesion grouping results and the clinical information, key lesions are screened to obtain descriptive information of the key lesions; An imaging report is generated based on the clinical information, the abnormal signs, the description of the key lesions, and the metabolic progression information of the lesions.

6. The method according to claim 1, characterized in that, The generation of the image report based on the clinical information, the abnormal signs information, and the descriptive information of the key lesions includes: Based on the medical images, the clinical information, and the descriptive information of the key lesions, similar image reports are selected from the knowledge base; Using the similar image reports as a reference, an image report is generated based on the clinical information, the abnormal signs information, and the descriptive information of the key lesions.

7. The method according to claim 6, characterized in that, The step of filtering similar image reports from the knowledge base based on the medical images, the clinical information, and the descriptive information of the key lesions includes: Calculate the first similarity between the medical image and the medical image corresponding to the image report in the knowledge base; Calculate the second similarity between the description information of the key lesion and the description information of the key lesion corresponding to the image report in the knowledge base; Calculate the third similarity between the clinical information and the clinical information corresponding to the image reports in the knowledge base; The first similarity, the second similarity, and the third similarity are weighted and summed to obtain the weighted similarity results for each image report; Similar image reports are selected from the image reports based on the weighted similarity.

8. An image report generation device, characterized in that, include: The first acquisition module is used to acquire the user's detection information, which includes medical images and clinical information. The first determining module is used to obtain abnormal sign information of each region based on the medical images and clinical information, and to obtain descriptive information of each lesion based on the medical images. The filtering module is used to filter the description information of key lesions based on the description information of each lesion and the clinical information when the user's historical examination information is not available. The generation module is used to generate an image report based on the clinical information, the abnormal signs information, and the description information of the key lesions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images