Medical image association labeling method and storage medium

Abstract

The invention discloses a medical image association labeling method and a storage medium. The method comprises the following steps: acquiring first medical image data of a target object, and determining multiple pieces of second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each piece of second medical image data according to a preset association matching rule; constructing an image group according to the first medical image data and the plurality of pieces of second medical image data, and determining an image association identifier corresponding to each piece of second medical image data according to the first association relationship; and according to the image association identifier, performing association display on the first medical image data and the plurality of second medical image data in the image group, thereby realizing medical image automatic association matching, structured assembly and precise identifier linkage display, and remarkably improving diagnosis consulting efficiency and image utilization effect.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for associating and annotating medical images and a storage medium. Background Technology

[0002] With the widespread use of medical imaging equipment, multimodal and multi-temporal image data are growing explosively, and clinicians urgently need to efficiently integrate related images to assist in accurate diagnosis and treatment decisions.

[0003] Existing solutions largely rely on manual screening or matching images based on simple filenames and examination times, lacking intelligent association rules based on image content. Doctors or technical personnel need to switch between multiple independent windows to view different sequences or modalities of images, which is cumbersome and prone to missing key association information, making it difficult to intuitively grasp the evolution of lesions or the correspondence of cross-modal features. Summary of the Invention

[0004] This invention provides a medical image association annotation method and storage medium to solve the problems of fragmented display of multi-source medical images and difficulty in intuitively grasping the evolution of lesions.

[0005] According to one aspect of the present invention, a method for associating and annotating medical images is provided, characterized in that it includes: Acquire first medical image data of the target object, and determine multiple second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each second medical image data according to a preset association matching rule; An image group is constructed based on the first medical image data and multiple second medical image data, and an image association identifier corresponding to each second medical image data is determined based on the first association relationship. The first medical image data and multiple second medical image data in the image group are associated and displayed according to the image association identifier.

[0006] According to another aspect of the present invention, a medical image association annotation device is provided, characterized in that it comprises: The first association relationship determination module is used to acquire the first medical image data of the target object, and determine multiple second medical image data associated with the first medical image data and the first association relationship between the first medical image data and each second medical image data according to the preset association matching rules. The association identifier determination module is used to construct an image group based on the first medical image data and multiple second medical image data, and to determine the image association identifier corresponding to each second medical image data according to the first association relationship; The display module is used to associate and display the first medical image data and multiple second medical image data in the image group according to the image association identifier.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the medical image association annotation method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the medical image association annotation method according to any embodiment of the present invention.

[0009] According to another aspect of the present invention, embodiments of this disclosure also provide a computer program product, including a computer program that, when executed by a processor, implements the medical image association annotation method as described in any of the embodiments of this disclosure.

[0010] The technical solution of this invention acquires first medical image data of a target object, determines multiple second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each of the second medical image data according to a preset association matching rule, thereby matching and determining the association relationship of medical images and improving image association efficiency. Next, an image group is constructed based on the first medical image data and the multiple second medical image data, and an image association identifier corresponding to each of the second medical image data is determined according to the first association relationship. By constructing the image group and generating corresponding association identifiers, structured organization and precise differentiation of medical images can be achieved. Finally, the first medical image data and the multiple second medical image data in the image group are displayed in association according to the image association identifiers. This allows for the linked display of multiple medical images, solving the problems of fragmented source medical image association display and difficulty in intuitively grasping lesion evolution. It achieves automatic association matching, structured grouping, and precise identifier-linked display of medical images, significantly improving diagnostic review efficiency and image utilization.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a medical image association annotation method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a medical image association annotation method provided in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the structure of a medical image association annotation device according to Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device that implements the medical image association annotation method of this invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0016] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0017] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0018] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0019] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0020] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0021] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0022] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0023] Example 1 Figure 1 The flowchart of a medical image association annotation method is provided in Embodiment 1 of the present invention. This embodiment is applicable to the association retrieval, structured organization and linkage display of multimodal and multi-sequence medical images. The method can be executed by a medical image association annotation device, which can be implemented in hardware and / or software. Optionally, it can be implemented by an electronic device, such as a mobile terminal, PC or server.

[0024] like Figure 1 As shown, the method may specifically include: S110. Obtain the first medical image data of the target object, and determine multiple second medical image data associated with the first medical image data and the first association relationship between the first medical image data and each second medical image data according to the preset association matching rules.

[0025] The target object can be understood as the main object currently being focused on and processed. In a medical context, it can be a patient or a specific examination / case. In certain research or specific contexts, it may also refer to laboratory animals, ex vivo organ samples, etc., used to clarify the ownership of medical image data and determine "whose, which part" this batch of medical images belongs to, serving as the main object for subsequent association, matching, and display. The first medical image data can be understood as the raw medical image data obtained from the target object, serving as the benchmark / main image. The first medical image data can be image types such as CT / MRI / ultrasound / pathological slides, serving as the core reference image for matching, association, and comparison with other medical images. The first medical image data can also include multiple disease types, such as tumors, cardiovascular diseases, and nervous system diseases. The association and matching rules can be understood as pre-set logic, conditions, or algorithms used to determine whether and how different medical images are related. The system automatically establishes the basis for relationships between images to determine "which images are bound together according to what standards." The second medical image data can be understood as at least one other medical image data that is related to the first medical image data, selected through association matching rules. This second image data serves as a reference image for the first medical image, aiding in comparison, analysis, or diagnosis. The first association relationship can be understood as the correspondence between the first medical image data and each second medical image data determined by the association matching rules. It describes the relationship between the two images and serves as the direct basis for subsequent labeling, grouping, and related display.

[0026] Based on the above scheme, optionally, the association matching rule includes multiple levels of sub-rules; determining the first association relationship between the first medical image data and multiple second medical image data according to the preset association matching rule includes: for each second medical image data, determining the target sub-rule corresponding to the first medical image data and the second medical image data from the multiple levels of sub-rules, and determining the first association relationship between the first medical image data and the second medical image data according to the target sub-rule.

[0027] The sub-rules can be understood as specific, single, and executable matching conditions or judgment logic that constitute the association matching rules. They are the smallest execution unit of association matching. Each sub-rule corresponds to a specific image association determination method, used to determine whether two images meet a certain type of association condition. The target sub-rule can be understood as a specific sub-rule ultimately selected from multiple levels of sub-rules to determine the first association relationship in this instance. It serves as the actual effective rule for this matching, clearly specifying which rule the two images are determined to be associated, ensuring the uniqueness and certainty of the relationship determination.

[0028] By adopting this technical solution and setting up multi-level association sub-rules, it is possible to match the target sub-rules of the corresponding level for different second medical image data, accurately determine the association relationship, improve the flexibility and adaptability of association judgment, and enable medical images of different types, modalities and examination purposes to establish associations at reasonable levels, ensuring that the association relationship is accurate and reliable, and meeting the fine matching needs in complex medical imaging scenarios.

[0029] S120. Construct an image group based on the first medical image data and multiple second medical image data, and determine the image association identifier corresponding to each second medical image data based on the first association relationship.

[0030] The image group can be understood as an image collection composed of first medical image data and multiple related second medical image data. It is used to organize scattered related images into a logical whole, facilitating unified management and display. The image association identifier can be understood as a label, number, field, or marker used to mark, distinguish, and identify the relationship of each image within the group. It can intuitively tell the user what the image is used for, or distinguish images with different relationships on the interface through color, border, or corner mark.

[0031] S130. Based on the image association identifier, the first medical image data and multiple second medical image data in the image group are associated and displayed.

[0032] The aforementioned related display can be understood as presenting the first medical image data and multiple second medical image data in an organized manner according to their relationship on the display interface. It is not a simple listing, but a logical side-by-side, superimposed, or linked display to improve diagnostic efficiency.

[0033] In this embodiment, during the display phase, the associated image group corresponding to the first medical image data can be loaded to achieve multi-dimensional synchronous display: (1) Image display area: It adopts a multi-window display with customizable layout, supports on-screen comparison, linked scaling and synchronous roaming. High-priority associated images are displayed in the center by default, and the window size and position can be adjusted by dragging. (2) Association Logic Display Area: The image association relationship is presented in the form of a visual knowledge graph. The nodes are image thumbnails, and the edges are the numerical values ​​of the matching degree between the image association identifier (distinguished by different colors) and the target. Clicking on the node allows you to view the complete symptom information, and clicking on the edge allows you to view the details of the image association identifier. (3) Diagnostic information display area: centrally displays the diagnostic opinions, annotations, and examination report summaries of all related images, and supports sorting by association priority and examination time sequence; (4) The display interface supports mode switching: standard mode (full display), focus mode (only display high priority associations), and time chain mode (display by examination time axis) to adapt to different diagnostic scenario requirements.

[0034] Based on the above scheme, optionally, the image association identifier includes at least association priority, association type, and examination time; the step of displaying the first medical image data and multiple second medical image data in the image group in association according to the image association identifier includes at least one of the following: classifying each second medical image data according to the association priority based on the image association identifier, and displaying the second medical image data with the highest association priority first; dividing multiple second medical image data under the same association priority into a target number of display groups according to the association type, and displaying the second medical image data in the display groups in association according to a preset rule, wherein the association type includes direct pathology and indirect complications; sorting multiple second medical image data according to the examination time, and displaying them in association according to the sorted results.

[0035] The association priority can be understood as the importance level of the association between the second medical image data and the first medical image data, used to determine the display order. The higher the priority, the earlier it is displayed, helping doctors to focus on key information first. The association type can be understood as a classification description of the relationship between the first and second medical images, used to group images according to their business significance, facilitating doctors to view them by pathology, complications, etc. The examination time can be understood as the time information corresponding to the acquisition / examination of medical image data, used to sort images by time, enabling the display of disease progression and follow-up comparison. The target quantity can be understood as a preset upper limit or fixed value of the number of images expected to be included in each display group, used to control the complexity of the interface and the aesthetics of the layout, preventing too many images in a group from causing screen congestion, or too few images from causing wasted space. If the target quantity is exceeded, redundant images may be automatically hidden or scroll bars may be enabled. The display group can be understood as a subset of images further divided according to association type under the same association priority, realizing a two-dimensional grouping method of first by association priority and then by relationship type, enhancing the clarity of the display. The preset rules can be understood as a set of rules used to control how images are arranged and displayed within a group, determining the layout, sorting, or display style within the group. Direct pathology can be understood as a type of association, indicating that the second medical image and the first medical image reflect the same primary disease or lesion, suggesting to doctors that these images are core diagnostic evidence and should be compared closely. Indirect complications can be understood as a type of association, indicating that the second medical image reflects lesions in other organs or systems caused by the primary disease.

[0036] This technical solution uses multi-dimensional identifiers such as association priority, association type, and examination time to classify, group, and sort images for display. It can prioritize the presentation of images with the highest relevance and distinguish them by type such as direct pathology and indirect complications, making the relationships between medical image data clear and easy to understand. At the same time, it can sort out lesion changes over time, greatly improving the efficiency and accuracy of diagnosis and reducing the risk of information confusion.

[0037] Based on the above scheme, optionally, the step of associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier includes: constructing a knowledge graph with the thumbnail of the first medical image data and multiple second medical image data as nodes and the image association identifier as edges, and displaying the knowledge graph.

[0038] The thumbnails can be understood as small-sized preview images generated after compressing or cropping the original medical image data. They serve as a visual representation of nodes in the knowledge graph, saving space while retaining basic visual information. The nodes can be understood as the basic building blocks of the knowledge graph, used for each medical image data, carrying the image itself and its metadata, and are clickable, viewable, and interactive objects within the graph. The knowledge graph can be understood as a structured knowledge network composed of nodes and edges, used to express entities and the relationships between entities, visually displaying scattered medical images in the form of a relational network, achieving an intuitive presentation and rapid retrieval of the relationships between images.

[0039] In one optional implementation, the first medical image data and multiple second medical image data can be displayed in different colors in the knowledge graph, with different colors representing different relationships.

[0040] This technical solution constructs a knowledge graph with thumbnails as nodes and association identifiers as edges, which intuitively presents the complex topological relationships between medical image data, supports the visual tracing of lesion evolution paths, and improves doctors' cognitive efficiency of the overall correlation of multimodal data.

[0041] Based on the above scheme, optionally, the association matching rule includes at least one of the following: the second medical image data and the first medical image data include image information of the same part of the target object, and the second medical image data and the first medical image data are acquired based on different types of medical imaging devices; the lesion sites of multiple medical image data have indirect complications.

[0042] The term "same location" can be understood as the area of ​​interest in the first and second medical images being completely identical or highly overlapping, used to define the spatial consistency of the associated images and ensure the comparison has clinical significance. The image information can be understood as the visual content and related metadata contained in the medical image, such as imaging range, anatomical landmarks, and lesion annotations, which is the information basis for determining "whether it is the same location." The different types of medical imaging devices that acquire medical images using different physical principles or imaging technologies are used to distinguish the imaging modalities of the images and are a key basis for determining multimodal association.

[0043] By adopting this technical solution, and by setting association and matching rules such as imaging of the same site by different devices and indirect complications of lesions, it can accurately cover various image association scenarios such as multimodal and pathological associations, realize intelligent matching and association of complex medical images, improve the comprehensiveness and rationality of association results, and provide a reliable basis for subsequent structured organization and linkage display.

[0044] Based on the above scheme, optionally, after associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier, the method further includes: in response to the annotation operation for the first medical image data, adding first annotation information to the first medical image data, and adding second annotation information and source information of the second annotation information to at least a portion of the second medical images in the image group according to the first annotation information; in response to the annotation operation for the third medical image data, adding first annotation information to the third medical image data, and adding second annotation information and source information of the second annotation information to at least a portion of the fourth medical image data according to the first annotation information, wherein the third medical image data is the first medical image data or the second medical image data, and the fourth medical image data includes medical image data in the image group other than the third medical image data.

[0045] The annotation operation can be understood as an interactive behavior performed by the user on a medical image to mark regions of interest or diagnostic conclusions, and is the source that triggers the annotation generation and synchronization mechanism. The first annotation information, the original annotation content directly added by the user on the triggering image (first or third medical image), is the original data of the entire annotation synchronization mechanism, including but not limited to diverse information such as geometric and semantic information. The second annotation information can be understood as the corresponding annotation content generated on the target image (second or fourth medical image) based on the first annotation information, achieving consistent annotation expression across images and avoiding repeated manual annotation by doctors. The source information can be understood as traceability information used to mark the source of the second annotation information, clarifying the image data, annotator, annotation time, and related basis to which the original first annotation information belongs, enabling traceability and verification of annotation information, clearly distinguishing between manual annotation and linked derived annotation, facilitating subsequent diagnostic review, image verification, and information traceability, and ensuring the standardization and credibility of the annotation information. The third medical image data can be understood as any image in which the user performs annotation operations. It can be the first medical image or any second medical image, thus generalizing the annotation trigger source and no longer limiting it to the first medical image, supporting annotation synchronization to be initiated from any node within the image group. The fourth medical image can be understood as the remaining medical image data in the image group other than the third medical image after the third medical image data has been annotated, defining the scope of the new round of annotation synchronization.

[0046] By adopting this technical solution, when annotating any image after association display, the annotation information can be synchronized to other related images in the image group, and the source information can be carried, avoiding repeated annotation operations and greatly improving the efficiency of medical image annotation; at the same time, it ensures the consistency and traceability of annotation of the same lesion and the same disease in multimodal and associated images, reduces omissions and mislabeling, and improves the accuracy and reliability of diagnosis and subsequent analysis.

[0047] Based on the above scheme, optionally, after associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier, the method further includes: updating the second annotation information according to the changed first annotation information in response to a change in the first annotation information; or, updating the first annotation information according to the changed second annotation information in response to a change in the second annotation information.

[0048] The change in the first annotation information can be understood as any modification, supplementation, or deletion of the first annotation information already added to the first / third medical image data. This includes changes to any content such as annotation parameters, lesion descriptions, and diagnostic conclusions. This is one of the core triggering conditions for the linked update process. When the first annotation information changes, the second annotation information on the associated image is updated synchronously to ensure the synchronization and accuracy of the annotation information and avoid problems such as changes to the original annotation and delays in derived annotations. The second annotation information can be understood as any modification, supplementation, or deletion of the second annotation information already added to the associated image (second / fourth medical image data). The changes can be related to the first annotation information or adjusted for the specific features of the associated image. This is a core triggering condition for the reverse linked update process. When the second annotation information changes, the corresponding first annotation information is updated synchronously, realizing bidirectional linkage of annotation information and adapting to the doctor's need to reverse-correct the baseline annotation after annotating the associated image.

[0049] An optional implementation method, (1) Operation synchronization: When a doctor annotates a certain medical image data (such as lesion delineation, measurement), enters diagnostic opinions, or modifies diagnostic conclusions, the system automatically synchronizes to the corresponding position of all related images in the same group and marks the synchronization source (such as "synchronized from CT plain scan image annotation"), supports synchronization cancellation and selective synchronization (the doctor can select the related images to be synchronized). (2) Information linkage: When the symptom information of the associated image is updated (such as the addition of new functional image symptom), the target matching degree is automatically recalculated, the image association identifier and the associated knowledge graph are updated, and an update reminder is pushed to the doctor; (3) Data archiving: After the diagnosis is completed, the associated image group (including all medical image data, image association identifiers, visualized knowledge graphs, and diagnostic information) is archived in a unified manner according to the naming rule of "Patient ID - Examination Date - Association Group ID". It supports multi-dimensional retrieval (such as association type, disease type, association rule ID). The archived data is stored in encrypted form to ensure the security of medical data.

[0050] By adopting this technical solution, when any annotation information changes, the corresponding annotation in the associated image can be automatically and synchronously updated, realizing the linkage modification and real-time synchronization of annotations within the group. This effectively avoids problems such as inconsistent annotations and repeated modifications, improves the coherence, accuracy and modification efficiency of medical image annotations, and ensures the uniformity and reliability of diagnostic information.

[0051] The technical solution of this invention acquires first medical image data of a target object, determines multiple second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each of the second medical image data according to a preset association matching rule, thereby matching and determining the association relationship of medical images and improving image association efficiency. Next, an image group is constructed based on the first medical image data and the multiple second medical image data, and an image association identifier corresponding to each of the second medical image data is determined according to the first association relationship. By constructing the image group and generating corresponding association identifiers, structured organization and precise differentiation of medical images can be achieved. Finally, the first medical image data and the multiple second medical image data in the image group are displayed in association according to the image association identifiers. This allows for the linked display of multiple medical images, solving the problems of fragmented source medical image association display and difficulty in intuitively grasping lesion evolution. It achieves automatic association matching, structured grouping, and precise identifier-linked display of medical images, significantly improving diagnostic review efficiency and image utilization.

[0052] Example 2 Figure 2This is a flowchart of a medical image association annotation method provided in Embodiment 2 of the present invention. This embodiment is a further refinement of the above embodiments, which further details the determination of image association identifiers corresponding to each second medical image data based on the first association relationship. Optionally, determining the image association identifiers corresponding to each second medical image data based on the first association relationship includes: determining the target matching degree corresponding to the image group based on the first medical image data, multiple second medical image data, and the association matching rule; establishing a first association identifier for the first medical image data and multiple second medical image data based on the first association relationship when the target matching degree is greater than a preset confidence threshold; determining the first sign information of the first medical image data, wherein the first sign information includes the examination time and object identifier of the target object; determining the second sign information corresponding to the first sign information based on historical medical image data; determining a second association relationship based on the first sign information and the second sign information; and determining the image association identifier based on the first association relationship and the second association relationship. For detailed implementation, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here.

[0053] like Figure 2 As shown, the method may specifically include: S210. Obtain first medical image data of the target object, and determine multiple second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each second medical image data according to a preset association matching rule.

[0054] S220. Construct an image group based on the first medical image data and multiple second medical image data, and determine the target matching degree corresponding to the image group based on the first medical image data, multiple second medical image data and the association matching rule.

[0055] The target matching degree can be understood as a quantitative score of the overall conformity between the first medical image, a certain second medical image, and the associated matching rules. It is used to objectively measure the reliability of the association and is a key indicator for deciding whether to generate a formal association label for the image.

[0056] Based on the above scheme, optionally, determining the target matching degree corresponding to the image group according to the first medical image data, multiple second medical image data, and the association matching rule includes: determining the single matching degree between each second medical image data and the first medical image data according to the association matching rule; determining the weight coefficient corresponding to each second medical image data according to the severity of the lesions in each second medical image data; determining the initial matching degree according to the single matching degree and the weight coefficient corresponding to each second medical image data; and determining the target matching degree corresponding to the image group according to the initial matching degree of the multiple second medical image data.

[0057] The single matching degree can be understood as the matching score between a second medical image and a first medical image, calculated based on association matching rules. It represents the similarity at the "rule level" and serves as the base score for subsequent weighted calculations. The severity of the lesion can be understood as the assessment result of the severity of the lesion in the medical image in terms of pathological progression and clinical risk. It is used to determine the weight of the image in the overall matching degree calculation, with severe lesions receiving higher influence. The weight coefficient can be understood as the numerical weight assigned to each second medical image based on the severity of the lesion, used to adjust its influence on the overall matching degree. The initial matching degree can be understood as the comprehensive matching score of a single image obtained by combining its single matching degree with the weight coefficient. It is the result of "clinical importance correction" for the single matching degree and is used for subsequent aggregation.

[0058] In this embodiment, the symptom information of the current image can be fully matched with the association matching rules to calculate the target matching degree (target matching degree = Σ (single matching degree × weight coefficient)). When the target matching degree is greater than or equal to the preset confidence threshold, all images of the associated modalities / diseases are automatically identified, and a first association identifier is added to the first medical image data and the second medical image data. The identifier includes the association type (direct pathology / indirect complication), association rule ID, matching degree value, association priority (based on a comprehensive ranking of matching degree and clinical importance), etc.

[0059] By employing this technical solution, the target matching degree of the image group is obtained by calculating the individual matching degree separately and setting weight coefficients in combination with the severity of the lesions. This allows for a more scientific and accurate quantification of the correlation between medical images, taking into account both the matching relationship of the images themselves and the actual condition of the lesions. This makes the image group screening, sorting, and display more in line with clinical diagnostic logic, thereby improving the reliability of the correlation results and the diagnostic reference value.

[0060] S230. When the target matching degree is greater than the preset confidence threshold, a first association identifier is established between the first medical image data and multiple second medical image data according to the first association relationship.

[0061] The preset reliability threshold can be understood as a pre-set minimum score threshold used to determine whether the target matching degree is reliable enough to establish an association identifier. As a decision switch, a formal association identifier is established only when the target matching degree is greater than this threshold, ensuring the reliability of the association. The first association identifier can be understood as a preliminary association identifier established based on the first association relationship after the target matching degree passes the threshold test. It is used to record the direct association information between the current images and is an important component of the image association identifier.

[0062] S240. Determine the first sign information of the first medical image data, wherein the first sign information includes the examination time and object identifier of the target object.

[0063] The first sign information can be understood as information extracted from the first medical image data that characterizes the basic examination attributes of the target object, used to match it with historical data to determine the correlation across time and examinations. The object identifier can be understood as a number, ID, or feature information used to uniquely identify the target object, used to uniquely distinguish different objects, and to ensure that historical images and current images belong to the same object.

[0064] In this embodiment, the image can be analyzed by a multi-dimensional image feature extraction unit to automatically identify and extract four-dimensional key feature information from each medical image data, including: spatial dimension: 3D coordinates of the lesion, anatomical location, and spatial relationship with surrounding tissues; morphological dimension: lesion size, shape, edge features, and internal density / signal uniformity; functional dimension: dynamic enhancement curve parameters, metabolic activity values ​​(applicable to functional imaging such as PET-CT), and hemodynamic parameters; clinical dimension: patient basic information, examination time series, and past medical history-related labels (such as the influence label of diabetes history on vascular lesion images). The extraction process can employ a multi-model fusion strategy: using a CNN model for morphological feature extraction, a Transformer model for sequence feature (time dimension) extraction, and a Graph CNN model for spatial relationship feature extraction. The extraction results are output through a weighted fusion algorithm to ensure the completeness and accuracy of the feature information.

[0065] S250. Determine the second sign information corresponding to the first sign information based on historical medical image data, and determine the second association relationship based on the first sign information and the second sign information.

[0066] The historical medical image data can be understood as a set of medical image data generated in the past that may be related to the first medical image. This data is used to compare with the current first medical image data, extract historical symptom information, and establish historical associations. The second symptom information can be understood as historical attribute information extracted from the historical medical image data that corresponds to the first symptom information. This information is used to compare with the first symptom information to determine the association between the historical and current medical image data. The second association can be understood as the association between the current and historical images obtained by comparing the first and second symptom information. This association is used to supplement historical follow-up associations, making the image association identification more complete and comprehensive.

[0067] In this embodiment, based on the clinical dimension signs (such as examination time and patient ID) of the first medical image data, the historical medical image data of the same patient is traced, the four-dimensional sign information of the historical medical image data is extracted, and reverse matching is performed with the first medical image data to supplement the missing association relationships in order to determine the image association identifier.

[0068] In one alternative implementation, doctors, professionals, and others can adjust the priority of image association labels based on clinical experience and add custom associations (with manual intervention labels and reasons). Manually added associations are automatically included in the candidate rule pool of the mapping library and can be updated to the official rule library after big data verification.

[0069] S260. Determine the image association identifier based on the first association relationship and the second association relationship.

[0070] S270. Based on the image association identifier, the first medical image data and multiple second medical image data in the image group are associated and displayed.

[0071] The technical solution of this invention generates a first association identifier by combining the target matching degree and the confidence threshold, and establishes a second association relationship by introducing symptom information such as examination time and object identifier as well as historical medical image data. This can accurately determine the image association identifier from multiple dimensions and levels, improve the reliability and distinguishability of the identifier, make the image association more in line with clinical practice, and provide solid support for subsequent efficient and accurate association display.

[0072] Example 3 Figure 3 This is a schematic diagram of the structure of a medical image association and annotation device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes: a first association determination module 310, an association identifier determination module 320, and a display module 330. Among them, The first association determination module 310 is used to acquire the first medical image data of the target object, determine multiple second medical image data associated with the first medical image data and the first association relationship between the first medical image data and each second medical image data according to a preset association matching rule; the association identifier determination module 320 is used to construct an image group based on the first medical image data and multiple second medical image data, and determine the image association identifier corresponding to each second medical image data according to the first association relationship; the display module 330 is used to display the first medical image data and multiple second medical image data in the image group according to the image association identifier.

[0073] The technical solution of this invention involves obtaining first medical image data of a target object through a first association relationship determination module. Based on preset association matching rules, multiple second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each of the second medical image data are determined. This allows for matching and determining association relationships between medical images, improving image association efficiency. Next, an association identifier determination module constructs an image group based on the first medical image data and the multiple second medical image data. Furthermore, based on the first association relationship, an image association identifier corresponding to each of the second medical image data is determined. By constructing the image group and generating corresponding association identifiers, structured organization and precise differentiation of medical images are achieved. Finally, a display module displays the first medical image data and the multiple second medical image data in the image group based on the image association identifiers. This enables multi-medical image linkage display based on image association identifiers, solving the problems of fragmented source medical image association display and difficulty in intuitively grasping lesion evolution. It achieves automatic association matching, structured grouping, and precise identifier linkage display of medical images, significantly improving diagnostic review efficiency and image utilization.

[0074] Optionally, the association matching rule includes multiple levels of sub-rules; the first association relationship determination module includes: a first association relationship determination sub-module. The first association relationship determination sub-module is used to determine, for each piece of the second medical image data, a target sub-rule corresponding to the first medical image data and the second medical image data from the multiple levels of sub-rules, and determine a first association relationship between the first medical image data and the second medical image data based on the target sub-rule.

[0075] Optionally, the association identifier determination module includes: a target matching degree determination submodule, a first association identifier determination submodule, a first symptom information determination submodule, a second association relationship determination submodule, and an association identifier determination submodule. Specifically, the target matching degree determination submodule is used to determine the target matching degree corresponding to the image group based on the first medical image data, multiple second medical image data, and the association matching rule; the first association identifier determination submodule is used to establish a first association identifier between the first medical image data and multiple second medical image data based on the first association relationship when the target matching degree is greater than a preset confidence threshold; the first symptom information determination submodule is used to determine the first symptom information of the first medical image data, wherein the first symptom information includes the examination time and object identifier of the target object; the second association relationship determination submodule is used to determine the second symptom information corresponding to the first symptom information based on historical medical image data, and to determine a second association relationship based on the first symptom information and the second symptom information; and the association identifier determination submodule is used to determine an image association identifier based on the first association relationship and the second association relationship.

[0076] Optionally, the target matching degree determination submodule is specifically used to determine the single matching degree between each second medical image data and the first medical image data according to the association matching rule; determine the weight coefficient corresponding to each second medical image data according to the severity of the lesions in each second medical image data; determine the initial matching degree according to the single matching degree and the weight coefficient corresponding to each second medical image data; and determine the target matching degree corresponding to the image group according to the initial matching degree of multiple second medical image data.

[0077] Optionally, the image association identifier includes at least association priority, association type, and examination time; the display module is specifically used to classify each second medical image data according to the association priority based on the image association identifier, and prioritize the display of the second medical image data with the highest association priority; to divide multiple second medical image data under the same association priority into a target number of display groups according to the association type, and to associate and display the second medical image data in the display groups according to preset rules, wherein the association type includes direct pathology and indirect complications; and to sort multiple second medical image data according to the examination time, and to associate and display them according to the sorted results.

[0078] Optionally, the display module is further configured to construct a knowledge graph using the thumbnail of the first medical image data and multiple pieces of the second medical image data as nodes and the image association identifier as edges, and to display the knowledge graph.

[0079] Optionally, the association matching rule includes at least one of the following: the second medical image data and the first medical image data include image information of the same part of the target object, and the second medical image data and the first medical image data are acquired based on different types of medical imaging devices; the lesion sites of multiple medical image data have indirect complications.

[0080] Optionally, the medical image association and annotation device further includes: an annotation module. The annotation module is configured to, in response to an annotation operation on the first medical image data, add first annotation information to the first medical image data after the first medical image data and multiple second medical image data in the image group are associated and displayed according to the image association identifier, and add second annotation information and source information of the second annotation information to at least a portion of the second medical images in the image group according to the first annotation information; the second annotation module is configured to, in response to an annotation operation on the third medical image data, add first annotation information to the third medical image data, and add second annotation information and source information of the second annotation information to at least a portion of the fourth medical image data according to the first annotation information, wherein the third medical image data is either the first medical image data or the second medical image data, and the fourth medical image data includes medical image data in the image group other than the third medical image data.

[0081] Optionally, the medical image association and annotation device further includes an update module. The update module is configured to, after the first medical image data and multiple second medical image data in the image group are associated and displayed according to the image association identifier, update the second annotation information according to the changed first annotation information in response to a change in the first annotation information; or, update the first annotation information according to the changed second annotation information in response to a change in the second annotation information.

[0082] The medical image association annotation device provided in the embodiments of the present invention can execute the medical image association annotation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0083] Example 4 Figure 4A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0084] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0085] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0086] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a medical image association annotation method.

[0087] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0088] In some embodiments, a medical image association annotation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the medical image association annotation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a medical image association annotation method by any other suitable means (e.g., by means of firmware).

[0089] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0090] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0091] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0092] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0093] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0094] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for associative annotation of medical images, characterized in that, include: Acquire first medical image data of the target object, and determine multiple second medical image data associated with the first medical image data and a first association relationship between the first medical image data and each second medical image data according to a preset association matching rule; An image group is constructed based on the first medical image data and multiple second medical image data, and an image association identifier corresponding to each second medical image data is determined based on the first association relationship. The first medical image data and multiple second medical image data in the image group are associated and displayed according to the image association identifier.

2. The method according to claim 1, characterized in that, The association matching rules include multiple levels of sub-rules; Determining a first association relationship between the first medical image data and multiple second medical image data according to a preset association matching rule includes: For each second medical image data, a target sub-rule corresponding to the first medical image data and the second medical image data is determined from the sub-rules at multiple levels, and a first association relationship between the first medical image data and the second medical image data is determined according to the target sub-rule.

3. The method according to claim 1, characterized in that, The step of determining the image association identifier corresponding to each of the second medical image data according to the first association relationship includes: The target matching degree corresponding to the image group is determined based on the first medical image data, multiple second medical image data, and the association matching rule; When the target matching degree is greater than a preset confidence threshold, a first association identifier is established between the first medical image data and multiple second medical image data based on the first association relationship; Determine the first sign information of the first medical image data, wherein the first sign information includes the examination time and object identifier of the target object; Based on historical medical image data, determine the second sign information corresponding to the first sign information, and determine the second correlation relationship based on the first sign information and the second sign information; The image association identifier is determined based on the first association relationship and the second association relationship.

4. The method according to claim 3, characterized in that, The step of determining the target matching degree corresponding to the image group based on the first medical image data, multiple second medical image data, and the association matching rule includes: The degree of single match between each second medical image data and the first medical image data is determined according to the association matching rules; The weighting coefficient for each second medical image data is determined based on the severity of the lesion in each second medical image data. An initial matching degree is determined based on the single matching degree and the weight coefficient corresponding to each of the second medical image data, and a target matching degree is determined based on the initial matching degrees of multiple second medical image data.

5. The method according to claim 1, characterized in that, The image association identifier includes at least association priority, association type, and examination time; the step of associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier includes at least one of the following: Each second medical image data is classified according to the association priority based on the image association identifier, and the second medical image data with the highest association priority is displayed first. Multiple second medical image data under the same association priority are divided into a target number of display groups according to the association type. The second medical image data in the display group are displayed in association according to a preset rule. The association type includes direct pathology and indirect complications. Multiple second medical image data are sorted according to the examination time, and the sorted results are displayed in association.

6. The method according to claim 1, characterized in that, The step of associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier includes: A knowledge graph is constructed using the thumbnail of the first medical image data and multiple second medical image data as nodes, and the image association identifier as edges, and the knowledge graph is displayed.

7. The method according to claim 1, characterized in that, The association matching rules include at least one of the following: The second medical image data and the first medical image data include image information of the same part of the target object, and the second medical image data and the first medical image data are acquired based on different types of medical imaging devices; Indirect complications exist at multiple lesion sites in the aforementioned medical image data.

8. The method according to claim 1, characterized in that, After associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier, the method further includes: In response to the annotation operation on the first medical image data, first annotation information is added to the first medical image data, and second annotation information and source information of the second annotation information are added to at least a portion of the second medical images in the image group according to the first annotation information; In response to the annotation operation on the third medical image data, first annotation information is added to the third medical image data, and second annotation information and source information of the second annotation information are added to at least a portion of the fourth medical image data based on the first annotation information, wherein the third medical image data is either the first medical image data or the second medical image data, and the fourth medical image data includes medical image data in the image group other than the third medical image data.

9. The method according to claim 8, characterized in that, After associating and displaying the first medical image data and multiple second medical image data in the image group according to the image association identifier, the method further includes: In response to a change in the first annotation information, the second annotation information is updated based on the changed first annotation information; or, In response to a change in the second annotation information, the first annotation information is updated based on the changed second annotation information.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the medical image association annotation method according to any one of claims 1-9.

Images