A medical image key point association detection method and system

Abstract

The present application discloses a method and system for detecting key point association in medical images, which relates to medical image processing technology, including: selecting at least a first medical image from a set of images to be detected; configuring reference points based on the first medical image; extracting reference boundary data based on pre-configured areas of interest and a unified coordinate system; performing local matching on the areas of interest in the first medical image based on the extracted reference boundary data to determine continuous key points in the direction of the corresponding lines; using the determined continuous key points as a reference, performing association detection on the second medical image based on a unified coordinate system; determining augmented key points from the second medical image based on the results of the key point association detection to complete the key point association detection. The method of the present application can improve the accuracy of key point detection and positioning, and is applicable to key point detection of medical images of various types of parts or organs.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method and system for detecting key point associations in medical images. Background Art

[0002] Key point extraction of medical images is an important computer vision technology that can be used to analyze key structures and features in medical images, such as tumors, bones, hearts, etc. Key point extraction usually involves finding some landmark points or regions in the image, which can serve as the basis for subsequent analysis.

[0003] In the prior art, a direct 3D key point extraction model is used for hospital images. However, the 3D key points output by the model extraction method in the prior art have large errors and the extraction accuracy is inaccurate. Summary of the invention

[0004] The embodiments of the present application provide a method and system for detecting the association of key points in medical images. By combining image processing and model recognition, as well as the association detection of key points, the accuracy of key point detection and positioning is improved, and the method is applicable to the key point detection of medical images of various types of parts or organs.

[0005] The present application embodiment provides a method for detecting key point association in medical images, including:

[0006] Acquire multiple medical images as an image set to be detected;

[0007] Selecting at least a first medical image and a second medical image from the set of images to be detected;

[0008] configuring reference points based on the first medical image and the second medical image, wherein when there is an overlapping area between the first medical image and the second medical image, configuring reference points for the first medical image and the second medical image based on the overlapping area, and when there is no overlapping area between the first medical image and the second medical image, matching a key point from the first medical image based on at least a partial area of ​​a typical part as a reference point, and reselecting a second medical image from the set of images to be detected so that it has an overlapping area with the first medical image;

[0009] Based on the reference points, a unified coordinate system is established, and a region of interest is determined in the first medical image according to the unified coordinate system;

[0010] Extracting reference boundary data according to the preconfigured region of interest and the unified coordinate system;

[0011] Based on the extracted reference boundary data, locally matching the region of interest in the first medical image to determine continuous key points in the direction of the corresponding line;

[0012] Using the determined continuous key points as a reference, performing association detection on the second medical image based on the unified coordinate system;

[0013] Determining the amplified key points from the second medical image according to the result of the key point association detection;

[0014] Repeating the key point amplification of the medical images in the image set to be detected based on the determined key points to complete the key point association detection.

[0015] Optionally, configuring reference points based on the first medical image and the second medical image includes:

[0016] Extracting line data from the first medical image and the second medical image, wherein the line data includes boundary data and line information inside the boundary, and the line information is connected to the boundary data or self-closed;

[0017] The extracted line data are superimposed and compared to determine whether there is a partially overlapping area;

[0018] When there is an overlapping area in the superposition comparison, a local position with high complexity of line data is selected from the overlapping area as a reference point;

[0019] When there is no overlapping area in the superposition comparison, the line data of the first medical image is matched with the contour of the typical part to determine the reference point from the matched partial lines.

[0020] Optionally, establishing a unified coordinate system based on the reference point, and determining a region of interest in the first medical image according to the unified coordinate system includes:

[0021] Based on the reference points, a unified coordinate system is established; and,

[0022] Segmenting the first medical image based on the coordinate system;

[0023] Extract the image features of each sub-segmentation and input them into the pre-trained recognition model to determine the probability that any sub-segmentation belongs to the region of interest;

[0024] A plurality of sub-divided connected regions whose probability values ​​belonging to the region of interest are greater than a preset probability threshold are determined as the region of interest.

[0025] Optionally, according to the preconfigured part of interest and the unified coordinate system, extracting reference boundary data includes:

[0026] Determine the position range of the pre-configured part of interest in the unified coordinate system;

[0027] According to the determined orientation, a plurality of boundary data of the preconfigured region of interest is extracted within the orientation range as reference boundary data.

[0028] Optionally, based on the extracted reference boundary data, locally matching the region of interest in the first medical image to determine continuous key points in the direction of the corresponding line includes:

[0029] Extracting line information in the connected areas of the plurality of sub-segments using a Canny algorithm, determining the continuous line length of each line information, and extracting a plurality of lines with the longest continuous line length;

[0030] For any extracted continuous line, according to the set step length, in any boundary data, sliding calculation is performed on the similarity with the any boundary data;

[0031] When the similarity is higher than a preset similarity threshold, continuous key points are determined from any continuous lines.

[0032] Optionally, determining continuous key points from any of the continuous lines includes:

[0033] Determine the sub-boundary data with the highest similarity to any of the continuous lines;

[0034] Determining key point information pre-configured in the sub-boundary data;

[0035] The key point information is mapped onto any of the continuous lines to determine continuous key points on any of the continuous lines.

[0036] Optionally, determining continuous key points from any of the continuous lines further includes:

[0037] For a portion of the lines within any of the continuous lines, selecting reference points from the determined continuous key points;

[0038] According to the reference key points pre-configured based on the typical parts, searching for corresponding key points within the pixel distance range between the reference points;

[0039] Determining the degree of match between a local area centered on the found key point in the first medical image and a local area corresponding to the reference key point;

[0040] When the matching degree is lower than the preset threshold, the key points found are corrected on the part of the lines until the matching degree is higher than the preset threshold.

[0041] Optionally, according to a result of key point association detection, determining the amplified key points from the second medical image includes:

[0042] superimposing the second medical image and the first medical image based on the overlapping area; and

[0043] Determine key points already included in the first medical image, and determine local areas that exist only in the second medical image based on pre-configured regions of interest;

[0044] Based on the local area that only exists in the second medical image, the extension direction of the line that already contains the key points is determined, and the key point search is performed in the extension direction to determine the enlarged key points.

[0045] An embodiment of the present application further proposes a medical image key point association detection system, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the medical image key point association detection method as described above are implemented.

[0046] The method of the embodiment of the present application combines image processing and model recognition, as well as the association detection of key points, to improve the accuracy of key point detection and positioning, and is applicable to key point detection in medical images of various types of parts or organs.

[0047] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0049] Figure 1 The following is a schematic diagram of the basic process of the medical image key point association detection method of this embodiment. DETAILED DESCRIPTION

[0050] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0051] The present application embodiment proposes a method for detecting key point association in medical images, such as Figure 1 As shown, the following steps are included:

[0052] In step S101, a plurality of medical images are obtained as a set of images to be detected. In some examples, the plurality of medical images may be of the same modality or different modalities, and may be medical images taken at different angles.

[0053] In step S102, at least a first medical image and a second medical image are selected from the image set to be detected. For example, two medical images can be randomly selected as the first medical image and the second medical image, or the medical images in the image set to be detected can be sorted according to their clarity, with images with high clarity being preferentially selected.

[0054] In step S103, reference points are configured based on the first medical image and the second medical image, wherein when there is an overlapping area between the first medical image and the second medical image, reference points are configured for the first medical image and the second medical image based on the overlapping area, and when there is no overlapping area between the first medical image and the second medical image, a key point is matched from the first medical image based on at least a partial area of ​​a typical part as a reference point, and a second medical image is reselected from the set of images to be detected so that it has an overlapping area with the first medical image. In some embodiments, for example, if the part of interest in the present application is a medical image of a person's palm, the corresponding typical part may also be a typical medical image of the palm, and the medical image of the typical part may be pre-stored in a local database for direct call.

[0055] In step S104, a unified coordinate system is established based on the reference point, and a region of interest is determined in the first medical image according to the unified coordinate system. In some embodiments, the region of interest may be a partial or entire image region of a site of interest, such as a heart, and the reference point may be outside or inside the region of interest.

[0056] In step S105, reference boundary data is extracted according to the pre-configured part of interest and the unified coordinate system. The pre-configured part of interest referred to in this application may be a typical part, such as a typical heart 3D model, or a typical CT image of a hand, brain, etc., which may be pre-stored in a local database for calling as a reference.

[0057] In step S106, based on the extracted reference boundary data, the region of interest in the first medical image is locally matched to determine continuous key points in the direction of the corresponding line. In some examples, the reference boundary of the corresponding region of interest can be extracted based on a local database, thereby serving as a reference for the first medical image of the key points to be detected, to extract continuous key points in the direction of the corresponding line.

[0058] In step S107, the determined continuous key points are used as references to perform association detection on the second medical image based on the unified coordinate system.

[0059] In step S108, based on the result of key point association detection, the enlarged key points are determined from the second medical image.

[0060] In step S109, key point amplification is repeatedly performed on the medical images in the image set to be detected based on the determined key points to complete key point association detection.

[0061] The method of the embodiment of the present application performs association detection of key points through image processing, and in subsequent embodiments, further combines model recognition. The method of the present application can improve the accuracy of key point detection and positioning, and is suitable for key point detection in medical images of various types of parts or organs.

[0062] In some embodiments, configuring reference points based on the first medical image and the second medical image includes:

[0063] Extracting line data from the first medical image and the second medical image, wherein the line data includes boundary data and line information inside the boundary, and the line information is connected to or self-closed with the boundary data. In a specific example, the image can be roughly processed into a binary image, and the line data from the first medical image and the second medical image can be determined.

[0064] The extracted line data are superimposed and compared to determine whether there is a partially overlapping area.

[0065] When there is an overlapping area in the superposition comparison, a local position with high complexity of line data is selected from the overlapping area as a reference point. By selecting the reference point, it is convenient to establish a coordinate system, and further accurately expand more key points from the second medical image, thereby improving the efficiency of key point detection.

[0066] When there is no overlapping area in the superposition comparison, the line data of the first medical image is matched with the contour of the typical part to determine the reference point from the matched partial lines.

[0067] In some embodiments, establishing a unified coordinate system based on the reference point, and determining a region of interest in the first medical image according to the unified coordinate system includes:

[0068] Based on the reference points, a unified coordinate system is established.

[0069] The first medical image is segmented based on the coordinate system.

[0070] The image features of each sub-segment are extracted and input into a pre-trained recognition model to determine the probability that any sub-segment belongs to the region of interest. In some examples, the recognition model can be a CNN model to achieve local image classification.

[0071] Determine the connected areas of multiple sub-segments whose probability values ​​belonging to the region of interest are greater than a preset probability threshold as the region of interest. The present application can roughly determine the location of the region of interest or part in the medical image by detecting the connected areas without the need for high-precision boundary calculations, which can greatly reduce the amount of calculation and complexity of the model.

[0072] In some embodiments, according to the preconfigured region of interest and the unified coordinate system, extracting reference boundary data includes:

[0073] Determine the azimuth range of the pre-configured part of interest in the unified coordinate system. For example, in some examples, when the reference point is not inside the part of interest, the azimuth range of the part of interest in the coordinate system can be roughly determined based on the established coordinate system, and the boundary data of the azimuth range of the pre-configured 3D heart in the reference point coordinate system can be determined, and multiple boundary data can be extracted within the azimuth range to facilitate subsequent matching.

[0074] According to the determined orientation, a plurality of boundary data of the preconfigured region of interest is extracted within the orientation range as reference boundary data.

[0075] In some embodiments, based on the extracted reference boundary data, locally matching the region of interest in the first medical image to determine continuous key points in the direction of the corresponding line includes:

[0076] The line information in the connected areas of the plurality of sub-divided regions is extracted using the Canny algorithm, and the continuous line length of each line information is determined to extract a plurality of lines with the longest continuous line length.

[0077] For any extracted continuous line, according to the set step length, in any boundary data, the similarity with the any boundary data is calculated by sliding.

[0078] When the similarity is higher than a preset similarity threshold, continuous key points are determined from any continuous lines. In this example, the key points to be checked are further matched and identified in combination with the pre-configured parts of interest.

[0079] In a specific example, a plurality of key points may be set for a preconfigured region of interest, so that in some embodiments, determining continuous key points from any continuous line includes:

[0080] Determine the sub-boundary data with the highest similarity to any of the continuous lines;

[0081] Determining key point information pre-configured in the sub-boundary data;

[0082] The key point information is mapped onto any of the continuous lines to determine the continuous key points on any of the continuous lines. That is, in the embodiment of the present application, the pre-configured key point information is directly mapped according to the similarity of the boundaries, so as to detect continuous key points for the local area of ​​the first medical image to be detected. In the embodiment of the present application, the continuous key points referred to are multiple key points in the direction of any continuous line.

[0083] In some embodiments, determining continuous key points from any of the continuous lines further comprises:

[0084] For a portion of the lines within any one of the continuous lines, reference points are selected from the determined continuous key points. Specifically, multiple reference points may be selected on the same continuous line.

[0085] According to the reference key point pre-configured based on the typical part, the corresponding key point is searched within the pixel distance range between the reference point. In some examples, a floating pixel distance range can be determined according to the distance between the pre-configured key point and the confirmed key point, so as to find the corresponding key point within the pixel distance range.

[0086] The degree of match between the local area centered on the found key point in the first medical image and the local area of ​​the corresponding reference key point is determined. If the degree of match is lower than a preset threshold, the found key point is corrected on the part of the line until the degree of match is higher than the preset threshold. Further, the accurate position of the key point of the medical image to be inspected can be determined by comparing the key point locally and correcting the local key point position.

[0087] In some embodiments, according to the result of the key point association detection, determining the augmented key points from the second medical image includes:

[0088] The second medical image is superimposed on the first medical image based on the overlapping area. In some examples, a new area of ​​interest may appear after the superposition.

[0089] The key points already included in the first medical image are determined, and based on the pre-configured parts of interest, the local areas that exist only in the second medical image are determined.

[0090] Based on the local area that only exists in the second medical image, the extension direction of the line of the key points that have been included is determined, and the key points are searched in the extension direction to determine the expanded key points. That is, for the new area that appears above, the extension direction of the line is further determined, so as to find other key points and realize the expansion of key points.

[0091] An embodiment of the present application further proposes a medical image key point association detection system, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the medical image key point association detection method as described above are implemented.

[0092] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present disclosure with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. It is not limited to the examples described in this specification or during the implementation of this application, and its examples will be interpreted as non-exclusive.

[0093] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art may use other embodiments when reading the above description.

[0094] The above embodiments are merely exemplary embodiments of the present disclosure. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present disclosure, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A method for detecting key point association in medical images, characterized in that: include: Acquire multiple medical images as an image set to be detected; Selecting at least a first medical image and a second medical image from the set of images to be detected; Configuring reference points based on the first medical image and the second medical image, wherein when there is an overlapping area between the first medical image and the second medical image, configuring reference points for the first medical image and the second medical image based on the overlapping area, and when there is no overlapping area between the first medical image and the second medical image, matching a key point from the first medical image based on at least a partial area of ​​a typical part as a reference point, and reselecting a second medical image from the set of images to be detected so that it has an overlapping area with the first medical image, wherein the typical part is a human organ or part of interest; Based on the reference points, a unified coordinate system is established, and a region of interest is determined in the first medical image according to the unified coordinate system; Extracting reference boundary data according to the preconfigured region of interest and the unified coordinate system; Based on the extracted reference boundary data, locally matching the region of interest in the first medical image to determine continuous key points in the direction of the corresponding line; Using the determined continuous key points as a reference, performing association detection on the second medical image based on the unified coordinate system; Determining the amplified key points from the second medical image according to the result of the key point association detection; Repeating the key point amplification of the medical images in the image set to be detected based on the determined key points to complete the key point association detection; Extracting line data from the first medical image and the second medical image, wherein the line data includes boundary data and line information inside the boundary, and the line information is connected to the boundary data or self-closed; The extracted line data are superimposed and compared to determine whether there is a partially overlapping area; When there is an overlapping area in the superposition comparison, a local position where the complexity of line data is higher than a preset threshold is selected from the overlapping area as a reference point; Determining the amplified key points from the second medical image according to the result of the key point association detection includes: superimposing the second medical image and the first medical image based on the overlapping area; and Determine key points already included in the first medical image, and determine local areas that exist only in the second medical image based on pre-configured regions of interest; Based on the local area that only exists in the second medical image, the extension direction of the line that already contains the key points is determined, and the key point search is performed in the extension direction to determine the enlarged key points.

2. The medical image key point association detection method according to claim 1, characterized in that: Configuring the reference point based on the first medical image and the second medical image also includes: When there is no overlapping area in the superposition comparison, the line data of the first medical image is matched with the contour of the typical part to determine the reference point from the matched partial lines.

3. The medical image key point association detection method according to claim 2, characterized in that: Establishing a unified coordinate system based on the reference point, and determining a region of interest in the first medical image according to the unified coordinate system includes: Based on the reference points, a unified coordinate system is established; and, Segmenting the first medical image based on the coordinate system; Extract the image features of each sub-segmentation and input them into the pre-trained recognition model to determine the probability that any sub-segmentation belongs to the region of interest; A plurality of sub-divided connected regions whose probability values ​​belonging to the region of interest are greater than a preset probability threshold are determined as the region of interest.

4. The medical image key point association detection method according to claim 3, characterized in that: According to the pre-configured region of interest and the unified coordinate system, extracting reference boundary data includes: Determine the position range of the pre-configured part of interest in the unified coordinate system; According to the determined orientation, a plurality of boundary data of the preconfigured region of interest is extracted within the orientation range as reference boundary data.

5. The medical image key point association detection method according to claim 4, characterized in that: Based on the extracted reference boundary data, locally matching the region of interest in the first medical image to determine continuous key points in the direction of the corresponding line includes: Extracting line information in the connected areas of the plurality of sub-segments using a Canny algorithm, determining the continuous line length of each line information, and extracting a plurality of lines with the longest continuous line length; For any extracted continuous line, according to a set step length, in any boundary data of the reference boundary data, slidingly calculates the similarity with the any boundary data; When the similarity is higher than a preset similarity threshold, continuous key points are determined from any continuous lines.

6. The medical image key point association detection method according to claim 5, characterized in that: Determining continuous key points from any of the continuous lines includes: Determine the sub-boundary data with the highest similarity to any of the continuous lines; Determining key point information pre-configured in the sub-boundary data; The key point information is mapped onto any of the continuous lines to determine continuous key points on any of the continuous lines.

7. The medical image key point association detection method according to claim 6, characterized in that: Determining continuous key points from any of the continuous lines further includes: For a portion of the lines within any of the continuous lines, selecting reference points from the determined continuous key points; According to the reference key points pre-configured based on the typical parts, searching for corresponding key points within the pixel distance range between the reference points; Determining the degree of match between a local area centered on the found key point in the first medical image and a local area corresponding to the reference key point; When the matching degree is lower than the preset threshold, the key points found are corrected on the part of the lines until the matching degree is higher than the preset threshold.

8. A medical image key point association detection system, characterized in that: The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the medical image key point association detection method as described in any one of claims 1 to 7 are implemented.

Images