Single sample learning path construction method for medical image key point detection

Abstract

The application provides a single sample learning path construction method for medical image key point detection. The method comprises: feature extraction and sample similarity measurement on images in a data set to obtain a similarity score; according to the similarity score, a single typical sample representative in a key point area structure is screened out from the data set; a pseudo label generator is trained based on the typical sample; pseudo label information of key points of other samples in the data set is generated by using the pseudo label generator; the pseudo label information is used as a supervision signal to train a key point detector, and a detection network of the key point detector adopts a U-Net network as a backbone network and a pre-trained VGG19 encoder as a feature extractor of the network; and the key point detector is used to detect key points of a target image and predict a key point heat map and a coordinate offset map. The method of the application reduces labeling cost, improves the precision and stability of key point detection, and enhances the generalization ability.

Description

A Single-Sample Learning Path Construction Method for Keypoint Detection in Medical Images Technical Field

[0001] This invention relates to the field of medical image analysis technology, and in particular to a single-sample learning path construction method for key point detection in medical images. Background Technology

[0002] Keypoint detection in medical images is a crucial task in medical image analysis, aiming to locate key points in clinically significant anatomical structures or lesion regions, such as joints and organ boundaries. In recent years, deep learning methods based on convolutional neural networks (CNNs) and their derivatives have made significant progress in this task. Traditional deep learning methods automatically learn feature representations in medical images by designing multi-layered convolutional neural network models, thereby achieving keypoint localization. However, these methods typically have complex network structures and numerous parameters, requiring large-scale labeled data to fully explore the intrinsic feature relationships and spatial correlations of keypoints. Only with a large training dataset and high-quality annotations can the model stably and accurately predict keypoint locations in new samples. When labeled data is insufficient, the model is prone to overfitting, resulting in decreased generalization ability and poor performance in practical medical applications.

[0003] Medical image datasets face inherent challenges in terms of both scale and quality. On the one hand, data acquisition is costly, involving complex medical equipment, specialized procedures, and patient privacy protection processes, which limits the size of the datasets. On the other hand, annotation requires specialized medical knowledge, is time-consuming and labor-intensive, and differences exist among annotators, resulting in a scarcity of high-quality annotated data. Since 2007, medical image challenges such as SPIE (International Society for Optical Engineering), MICCAI (Conference on Computational Medical Imaging and Computer-Aided Intervention), and ISBI (IEEE International Symposium on Biomedical Imaging) have released a series of keypoint detection datasets, but very few datasets have been publicly released, and they lag significantly behind the field of natural image computer vision in terms of both quantity and scale.

[0004] Therefore, traditional deep learning methods, due to their complex network structures and numerous parameters, require a large amount of labeled data to fully learn keypoint features. However, medical image data is scarce, causing models to fail to capture the spatial correlation information of keypoints, resulting in overfitting and insufficient generalization. While self-supervised and semi-supervised learning reduce the dependence on labels, self-supervised learning relies on pre-designed tasks and may not be directly adaptable to keypoint detection tasks; semi-supervised learning requires a large amount of unlabeled data, and the quality of pseudo-labels is affected by the initial model, easily accumulating errors. Existing methods for few-sample / single-sample learning typically assume a consistent distribution among samples, but individual differences in medical images are significant, and a single sample may not represent the overall features, leading to matching errors or limited generalization ability. Summary of the Invention

[0005] In view of this, the present invention provides a single-sample learning path construction method for key point detection in medical images to solve the above problems.

[0006] This invention provides a single-sample learning path construction method for keypoint detection in medical images, comprising: extracting features and measuring sample similarity from images in a dataset to obtain a similarity score; selecting a representative single typical sample from the dataset based on the similarity score, wherein the representative typical sample refers to a single labeled image in the medical image dataset whose keypoint features are most significant and stable, whose region has a clear anatomical structure, clear boundaries, and is easy to distinguish, and whose keypoint features are consistent with the features of other samples at the same anatomical location; training a pseudo-label generator based on the typical sample; generating pseudo-label information for keypoints of other samples in the dataset using the pseudo-label generator; training a keypoint detector using the pseudo-label information as a supervision signal, wherein the detection network of the keypoint detector adopts a U-Net network as the backbone network and a pre-trained VGG19 encoder as the feature extractor of the network; and performing keypoint detection on the target image through the keypoint detector and predicting keypoint heatmaps and coordinate offset maps.

[0007] In another implementation of the present invention, the step of performing feature extraction and sample similarity measurement on the images in the dataset to obtain a similarity score includes: using a scale-invariant feature transformation algorithm to extract multi-scale feature points from the medical images; and calculating the similarity between the images based on the extracted feature points to obtain a similarity score.

[0008] In another implementation of the present invention, for each medical image X, the set of key points obtained by the multi-scale feature point extraction is represented as:

[0009]

[0010] Where K represents the number of key points.

[0011] In another implementation of the present invention, for image m, the similarity score Represented as:

[0012]

[0013] Where K represents the number of keypoints in m, and N represents the number of samples in the training set. Let represent the similarity score between the i-th key point of image m and image j.

[0014] In another implementation of the present invention, the step of generating pseudo-label information for key points of other samples in the dataset using the pseudo-label generator includes: inputting sample images from the dataset into the pseudo-label generator; selecting a key point in the sample image, randomly cropping an image patch containing the key point from the sample image, and performing data augmentation operations on the image patch, including rotating and color dithering the image patch, to generate preliminary pseudo-labels; using two feature extractors to perform multi-scale feature extraction on the sample image and the image patch respectively, obtaining a series of feature representations from coarse to fine; calculating the coordinates of the key point in the image patch at each scale by using a downsampling ratio, extracting the corresponding features, and performing cosine similarity calculation with the features at the scale corresponding to the sample image to generate a similarity map at each scale; multiplying the similarity maps at each scale layer by layer, and using... The operation selects the maximum value point and uses this point as a pseudo-label for the key point in the sample image.

[0015] In another implementation of the present invention, the loss function of the pseudo-label generator is expressed as:

[0016]

[0017] in, It is a probability matrix. The true values ​​constructed for the probability matrix. This represents the cross-entropy loss.

[0018] In another implementation of the present invention, the method further includes: defining a joint loss function for each keypoint of the input image A in order to enable the keypoint detector to simultaneously learn heatmap prediction and offset correction.

[0019]

[0020] Where k represents the k-th key point of the input image A, Indicates heatmap loss. This represents a predicted heatmap. Represents a true heatmap. This represents the loss in the X-axis displacement offset diagram. This represents the predicted X-axis offset plot. This represents the true X-axis offset. This represents the loss in the Y-axis displacement offset diagram. This represents the predicted Y-axis offset plot. This represents the true Y-axis offset.

[0021] In another aspect, the present invention provides a single-sample learning path construction system for keypoint detection in medical images, comprising: an image preprocessing module for extracting features and measuring sample similarity of images in a dataset to obtain a similarity score; an image filtering module for selecting a single representative typical sample from the dataset based on the similarity score, wherein the representative typical sample refers to a single labeled image in which the keypoint features are most significant and stable in the medical image dataset, the region has a clear anatomical structure, clear boundaries and is easy to distinguish, and the keypoint features are consistent with the features of other samples at the same anatomical location; a pseudo-label generation module for training a pseudo-label generator based on the typical sample and generating pseudo-label information for keypoints of other samples in the dataset using the pseudo-label generator; a keypoint detector training module for training a keypoint detector using the pseudo-label information as a supervision signal, wherein the detection network of the keypoint detector adopts a U-Net network as the backbone network and a pre-trained VGG19 encoder as the feature extractor of the network; and a keypoint detection module for detecting keypoints in the target image using the keypoint detector and predicting keypoint heatmaps and coordinate offset maps.

[0022] In another aspect, the present invention provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of a single-sample learning path construction method for medical image keypoint detection as described in any of the preceding claims.

[0023] In another aspect, the present invention provides a computer storage medium storing a computer program that, when executed by a processor, implements the steps of a single-sample learning path construction method for medical image keypoint detection as described in any of the preceding claims.

[0024] The single-sample learning path construction method for keypoint detection in medical images of this invention requires only one or a very small number of labeled samples to train a usable keypoint detector, making it suitable for rare diseases or data-constrained scenarios and significantly reducing labeling costs. It ensures representativeness through automatic selection of typical samples and combines SIFT local invariance with deep semantic multi-scale matching to generate pseudo-labels, improving pseudo-label quality and detection stability and reducing performance degradation caused by misleading pseudo-labels. Multi-scale cascaded matching and data augmentation make pseudo-label generation robust to deformation, rotation, scale, and imaging differences. The combined self-training loop further enhances adaptability to unlabeled samples and strengthens generalization ability. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. By reading the detailed description of the embodiments below, the advantages and benefits of the solutions will become clear to those skilled in the art. The accompanying drawings are only for illustrating preferred embodiments and are not intended to limit the present invention. In the accompanying drawings:

[0026] Figure 1 is a schematic flowchart of a single-sample learning path construction method for key point detection in medical images according to an embodiment of the present invention.

[0027] Figure 2 is a flowchart of a single-sample learning path construction method for medical image key point detection according to an embodiment of the present invention.

[0028] Figure 3 is a schematic diagram of a typical sample selection process according to an embodiment of the present invention.

[0029] Figure 4 is a schematic diagram of the training process of a pseudo-label generator according to an embodiment of the present invention.

[0030] Figure 5 is a schematic diagram of a key point detection network according to an embodiment of the present invention.

[0031] Figure 6 is a schematic diagram of the key point detection effect of an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art should fall within the protection scope of the present invention.

[0033] Figure 1 is a schematic flowchart of a single-sample learning path construction method for key point detection in medical images provided by an embodiment of the present invention. As shown in Figures 1 and 2, this embodiment mainly includes:

[0034] S101. Perform feature extraction and sample similarity measurement on the images in the dataset to obtain similarity scores.

[0035] S102. Based on the similarity score, select a single typical sample that is representative in terms of key point region structure from the dataset. The representative typical sample refers to a single labeled image in the medical image dataset where the key point features are most significant and stable, the region has clear anatomical structure, clear boundaries and is easy to distinguish, and the key point features are consistent with the features of other samples at the same anatomical location.

[0036] For example, as shown in Figure 3, a typical sample with high learning value is selected from the dataset. The selection criteria include: the features corresponding to the key points are representative in the entire dataset; the anatomical structure of the region where the key points are located is clear, the boundaries are clear and easy to distinguish; and the key point features of the sample are consistent with the features of other samples at the same anatomical location, so as to achieve better generalization when matching single samples.

[0037] S103. Based on the typical samples, train the pseudo-label generator.

[0038] S104. Use the pseudo-label generator to generate pseudo-label information for key points of other samples in the dataset.

[0039] For example, based on selected typical samples, a multi-scale feature extraction method is used to achieve feature matching of key points from global to local through cascaded feature comparison.

[0040] Specifically, data augmentation (such as rotation and scaling) is performed on labeled and unlabeled samples to generate image patches. Similar regions of anatomical structures are captured in the augmented samples to generate preliminary pseudo-labels. The pseudo-label generator is trained using only a single labeled sample, but through data augmentation and representative feature representation, it can capture the common anatomical features of most samples in the dataset, providing effective supervision signals for subsequent training.

[0041] S105. The pseudo-label information is used as a supervision signal to train the key point detector. The detection network of the key point detector adopts U-Net network as the backbone network and a pre-trained VGG19 encoder as the feature extractor of the network.

[0042] For example, training set pseudo-labels generated by a pseudo-label generator are used to learn the model using the U-Net architecture as the backbone network.

[0043] S106. The key point detector is used to detect key points in the target image and predict key point heatmaps and coordinate offset maps.

[0044] For example, the network simultaneously predicts keypoint heatmaps and coordinate offset maps, and uses global contextual information and local detail features for joint optimization to improve detection accuracy.

[0045] The single-sample learning path construction method for keypoint detection in medical images of this invention requires only one or a very small number of labeled samples to train a usable keypoint detector, making it suitable for rare diseases or data-constrained scenarios and significantly reducing labeling costs. It ensures representativeness through automatic selection of typical samples and combines SIFT local invariance with deep semantic multi-scale matching to generate pseudo-labels, improving pseudo-label quality and detection stability and reducing performance degradation caused by misleading pseudo-labels. Multi-scale cascaded matching and data augmentation make pseudo-label generation robust to deformation, rotation, scale, and imaging differences. The combined self-training loop further enhances adaptability to unlabeled samples and strengthens generalization ability.

[0046] In another implementation of the present invention, the step of performing feature extraction and sample similarity measurement on the images in the dataset to obtain a similarity score includes: using a scale-invariant feature transformation algorithm to extract multi-scale feature points from the medical images; and calculating the similarity between the images based on the extracted feature points to obtain a similarity score.

[0047] In another implementation of the present invention, for each medical image X, the set of key points obtained by the multi-scale feature point extraction is represented as:

[0048]

[0049] Where K represents the number of key points.

[0050] For example, the Scale-Invariant Feature Transform (SIFT) algorithm is used to extract multi-scale feature points from medical images. The SIFT algorithm constructs a Gaussian pyramid and detects local extrema in scale space, obtaining keypoint descriptors that are robust to changes in rotation, scaling, and illumination. These candidate keypoints are spatially similar to or overlap with the actual medical keypoint regions. For each medical image X, the SIFT algorithm extracts a set of K keypoints.

[0051] In another implementation of the present invention, for image m, the similarity score Represented as:

[0052]

[0053] Where K represents the number of keypoints in m, and N represents the number of samples in the training set. Let the similarity score between the i-th keypoint of image m and image j be expressed by the formula:

[0054]

[0055] In the formula, For feature extractor, This represents the i-th candidate keypoint extracted in image m using the SIFT algorithm. The feature vector at that location, In the image Key points of the candidate The eigenvector at that location.

[0056] Keypoints extracted from different medical images using the SIFT algorithm do not have a one-to-one correspondence; therefore, it is necessary to calculate the similarity of each keypoint in image X. Since each sample has the same resolution, to reduce computational cost, the image... Chinese correspondence k points with a radius of 200 pixels centered at pixel coordinates and The similarity of feature vectors is achieved through... The similarity value obtained from the operation is assigned to In the feature similarity calculation process, the cosine similarity function is selected. It can be expressed by the formula:

[0057]

[0058] In the formula, This represents the feature vector. It is calculated by setting the similarity score between each image in the sample set and other images. This method can quantify the representativeness of each image sample to the entire dataset to a certain extent, thereby selecting the typical samples with the highest similarity to most image features and using them as labeled samples for subsequent single-sample learning. This can effectively improve the generalization performance and detection accuracy of single-sample models in medical image keypoint detection tasks.

[0059] In another implementation of the present invention, the step of generating pseudo-label information for key points of other samples in the dataset using the pseudo-label generator includes: inputting sample images from the dataset into the pseudo-label generator; selecting a key point in the sample image, randomly cropping an image patch containing the key point from the sample image, and performing data augmentation operations on the image patch, including rotating and color dithering the image patch, to generate preliminary pseudo-labels; using two feature extractors to perform multi-scale feature extraction on the sample image and the image patch respectively, obtaining a series of feature representations from coarse to fine; calculating the coordinates of the key point in the image patch at each scale by using a downsampling ratio, extracting the corresponding features, and performing cosine similarity calculation with the features at the scale corresponding to the sample image to generate a similarity map at each scale; multiplying the similarity maps at each scale layer by layer, and using... The operation selects the maximum value point and uses this point as a pseudo-label for the key point in the sample image.

[0060] For example, a local region centered on keypoints is cropped out, and then a probability distribution map of the keypoints is obtained through normalization. The cross-entropy loss is calculated by comparing the map with the true values, and this map is used as a supervisory signal to train the entire network. The entire process achieves multi-scale matching from coarse to fine, and through layer-by-layer optimization, the model is able to capture stable anatomical structural information.

[0061] As shown in Figure 4, the specific process for training the pseudo-label generator is as follows:

[0062] First, the sample image Size adjustment The coordinates of a key point in the image are ,exist A random image patch containing the target key point is cropped from the image. Its size is .

[0063] right After performing the corresponding data augmentation operations, the coordinates of the target key points will change accordingly. .

[0064] Use feature extractors to extract respectively and Mapping to a multi-scale feature space yields cascaded feature representations at different scales, denoted as... and , No. The characteristics of a layer are denoted as .

[0065] Based on the corresponding downsampling ratio, using coordinates in Calculate the key points in the first place The coordinates of the layer are:

[0066]

[0067] Cascaded features refer to a series of features extracted from different layers of a deep network, with each layer's features reflecting information about the image at different scales and semantic levels. In this invention, cascaded features progressively extract sample image feature information from deep to shallow layers.

[0068] Features of each layer Extract The characteristics of the location are denoted as Then compare the sample images. In the Layer characteristics Features of key points at cropped image patches cosine similarity map Its formula is expressed as:

[0069]

[0070] In the formula, Indicates the inner product. express Norm.

[0071] Different levels of similarity maps serve different purposes. The deepest layer is responsible for distinguishing the approximate regions of target key points in the entire image, while shallower layers provide more precise location of key points.

[0072] For the Layer, if this layer is the deepest layer, let the region of interest matrix be... When it is a shallower layer, From China and Israel A side with a length of is cut out from the center. The square area will conduct The operation yields a normalized probability matrix. Expressed as a formula:

[0073]

[0074] In the formula, This is the temperature coefficient.

[0075] Construct the true values ​​for the probability matrix obtained at each layer. It is centered on the true coordinates of the key points, and the size of the matrix is ​​related to the probability matrix of each layer. The sizes are the same, the center point is set to 1, and the rest are set to 0.

[0076] For each layer, the true value With probability matrix The difference is achieved using cross-entropy loss. The loss function of the network is calculated by summing the losses of each layer. ,

[0077] After the pseudo-label generator is trained, it performs pseudo-label generation on other sample images in the training set.

[0078] First, image patches containing each key point are extracted from the labeled sample images. Then, a feature extractor is used to obtain cascaded features from the image patches containing key points and the unlabeled images.

[0079] Then, features of key points in image patches at different scales are used. Multi-scale cosine similarity calculations are performed with the sample images to obtain similarity maps for each layer. Finally, the similarity maps of each layer are multiplied layer by layer, and then... The operation selects the maximum value point, which becomes the pseudo-label for the key point in the unlabeled sample image.

[0080] In another implementation of the present invention, the loss function of the pseudo-label generator is expressed as:

[0081]

[0082] in, It is a probability matrix. The true values ​​constructed for the probability matrix. This represents the cross-entropy loss.

[0083] For example, as shown in Figure 5, during the keypoint detection network training phase, a novel keypoint detector is trained using generated pseudo-labels as supervisory signals. The detection network employs a U-Net network as its backbone, with a pre-trained VGG19 encoder as its feature extractor. In the decoder section, multi-scale features of the image are generated through upsampling. The network output consists of three parts: a keypoint heatmap, and coordinate offset maps along the X and Y axes for predicted coordinates. In the keypoint heatmap, the value of each pixel represents the probability that the location is a keypoint. The coordinate offset maps along the X and Y axes are used for fine-tuning the coarse localization results of the heatmap.

[0084] For the input image The first in The key points have the following true coordinates: Considering the errors introduced by pseudo-tags themselves, at real key points... Set a radius around it. A circular region, where the probability of each pixel value within the region is set to 1, i.e., a key point. The probability heatmap is represented as , The following conditions must be met:

[0085]

[0086] Define the offset map as and , representing the coordinates of each pixel in the image. Value and Values ​​and Key Points Real coordinates The relative offset is then normalized. and It can be expressed by the following formula:

[0087]

[0088]

[0089] To enable the network to learn heatmap prediction and offset correction simultaneously, a joint loss function is defined for each keypoint of the input image A. Its formula is as follows:

[0090]

[0091] In the formula, k represents the k-th key point of the input image A. This represents the heatmap loss, calculated by predicting the heatmap. Compared to real heatmaps The binary cross-entropy loss between them is obtained; This represents the loss in the X-axis displacement map, which is calculated to predict the X-axis displacement map. Offset from the actual X-axis of Loss is gained; similarly, This represents the loss in the Y-axis displacement map, which is calculated to predict the Y-axis displacement map. Offset from the actual Y-axis of The loss was recovered.

[0092] The network training loss is obtained by summing the losses of all keypoints. This joint loss design ensures that the network can not only identify the approximate location of keypoints globally through heatmaps, but also accurately correct the keypoint locations using offset maps.

[0093] In another implementation of the present invention, the local saliency operator can be replaced: SIFT can be replaced with local descriptors such as SURF, ORB, AKAZE, or SIFT can be replaced with a learning keypoint detector (such as SuperPoint) to adapt to different computing resources and speed requirements.

[0094] In another implementation of the present invention, a differentiated similarity measure can be used: in addition to cosine similarity (CosSim), mutual information (MI), normalized correlation coefficient (NCC), a metric learning-based metric function, or a learnable similarity network can be used to replace or supplement it.

[0095] In another implementation of the present invention, the backbone network can be replaced: U-Net can be replaced with U-Net++ or UNet-3+ with attention mechanism, or a Transformer-based segmentation / detection architecture; the encoder can be ResNet, EfficientNet, Swin Transformer, etc.

[0096] Example 1

[0097] As shown in Figure 6, a comparative experiment was conducted on the DHA hand X-ray image dataset, comparing it with other single-sample learning path construction methods for medical image keypoint detection. The experimental results show that, in the hand X-ray image keypoint detection task, the method proposed in this invention achieves better results in both localization accuracy and detection robustness.

[0098] Furthermore, a comparative experiment was conducted with deep learning methods on the DHA hand X-ray image dataset. With a very small number of labeled samples, the present invention has significant advantages over traditional deep learning methods.

[0099] In addition, comparative experiments with other single-sample learning methods were conducted on the 2015 ISBI head X-ray image dataset, achieving better keypoint detection results with only a single labeled sample.

[0100] The method of this invention reduces the dependence on large-scale labeled data, thereby reducing the cost of data acquisition and labeling; it improves the accuracy and stability of key point detection under single-sample learning and avoids overfitting; it adapts to individual differences in medical images and enhances generalization ability through representative samples and feature matching.

[0101] Another aspect of the present invention provides a single-sample learning path construction system for keypoint detection in medical images, comprising:

[0102] Image preprocessing module: Performs feature extraction and sample similarity measurement on images in the dataset to obtain similarity scores.

[0103] Image filtering module: Based on the similarity score, select a single typical sample that is representative in terms of key point region structure from the dataset. The representative typical sample refers to a single labeled image in the medical image dataset whose key point features are the most significant and stable, whose region has clear anatomical structure, clear boundaries and is easy to distinguish, and whose key point features are consistent with the features of other samples at the same anatomical location.

[0104] Pseudo-label generation module: Based on the typical samples, train a pseudo-label generator; use the pseudo-label generator to generate pseudo-label information for key points of other samples in the dataset.

[0105] Keypoint detector training module: The keypoint detector is trained using the pseudo-label information as a supervision signal. The detection network of the keypoint detector adopts U-Net network as the backbone network and a pre-trained VGG19 encoder as the feature extractor of the network.

[0106] Key point detection module: The key point detector is used to detect key points in the target image and predict key point heatmaps and coordinate offset maps.

[0107] The single-sample learning path construction system for keypoint detection in medical images of this invention requires only one or a very small number of labeled samples to train a usable keypoint detector, making it suitable for rare diseases or data-constrained scenarios and significantly reducing labeling costs. It ensures representativeness through automatic selection of typical samples and combines SIFT local invariance with deep semantic multi-scale matching to generate pseudo-labels, improving pseudo-label quality and detection stability and reducing performance degradation caused by misleading pseudo-labels. Multi-scale cascaded matching and data augmentation make pseudo-label generation robust to deformation, rotation, scale, and imaging differences. The combined self-training loop further enhances adaptability to unlabeled samples and strengthens generalization ability.

[0108] In another aspect of the present invention, the electronic device includes: a processor, a memory, and a communication bus and a communication interface.

[0109] in:

[0110] The processor, memory, and communication interface communicate with each other via a communication bus.

[0111] A communication interface is used to communicate with other electronic devices or servers.

[0112] The processor is used to execute programs, specifically, it can execute any of the steps of the single-sample learning path construction method for medical image keypoint detection in the above embodiments.

[0113] Specifically, the program may include program code, which includes computer operation instructions.

[0114] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0115] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0116] Specifically, the program can be used to cause the processor to execute the steps of any of the single-sample learning path construction methods for medical image keypoint detection described in the embodiments. The specific implementation of each step in the program can be found in the corresponding descriptions of the steps and units executed in any of the single-sample learning path construction methods for medical image keypoint detection described above, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments.

[0117] An exemplary embodiment of this application also provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods of various embodiments of this application.

[0118] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0119] Specific embodiments of the present invention have now been described. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results.

[0120] It should be noted that all directional indications (such as up, down, left, right, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship between the components in a certain order (as shown in the figure). If the specific order changes, the directional indication will also change accordingly.

[0121] In the description of this invention, the terms "first" and "second" are used only for convenience in describing different components or names, and should not be construed as indicating or implying a sequential relationship, relative importance, or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" and "second" may explicitly or implicitly include at least one of that feature.

[0122] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0123] It should be noted that although specific embodiments of the present invention have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of the present invention. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of the present invention.

[0124] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be an improper limitation of the embodiments of the present invention.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A single-sample learning path construction method for keypoint detection in medical images, characterized in that, include: Feature extraction and sample similarity measurement are performed on the images in the dataset to obtain similarity scores, including: extracting multi-scale feature points from the medical images using a scale-invariant feature transform algorithm; calculating the similarity between images based on the extracted feature points to obtain similarity scores; for each medical image X, the set of key points obtained from the multi-scale feature point extraction is represented as: Where K represents the number of key points; for image m, the similarity score Represented as: Where K represents the number of keypoints in m, and N represents the number of samples in the training set. The similarity score between the i-th keypoint of image m and image j can be expressed by the formula: In the formula, For feature extractor, This represents the i-th candidate keypoint extracted in image m using the SIFT algorithm. The feature vector at that location, In the image Key points of the candidate The feature vector at the location is obtained; based on the similarity score, a single typical sample with representative key point region structure is selected from the dataset. The representative typical sample refers to a single labeled image in the medical image dataset where the key point features are most significant and stable, the region has a clear anatomical structure, clear boundaries and is easy to distinguish, and its key point features are consistent with the features of other samples at the same anatomical location; a pseudo-label generator is trained based on the typical sample; pseudo-label information of key points of other samples in the dataset is generated using the pseudo-label generator; the pseudo-label information is used as a supervision signal to train the key point detector, the detection network of the key point detector adopts U-Net network as the backbone network and pre-trained VGG19 encoder as the feature extractor of the network; the key point detector is used to detect key points in the target image and predict key point heatmaps and coordinate offset maps.

2. The method according to claim 1, characterized in that, The process of generating pseudo-label information for key points of other samples in the dataset using the pseudo-label generator includes: inputting sample images from the dataset into the pseudo-label generator; selecting a key point in the sample image, randomly cropping an image patch containing the key point from the sample image, and performing data augmentation operations on the image patch, including rotating and color jittering, to generate preliminary pseudo-labels; using two feature extractors to perform multi-scale feature extraction on the sample image and the image patch respectively, obtaining a series of feature representations from coarse to fine; calculating the coordinates of the key point in the image patch at each scale by using a downsampling ratio, extracting the corresponding features, and performing cosine similarity calculation with the features at the scale corresponding to the sample image to generate a similarity map at each scale; multiplying the similarity maps at each scale layer by layer, and using... The operation selects the point with the maximum value and uses this point as a pseudo-label for the key point in the sample image.

3. The method according to claim 2, characterized in that, The loss function of the pseudo-label generator is expressed as: in, It is a probability matrix. The true values ​​constructed for the probability matrix. This represents the cross-entropy loss.

4. The method according to claim 1, characterized in that, Also includes: To enable the keypoint detector to simultaneously learn heatmap prediction and offset correction, a joint loss function is defined for each keypoint in the input image A: Where k represents the k-th key point of the input image A, Indicates heatmap loss. This represents a predicted heatmap. Represents a true heatmap. This represents the loss in the X-axis displacement offset diagram. This represents the predicted X-axis offset plot. This represents the true X-axis offset. This indicates the loss in the Y-axis displacement offset diagram. This represents the predicted y-axis offset plot. This represents the actual Y-axis offset.

5. A single-sample learning path construction system for key point detection in medical images, characterized in that, include: Image preprocessing module: Performs feature extraction and sample similarity measurement on images in the dataset to obtain similarity scores, including: extracting multi-scale feature points from medical images using a scale-invariant feature transform algorithm; calculating the similarity between images based on the extracted feature points to obtain similarity scores; for each medical image X, the set of key points obtained from the multi-scale feature point extraction is represented as: Where K represents the number of key points; for image m, the similarity score Represented as: Where K represents the number of keypoints in m, and N represents the number of samples in the training set. The similarity score between the i-th keypoint of image m and image j can be expressed by the formula: In the formula, For feature extractor, This represents the i-th candidate keypoint extracted in image m using the SIFT algorithm. The feature vector at that location, In the image Key points of the candidate The image filtering module selects representative single typical samples from the dataset based on the similarity score. A representative typical sample refers to a single labeled image in the medical image dataset where the keypoint features are most significant and stable, the region has a clear anatomical structure, clear boundaries, and is easily distinguishable, and its keypoint features are consistent with those of other samples at the same anatomical location. The pseudo-label generation module trains a pseudo-label generator based on the typical samples and uses the pseudo-label generator to generate pseudo-label information for keypoints of other samples in the dataset. The keypoint detector training module uses the pseudo-label information as a supervisory signal to train the keypoint detector. The detection network of the keypoint detector uses a U-Net network as the backbone network and a pre-trained VGG19 encoder as the network's feature extractor. The keypoint detection module performs keypoint detection on the target image using the keypoint detector and predicts keypoint heatmaps and coordinate offset maps.

6. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the single-sample learning path construction method for medical image keypoint detection as described in any one of claims 1 to 4.

7. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the steps in the single-sample learning path construction method for medical image keypoint detection as described in any one of claims 1 to 4.

Images