A medical image similarity detection method using neural network feature matching, and an electronic device, computer readable storage medium and computer software product

Abstract

A medical image similarity detection method using neural network feature matching, an electronic device, a computer readable storage medium and a computer software product. The method comprises: using a trained convolutional neural network model to identify the type of an input medical paper image, and dividing it into four categories: flow image, fluorescence image, granular fluorescence image and other images; image preprocessing is performed on the four types of images; feature extraction is performed on the preprocessed images of different types, and the extracted features are matched; whether two images are similar is judged according to the number of matched feature points and the proportion of the number of matched points in the total number of points; if similar, further analyze the transformation parameters; for the detected similar image pairs, draw feature matching points, label similar regions, then visualize the output result image and output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a medical image similarity detection method using neural network feature matching, an electronic device, a computer readable storage medium and a computer software product. BACKGROUND

[0002] In modern medical research and clinical practice, image data as an important data carrier is widely used in disease diagnosis, efficacy evaluation, scientific research paper publication and other aspects. Especially in medical scientific research papers, images are not only used to visually display research results, but also often used as key evidence to support experimental conclusions. However, in the process of paper writing and submission, the phenomenon of image reuse, misuse or even forgery is common, which seriously damages academic integrity and scientific research quality. Therefore, establishing an efficient, accurate and intelligent image similarity detection mechanism is of great significance to improve the authenticity and credibility of scientific research results.

[0003] At present, most medical institutions and scientific research units mainly rely on manual review to check the repeatability and similarity of images in papers. This method not only consumes time and effort, but also is easily affected by subjective judgment, resulting in a high rate of missed detection or misjudgment. In addition, some hospitals try to use image duplication checking services provided by third parties, which improves the detection efficiency to some extent, but also brings many problems: such as high service cost, long response cycle, inability to meet large-scale concurrent processing needs, and most importantly, data privacy and security issues. Since medical images often contain sensitive information of patients, if they are processed by external agencies, it is easy to cause data leakage risk, especially in unpublished research results, this potential threat is more prominent.

[0004] Existing image similarity detection techniques mainly include traditional image processing algorithms, such as histogram comparison, structural similarity index (SSIM), hash algorithm, SIFT (Scale Invariant Feature Transform) and SURF (Speeded Up Robust Features) feature point matching methods. A histogram comparison is shown in Figure 2 These methods have achieved certain results in the field of general images, but have obvious limitations in the professional application scenario of medical images. For example, medical images usually have high resolution and complex internal structure, and traditional methods are difficult to effectively capture their deep semantic information; at the same time, medical images often have scale changes, rotation shifts, brightness differences and other problems in the acquisition process due to factors such as device parameters, shooting angles and imaging conditions, resulting in large appearance differences between images, which affects the accuracy of similarity judgment.

[0005] In addition, the current technical means generally lacks the ability of automatic identification and classification of image types, and cannot perform differential processing according to medical images of different modalities (such as X-ray, CT, MRI, ultrasound, etc.), further reducing the applicability and generalization ability of the system. In the face of the increasing amount of medical image data and the diversification of image types, an intelligent image similarity detection system with adaptive learning and high-precision identification capability is urgently needed.

[0006] More importantly, the present application can be deployed in a hospital local server environment, completely avoiding the privacy leakage risk brought by uploading sensitive data to the cloud or third-party platform. Local deployment not only guarantees the security and controllability of data, but also significantly reduces the long-term operation cost and improves the self-management ability of the hospital to the image duplication detection system. By introducing artificial intelligence technology, the present application is expected to greatly improve the automation level of image review, reduce manual intervention, improve overall work efficiency, and provide strong support for quality control of medical research.

[0007] In summary, the present application proposes an efficient, intelligent and secure solution to the problems of low image recognition accuracy, poor adaptability and insufficient privacy protection in the prior art, and has good application prospect and promotion value. SUMMARY

[0008] The purpose of the embodiments of the present application is to provide a medical image similarity detection method using neural network feature matching, electronic equipment, computer readable storage medium and computer software product.

[0009] A medical image similarity detection method using neural network feature matching, comprising:

[0010] The trained convolutional neural network model is used for type identification of the input medical paper image, which is divided into four categories: flow image, fluorescence image, granular fluorescence image and other images;

[0011] The images of the four categories are preprocessed respectively, wherein the preprocessing of the flow image includes cutting the edge, removing the coordinate axis, improving the resolution and grayscale; the preprocessing of the fluorescence image / granular fluorescence image and other images includes reducing the resolution, grayscale and balancing the brightness;

[0012] The features of the preprocessed images of different types are extracted, and the extracted features are matched, wherein: the SIFT algorithm is used for feature extraction and matching of the flow image; the AKAZE algorithm is used for feature extraction and matching of the fluorescence image / granular fluorescence image and other images;

[0013] According to the number of matched feature points and the proportion of the number of matched points in the total number of points, it is determined whether the two images are similar; if similar, the transformation parameters are further analyzed;

[0014] For the detected similar image pairs, the feature matching points are drawn, the similar regions are labeled, and then the result image is visualized and output.

[0015] An electronic device includes a processor, a memory, and a computer program executable on the processor, and the processor implements the above method when executing the program.

[0016] A computer readable storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the above method.

[0017] A computer software product includes a computer program / instruction, which is executed by a processor to implement the above method.

[0018] The above embodiments of the present application propose a medical image similarity detection method based on deep learning and feature matching combination, which has the following significant technical effects:

[0019] 1. High accuracy intelligent classification and similarity detection

[0020] The deep learning technology is used to automatically classify medical images, and targeted preprocessing and feature matching strategies are adopted according to different categories of images, which effectively improves the accuracy of image similarity detection.

[0021] In actual tests, the overall accuracy of the method in image classification and similarity judgment can reach more than 90%, with good recognition ability.

[0022] 2. High efficiency batch processing ability, significantly improving detection efficiency

[0023] The system supports automatic and batch processing of a large number of medical paper image data without manual intervention.

[0024] The image processing speed can reach 20 images per minute, greatly shortening the image review time period and significantly improving the work efficiency of duplicate checking and quality control.

[0025] 3. Strong robustness, adapt to complex image changes

[0026] Through systematic image preprocessing (such as scale normalization, grayscale conversion, brightness balance, etc.) and post-processing (such as feature point labeling, region visualization), the problems of multi-scale, rotation, noise interference, etc. in medical images are effectively solved.

[0027] The adaptability of the algorithm to different imaging conditions and image transformations is improved, and the stability and robustness of the system are enhanced.

[0028] 4. Intuitive visual output to assist manual review

[0029] For the detected similar image pairs, the system can automatically generate a visualization result graph containing feature matching points and corresponding region annotations.

[0030] The visualization information is clear and intuitive, which helps researchers or reviewers quickly locate similar content, providing strong support for the authenticity review of medical papers.

[0031] 5. Low cost, high security, and protection of data privacy

[0032] Compared with the traditional image duplication detection method relying on third-party services, the present application can be deployed on a local server in the hospital, avoiding the uploading of sensitive images to external platforms.

[0033] This significantly reduces the cost of image duplication detection in hospitals, while greatly improving the privacy protection level and security of medical data, preventing data leakage risks.

[0034] 6. Modular design, easy to extend and maintain

[0035] The system adopts a modular architecture design, with clear responsibilities and low coupling between function modules, facilitating flexible deployment, function expansion, and system upgrade according to the actual needs of the hospital.

[0036] Supporting high availability and long-term maintainability, it is suitable for popularization and application in various medical institutions.

[0037] In summary, the present application not only performs well in image recognition accuracy, processing efficiency, and robustness, but also takes into account data security, cost control, and system scalability, with wide application prospects and promotional value. BRIEF DESCRIPTION OF DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments described in the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0039] Figure 1 The method flowchart in an embodiment of the present application;

[0040] Figure 2 The histogram comparison diagram in an embodiment of the present application;

[0041] Figure 3 A schematic diagram showing the matching points of two images in an embodiment of the present application;

[0042] Figure 4 A schematic diagram of the similar part in two images in an embodiment of the present application;

[0043] Figure 5 A "matching connection" between two images in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to enable persons skilled in the art to better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should fall within the scope of protection of the present application.

[0045] The present application provides a medical image similarity detection method based on neural network feature matching, aiming to solve the problems of ineffective identification of repeated or highly similar medical images, weak image classification ability, low accuracy of duplicate checking, and insufficient data privacy protection in the prior art. This method combines the advantages of deep convolutional neural networks and traditional image feature extraction algorithms to achieve automatic classification and fine similarity detection of medical images, and supports differential processing of multiple image types.

[0046] The present application provides a medical image similarity detection method using neural network feature matching, as shown in Figure 1 The method comprises the following steps:

[0047] S110: Use the trained convolutional neural network model to identify the type of the input medical paper image, and divide it into four categories: flow image, fluorescence image, granular fluorescence image, and other images.

[0048] First, image classification is performed, which can utilize a trained Convolutional Neural Network (CNN) model to identify the type of input medical paper images and divide them into four categories: streaming images, fluorescent images, granular fluorescent images, and other images. This allows for differentiated processing strategies based on different image categories. In the image classification stage, a trained Convolutional Neural Network (CNN) model is used to identify the type of input medical paper images. This process involves not only the basic principles of CNN and its applications, but also the selection of model design, training process, and suitability analysis.

[0049] A Convolutional Neural Network (CNN) is a deep learning model specifically designed to handle data with grid-like structures, such as images. CNN mimics the way the human visual system works to automatically learn and extract feature representations from input data, and has achieved remarkable success in computer vision tasks, including but not limited to image classification, object detection, and image segmentation. It automatically learns the feature representation of input data through a series of convolutional layers, pooling layers, and fully connected layers, and finally realizes tasks such as classification or regression. The main advantage of CNN is that it can effectively capture the spatial hierarchy of input data while reducing the number of parameters, thus avoiding overfitting problems.

[0050] The basic component layers of CNN include:

[0051] 1. Convolutional Layer

[0052] Function: Convolutional layer is one of the core components of CNN, which uses a set of learnable filters (also known as kernels or convolution kernels) to scan the input data to generate feature maps. Each filter focuses on a specific type of feature, such as edges or textures.

[0053] Connection method: Each neuron in the convolutional layer is only connected to a local region of input data, which reduces the number of parameters and helps improve computational efficiency and reduce the risk of overfitting.

[0054] 2. Activation Function Layer

[0055] Action: Usually followed by convolutional layers, activation functions such as ReLU, Sigmoid, etc. are used to increase the non-linear representation ability of the network, so that the model can learn more complex patterns.

[0056] 3、Pooling Layer

[0057] Action: Pooling layers are used to reduce the spatial dimensions of feature maps, reduce computational complexity and control overfitting. The most commonly used pooling operation is Max Pooling, which selects the maximum value in each sliding window as the output, thus preserving the most important information.

[0058] 4、Normalization Layer

[0059] Action: Such as Batch Normalization, used to accelerate the training process, stabilize gradient descent, and allow higher learning rates, thereby improving model performance.

[0060] 5、Fully Connected Layer

[0061] Action: In the final stage of CNN, the fully connected layer maps all features to a fixed length vector, and then makes the final prediction based on this vector. This is a key step to implement classification or regression tasks.

[0062] 6、Dropout Layer

[0063] Action: To prevent overfitting, the Dropout layer randomly "discards" some neurons, forcing the network to learn more robust feature representations.

[0064] In a typical CNN architecture, the input data first passes through a series of convolutional and pooling layers, each receiving input from the previous layer and producing new feature maps as output to the next layer. Convolutional layers are responsible for feature extraction, while pooling layers reduce the size of feature maps. Alternating use of the two can effectively capture multi-scale information. Finally, after several convolutions and pooling, the feature maps are flattened and passed through one or more fully connected layers for final classification decisions.

[0065] Although the above mentions the general structure of CNN, for specific problems to be solved, the structure of CNN needs to be designed according to the characteristics of the problem, mainly including determining the number and configuration of convolutional layers, pooling layers, fully connected layers, etc.

[0066] In this application, preferably, the convolutional neural network model can include three convolutional layers, three pooling layers, and two fully connected layers in structure. The convolutional layers are used to extract local texture and structural features of the image, the pooling layers are used to reduce the image dimension and enhance the scale invariance, and the fully connected layers are responsible for the final image classification decision. The model is trained and optimized through a large amount of labeled data, and has a high medical image classification accuracy.

[0067] The design of three convolutional layers allows the model to gradually extract features from low-level to high-level. For example, each convolutional layer is equipped with a certain number of filters that can gradually extract features from low-level to high-level. The first layer may learn basic features such as edges and colors, while subsequent layers can capture more complex shapes and patterns. The design of three pooling layers helps to further compress the amount of information while retaining the most important features.

[0068] The three pooling layers mentioned above can be followed by a pooling layer after each convolutional layer, which helps to further compress the spatial size of the feature map while retaining the most important feature information. The presence of the pooling layer also increases the model's invariance to input transformations such as translation and rotation.

[0069] The design of two fully connected layers, where the last convolutional layer and pooling layer can be followed by two fully connected layers. The first fully connected layer can map all learned features to a fixed-length vector, and the second fully connected layer makes the final classification decision based on the vector. This design allows the model to perform complex nonlinear transformations in high-dimensional space, thereby improving classification accuracy.

[0070] In a preferred embodiment of the present application, a CNN architecture containing three convolutional layers, three pooling layers, and two fully connected layers is used. This design aims to effectively extract features in medical images and adapt to the unique challenges of medical images, such as high resolution and complex internal structure. Once the network structure is determined, the next step is the training process, which uses a large amount of labeled data to learn the CNN structure, adjusts the network weights through the backpropagation algorithm to minimize the error between the predicted result and the actual label. In short, first design a specific CNN structure, then use a large-scale labeled dataset in a specific field to train this structure, gradually optimize the model parameters, and finally obtain a model that can make accurate predictions for specific problems. This process ensures that the model not only learns the basic features of the input data, but also performs efficiently and accurately when faced with specific applications. The summary is as follows:

[0071] Efficient feature extraction: The design of three convolutional layers plus three pooling layers is very suitable for processing detailed features at different scales, which is particularly important for medical images, as medical images often contain rich detailed information.

[0072] Reducing Overfitting: Proper design of the number of layers can help prevent the model from overfitting the training data while maintaining sufficient expressive power to distinguish between different image categories.

[0073] Flexibility and Scalability: This architecture can be easily adjusted according to specific needs, such as adding more convolutional layers or changing the number of units in the fully connected layers, to adapt to specific application scenarios.

[0074] In summary, the design of three convolutional layers, three pooling layers, and two fully connected layers is based on a deep understanding of the characteristics of medical images and a comprehensive consideration of model performance. This design not only ensures that the model can effectively learn the key features of the input image, but also takes into account computational efficiency and generalization ability, making it an ideal choice for processing medical image classification tasks.

[0075] The training process of the CNN model includes:

[0076] 1. Data Preparation: First, a large number of labeled medical images need to be collected as the training set. These images should cover all target categories (streaming images, fluorescent images, granular fluorescent images, and other types of medical images) and be as diverse as possible to ensure that the model has good generalization ability.

[0077] In addition, in the medical field, it is often a challenge to obtain sufficient labeled data for training machine learning models. Due to the need for professional knowledge in the labeling process, and the need to properly handle data privacy and security issues, it is often difficult to obtain a large amount of high-quality training data. In the face of such situations, transfer learning provides an effective solution. Transfer learning is a machine learning method that applies a model trained on one task to a related but different task. In this way, the feature representation learned by the pre-trained model can be used to speed up the learning process of the new task and reduce the need for large amounts of labeled data. For medical image analysis, this means that we can use a deep learning model trained on a larger and more diverse image dataset (such as ImageNet) as a starting point, and then fine-tune it for specific medical image classification tasks. In summary, when faced with the problem of insufficient medical image data, transfer learning provides a powerful and flexible framework that allows the development of high-performance image classification systems even in resource-limited situations. This method not only saves a lot of time and cost, but also helps to promote the progress of medical image analysis.

[0078] 2. Initialize Weights: Initialize the weights in all convolutional layers and fully connected layers with random values.

[0079] Forward Propagation: Input an image, and after processing through each layer, output the predicted class probability distribution.

[0080] Loss Calculation: Use the cross-entropy loss function to measure the difference between the predicted result and the true label.

[0081] Backpropagation: Adjust the weights in the network according to the loss gradient to minimize the loss function.

[0082] Optimization Algorithm: Commonly used optimization algorithms such as Adam (Adaptive Gradient Algorithm) or SGD (Stochastic Gradient Descent) can be used for optimization. These algorithms can dynamically adjust the learning rate to help the model converge faster.

[0083] 3. Hyperparameter Tuning: To achieve the best performance, hyperparameters such as learning rate, batch size, convolution kernel size, etc. may need to be adjusted. This is usually done through experimental verification.

[0084] The convolutional neural network (CNN) model trained by the above scheme can identify the type of input medical paper image and classify it into four categories: flow image, fluorescence image, particle fluorescence image, and other image. The process can be implemented according to the following steps:

[0085] 1. Data Preprocessing

[0086] Before inputting the image into the already trained CNN model, data preprocessing is needed first. This step includes but is not limited to adjusting the image size to meet the model input size requirements, normalizing pixel values, etc. For specific types of medical images, additional preprocessing steps may also be required, such as grayscale, brightness adjustment, etc., to better extract features.

[0087] 2. Input to CNN Model

[0088] The preprocessed image is input into the CNN model as input. The first part of the model is a feature extractor composed of multiple convolutional layers and pooling layers. These layers can automatically learn local features in the input image and gradually build more complex and abstract feature representations through the stacking of multiple layers.

[0089] Convolutional Layer: Used to detect various patterns or features in the input image, such as edges, textures, etc.

[0090] Pooling Layer: Reduces the spatial dimension of the feature map while enhancing the model's invariance to input changes such as translation and rotation.

[0091] 3. Feature Extraction and Mapping

[0092] As the image data passes through a series of convolutional and pooling layers, the model gradually extracts key features of the image and maps them into a low-dimensional space. This process transforms the original high-dimensional image data into a feature vector that contains rich semantic information.

[0093] 4. Fully Connected Layers and Classification Decision

[0094] The feature vector output from the last pooling layer then enters the fully connected layers. The first fully connected layer is responsible for integrating all the features learned previously, while the second fully connected layer makes the final classification decision based on these features. Specifically, a Softmax activation function is usually applied after the last fully connected layer, which converts the scores for each class into probability values, determining which class the image is most likely to belong to.

[0095] 5. Output Classification Results

[0096] Based on the probability distribution provided by the Softmax layer, the class with the highest probability value is selected as the prediction result. This means that each input medical paper image will be assigned to one of the four predetermined categories: flow image, fluorescence image, granular fluorescence image, or other image. The entire process not only relies on the carefully designed architecture within the model but also benefits from the previous training using a large labeled dataset, which ensures that the model can accurately identify different types of medical images.

[0097] 6. Post-processing (Optional)

[0098] In some application scenarios, post-processing can be performed on the results output by the CNN model to further improve classification accuracy or meet specific requirements. For example, a confidence threshold can be set to filter out samples with uncertain classification, or the Non-Maximum Suppression (NMS) technique can be used to address overlapping regions.

[0099] In this way, the pre-trained CNN model can effectively identify the types of new medical paper images and correctly classify them into one of the four categories mentioned above, providing a foundation for subsequent similarity detection.

[0100] S120: Image preprocessing is performed for each of the four categories of images.

[0101] Image preprocessing is a crucial step to ensure that subsequent steps work effectively. Different types of medical images require different preprocessing strategies due to their imaging principles, application scenarios, and visual characteristics. Here are specific preprocessing methods and reasons for streaming images, fluorescent images / granular fluorescent images, and other images.

[0102] Preprocessing for streaming images includes cutting edges, removing coordinate axes, increasing resolution, and grayscale. Among them:

[0103] Cutting edges and removing coordinate axes: Streaming images often contain additional non-image information such as coordinate axes or legends, which are not helpful for understanding the image content and may interfere with the feature extraction process.

[0104] Increasing resolution: In order to capture the detailed features in the image more accurately, sometimes it is necessary to increase the resolution of the image. This is particularly important for streaming images, as these images often contain subtle structural changes, and high resolution helps accurately identify these changes.

[0105] Grayscale: Converting color images to grayscale images can reduce data dimensions while highlighting texture and shape information in images. After grayscale, the model is easier to focus on the content of the image rather than color information, which is particularly useful when analyzing streaming images.

[0106] Preprocessing for fluorescent images / granular fluorescent images includes reducing resolution, grayscale, and balancing brightness. Among them:

[0107] Reducing resolution: While increasing resolution helps capture details, for fluorescent images, too high a resolution can increase computational complexity without additional information gain. Appropriately reducing the resolution can help speed up processing while maintaining enough details for feature extraction.

[0108] Grayscale: Similar to streaming images, fluorescent images are often converted to grayscale images to simplify feature representation and focus on brightness information. This is because the main information of fluorescent images is usually expressed through brightness changes rather than color.

[0109] Balancing brightness: Fluorescence images can be affected by the shooting conditions, resulting in uneven brightness. Through the brightness balancing operation, this difference can be eliminated, making the images taken under different conditions consistent, thereby improving the accuracy of feature extraction and matching.

[0110] The preprocessing of other images includes reducing resolution, grayscale, and balancing brightness. Among them:

[0111] Reducing resolution: Similar to fluorescence images, other types of medical images can also benefit from appropriate resolution adjustment. Reducing resolution can reduce the computational burden without affecting the core information.

[0112] Grayscale: Converting color images to grayscale images helps simplify feature representation, making it easier for the model to learn useful patterns.

[0113] Balancing brightness: Images taken under different imaging devices or environmental conditions may have brightness differences. By standardizing the brightness, the impact of these external factors on the feature extraction process can be reduced, improving the consistency and reliability of the algorithm.

[0114] The above different types of images have their unique characteristics and challenges:

[0115] Flow images: Usually involve complex dynamic processes, so higher resolution is needed to capture subtle changes, and unnecessary background elements are removed to focus on the main content.

[0116] Fluorescence images and particle fluorescence images: mainly rely on brightness changes to display structures, so grayscale and brightness balancing are particularly important. In addition, considering that fluorescence images may have higher original resolution, moderately reducing resolution helps simplify processing without losing important information.

[0117] Other images: This category covers a variety of types of medical images, each with different characteristics, but they all need to go through standardized preprocessing steps to enhance feature extraction, such as grayscale and brightness adjustment.

[0118] In summary, different preprocessing strategies are adopted for different types of medical images to better adapt to their respective characteristics and challenges, thereby improving the efficiency and accuracy of subsequent feature extraction and matching steps. These preprocessing steps not only consider the physical properties of the images themselves, but also take into account the problems that may be encountered in practical applications, aiming to provide an optimized foundation for the effective execution of subsequent analysis tasks.

[0119] S130: Feature extraction is performed on the preprocessed images of different types, and the extracted features are matched.

[0120] In image similarity detection, feature extraction and matching are core steps. By extracting unique features from different types of images and finding the corresponding relationships between these features, similar images can be effectively identified. However, due to the different visual characteristics and complexity of different types of images, selecting appropriate algorithms and parameters is crucial for improving detection accuracy. Here are specific feature extraction and matching strategies for streaming images, fluorescent images / granular fluorescent images, and other types of images, along with their reasons.

[0121] 1. Streaming Images

[0122] Algorithm Selection: SIFT

[0123] Reason: Streaming images usually contain complex dynamic structures and rich detail information. SIFT (Scale-Invariant Feature Transform) algorithm performs well in handling such images due to its excellent scale and rotation invariance. It can accurately locate and describe key points, maintaining high matching accuracy even in the presence of scale changes, rotations, or partial occlusions.

[0124] Specific Operation: Use the SIFT algorithm to extract the key points of the image and calculate the descriptor of each key point. Then, based on the distance (such as Euclidean distance) between the descriptors, find the similar points and corresponding relationships between images.

[0125] 2. Fluorescent Images / Granular Fluorescent Images

[0126] Algorithm Selection: AKAZE

[0127] Reason: Fluorescent images and granular fluorescent images usually have high contrast and rich texture information, but they may also be affected by imaging conditions such as uneven brightness or noise interference. AKAZE (Accelerated-KAZE) algorithm, compared to SIFT, can provide faster speed and higher accuracy while maintaining good scale and rotation invariance, especially in handling small-scale details.

[0128] Threshold setting: Set the matching threshold to 20. A lower threshold means stricter requirements for feature point matching, which helps improve the accuracy of matching, especially suitable for fluorescence images with high analysis requirements, ensuring accurate identification of similar points even with subtle differences.

[0129] Optimized matching: RANSAC

[0130] Reason: To further improve matching quality, the RANSAC (Random Sample Consensus) algorithm is used to eliminate false matches. RANSAC finds the set of the largest number of consistent matches through iterative random sampling and consistency checks, thus obtaining more reliable matching results.

[0131] Specific steps: First, use the AKAZE algorithm to extract key feature points from the image, then apply the RANSAC algorithm to optimize the matching, and finally obtain a set of high-quality key point pairs.

[0132] 3. Other Images

[0133] Algorithm selection: AKAZE

[0134] Rationale: While other types of medical images are diverse, scale invariance and rotation invariance are still important considerations in most cases. The AKAZE algorithm is an ideal choice due to its efficiency and robustness. Compared to fluorescence images, the images here may be more diverse, thus allowing for slightly more lenient matching criteria.

[0135] Threshold setting: Set the matching threshold to 40. A higher threshold allows the algorithm to tolerate a certain degree of error when dealing with more complex image content, adapting to a wider range of application scenarios, although this may mean sacrificing some accuracy in certain situations.

[0136] Specific operation: Similar to the processing flow of fluorescence images, first use the AKAZE algorithm to extract feature points, then perform preliminary matching through a set threshold, and finally, the RANSAC algorithm can be used to further optimize the matching results as needed.

[0137] The reason for choosing different algorithms and thresholds for different types of images lies in:

[0138] Due to their complex structure and detail requirements, streaming images are better suited to use the SIFT algorithm to ensure high matching accuracy.

[0139] Fluorescence images and particle fluorescence images benefit from the efficient processing capabilities and detailed feature capture capabilities provided by the AKAZE algorithm, while the low matching threshold ensures high accuracy during analysis.

[0140] For other types of images, considering their diversity and potential complex backgrounds, the AKAZE algorithm is also applicable, but with a relatively loose matching threshold to adapt to a wider range of application scenarios without excessively affecting accuracy.

[0141] This hierarchical and targeted approach not only improves the effectiveness of feature extraction and matching, but also lays a solid foundation for subsequent similarity analysis.

[0142] S140: Determine whether the two images are similar based on the number of matched feature points and the proportion of matched points to the total number of points. If they are similar, further analyze their transformation parameters.

[0143] After feature extraction and matching, the next step is similarity analysis. The goal of this stage is to determine whether two images are similar based on the matching results and further analyze their geometric transformation relationship (such as scale, rotation, and flip). Here is a detailed description:

[0144] S141: Preliminary similarity judgment.

[0145] First, based on the feature point matching results obtained from the previous step, a preliminary similarity assessment is made. The main basis is two key indicators:

[0146] Number of matched feature points: Calculate the number of successfully matched feature points between two images. Generally, if two images have a high degree of similarity, there will be more matching points between them. As shown in Figure 1, it represents the matching points of two images. Figure 3

[0147] Proportion of matched points to total points: In addition to the number of matching points alone, the proportion of these matching points to the total number of key points detected in each image is also considered. This proportion can reflect the distribution of matching points in the entire image and their importance. For example, even if the number of matching points is large, but the proportion is low relative to the total number of feature points, it may mean that the two images are not truly similar.

[0148] By setting appropriate thresholds, it can be determined which image pairs are considered "similar". For example, if the number of matching points exceeds a certain preset value and the proportion of matching points to the total number of points is higher than a certain percentage, then the two images are considered similar.

[0149] S142: Further similarity analysis.

[0150] Once it is determined that the two images are similar, further analysis of their specific transformation parameters is needed. This includes but is not limited to the following common geometric transformations:

[0151] ​Scale Variation: Analyzing the relative size difference between two images. Even if the content is the same, the actual dimensions of the images can vary due to differences in shooting distance or device settings.

[0152] Rotation Angle: Identifying whether there is a rotational offset between one image relative to another. This is crucial for accurately understanding the relationship between images, especially in medical image analysis, as even a small angular difference can impact diagnostic results.

[0153] Flip / Mirror Transformation: Checking for horizontal or vertical flips. This is also common in practical applications, especially when dealing with images from different sources or edited through different processes.

[0154] To achieve the above analysis, these transformation parameters can be obtained by calculating the Homography Matrix between the matched point pairs. Homography transformation is a two-dimensional projection transformation that describes the mapping relationship between planes, suitable for analyzing the affine transformation (such as translation, rotation, scaling, etc.) and more complex perspective transformation between images.

[0155] In an example, assume a pair of medical images A and B are considered similar, and a series of corresponding feature point pairs are obtained through feature matching. The following operations are as follows:

[0156] Calculate the Homography Matrix H between these matched point pairs, which contains all the transformation information (including scale, rotation, and translation, etc.) from image A to image B.

[0157] According to the H matrix, specific transformation parameters can be calculated, such as:

[0158] Scale Factor: Indicates how many times image A is enlarged or reduced relative to image B.

[0159] Rotation Angle: Gives how many degrees image A is rotated relative to image B.

[0160] Translation Vector: Shows the displacement amount of image A relative to image B in the x-axis and y-axis directions.

[0161] These transformation parameters not only help verify the true similarity of two images, but also provide valuable information for subsequent possible applications, such as image alignment, stitching, or tamper detection, etc.

[0162] As shown in the figure, the similar parts in the two images are marked with black boxes. Figure 4

[0163] ​In summary, similarity analysis is not just a simple counting of the number and proportion of matching points, but also needs to explore the various transformation relationships between images in depth, so as to comprehensively and accurately evaluate the similarity degree of images. This process is crucial to ensure the accuracy and reliability of medical image similarity detection.

[0164] S150: For the detected similar image pairs, draw feature matching points, label similar regions, and then visualize the output result image and output.

[0165] After completing feature extraction, matching, and similarity analysis, in order to facilitate users to intuitively understand the detection results and provide support for further manual verification or report generation, the next step is to post-process and visualize the output of the detected similar image pairs. This process mainly includes the following steps:

[0166] S151: Draw feature matching points.

[0167] Define matching points: Based on the matching feature point pairs calculated in the previous step (found by SIFT or AKAZE algorithm), specify the specific location of each matching point pair in both images.

[0168] Draw matching points: Mark the location of these matching points on the original image with obvious markers (such as circles or crosses). Usually, different colors can be used to distinguish different types of matching points (for example, correctly matched points can be represented by green, and incorrectly matched points can be represented by red). This step helps to visually display which parts are considered as key areas of similarity.

[0169] S152: Label similar regions.

[0170] Connect matching points: For each matching point pair, draw a line between the two images to connect them, forming a so-called "matching line", as shown in Figure 5 This line not only clearly shows which points correspond to each other, but also helps observers quickly identify the correspondence between images and possible transformations (such as rotation, scaling, etc.).

[0171] Highlight similar regions: In addition to individually labeling each matching point, a series of closely related matching points can be highlighted as a whole. For example, a rectangular frame or transparent block can be drawn to highlight these similar regions. This makes it easier for users to notice local areas with high similarity.

[0172] S153: Visualize the output result image and output the matching results of each type of similar image pair to the result file.

[0173] Integration: Combine the plotted matching points, lines, and highlighted areas into a comprehensive result image. Ideally, this result image should include both input images and their corresponding annotations for direct comparison and evaluation.

[0174] Format Selection: Choose the appropriate output format based on practical needs. Common practices include:

[0175] Static Images: Save the processed images in formats like PNG, JPEG, etc., suitable for report generation or archiving.

[0176] Dynamic Interactive Interface: If implemented within a software system, design an interactive interface that allows users to zoom in on specific areas, switch display modes (e.g., show only matching points without lines), adjust transparency, and more, providing a flexible and detailed analysis experience.

[0177] Additional Information: Besides basic visual annotations, add auxiliary information alongside the images, such as matching scores, transformation parameters (e.g., scale change rate, rotation angle), confidence levels, etc., to provide users with more background knowledge and data support for decision-making.

[0178] In a specific application example, suppose that the analysis of two suspected similar medical images A and B has been completed, and several reliable matching point pairs have been determined. The following operation process is as follows:

[0179] Mark the positions of all matching points on images A and B with red circles respectively;

[0180] Connect each pair of matching points with blue lines to form lines across the two images;

[0181] For those matching point groups that gather together and show strong similarity, use a yellow semi-transparent rectangular frame to enclose them, emphasizing the importance of these areas;

[0182] Finally, display images A and B with all annotations side by side on a PDF report page, and attach a brief written explanation to explain the main findings and significance of this matching.

[0183] Through such detailed and comprehensive post-processing and visual output, not only does it greatly improve the readability and usability of the results, but also provides a powerful support tool for subsequent manual review. This is crucial for ensuring the accuracy and reliability of the medical image similarity detection system.

[0184] Moreover, when handling sensitive information such as medical images, ensuring data security and privacy protection is of utmost importance. Especially in cases where the system consists of multiple modules that need to communicate and exchange data with each other, the security of data transmission becomes a critical consideration. To this end, using encrypted channels to protect data transmission within and outside the system is an effective strategy. Here is a detailed expansion on how to implement this measure:

[0185] 1. Importance of Data Encryption

[0186] Privacy Protection: Medical images and other health-related information are highly sensitive data types, and leaks can lead to serious consequences. Data transmission through encrypted channels can effectively prevent unauthorized access.

[0187] Integrity Assurance: In addition to confidentiality, encryption also ensures that data is not tampered with or damaged during transmission, which is crucial for maintaining the accuracy of medical diagnoses and research work.

[0188] Compliance Requirements: Many countries and regions have strict laws and regulations (such as GDPR, HIPAA) on the processing of medical data, requiring appropriate technical means to protect patient information. Using encrypted channels helps meet these legal requirements.

[0189] 2. Methods to Implement Encrypted Channels

[0190] The main ways to implement encrypted channels include but are not limited to the following:

[0191] SSL / TLS Protocol: This is one of the most commonly used network layer encryption methods, which can provide end-to-end secure communication between clients and servers. All modern browsers support HTTPS, which is an HTTP protocol based on SSL / TLS, making it an ideal choice for protecting data transmission in web applications.

[0192] VPN (Virtual Private Network): For enterprise-level applications or cross-organizational data sharing scenarios, establishing a secure virtual private network connection can ensure the security of data throughout the transmission path. This method is particularly suitable for communication between different departments within a hospital or between a hospital and other medical institutions.

[0193] Message Queue and API Gateway Encryption: If the system architecture relies on a microservices architecture, messages passed between services may require additional security measures. By configuring message queues (such as RabbitMQ, Kafka) or API gateways (such as AWS API Gateway) to enable TLS encryption, communication between services can be protected.

[0194] Point-to-point encryption: For specific application scenarios, such as frequent and large data exchange between two fixed terminals, point-to-point encryption schemes can be directly implemented. This approach usually involves implementing encryption algorithms (such as AES, RSA) directly at the application level to ensure that each transmission is secure.

[0195] 3. Configuring and managing encrypted channels

[0196] Key management: Effective key management is the foundation of successful deployment of encrypted channels. Strict policies must be developed for key generation, storage, distribution, and rotation, and key usage must be audited regularly to prevent key leakage.

[0197] Certificate Authority (CA): For SSL / TLS-based solutions, digital certificates must be obtained from a trusted certificate authority. This not only enhances user trust in the system, but also simplifies the certificate verification process.

[0198] Monitoring and logging: Continuously monitor the status of encrypted channels to promptly detect and respond to any abnormal activities. At the same time, detailed logs are kept for post-analysis and tracking of potential security incidents.

[0199] 4. Application examples

[0200] Suppose in a large hospital system, multiple departments such as radiology and pathology need to share patient medical image data. To ensure the security of these high-sensitivity data during transmission, the following steps can be taken:

[0201] Establish SSL / TLS-based encrypted connections between workstations and servers in each department to ensure that each image request and transmission is completed within an encrypted channel.

[0202] For remote consultation or cross-institutional collaboration, use VPN technology to create a secure private network environment, allowing participants to efficiently view and discuss cases in a secure environment.

[0203] If the system adopts a microservice architecture, ensure that data exchange between each microservice is encrypted, for example, by configuring the API gateway to enforce the TLS protocol.

[0204] Through the above measures, not only can the ability of the medical information system to resist external attacks be significantly improved, but also the security of internal data flow can be enhanced, ultimately providing more reliable service guarantees for patients.

[0205] The present application proposes a medical image similarity detection method based on deep learning and feature matching, which has the following significant technical effects:

[0206] 1. High accuracy intelligent classification and similarity detection

[0207] Using deep learning technology to automatically classify medical images and take targeted preprocessing and feature matching strategies based on different categories of images effectively improves the accuracy of image similarity detection.

[0208] In actual tests, the overall accuracy of this method in image classification and similarity judgment can reach more than 90%, with good recognition ability.

[0209] 2. High efficiency batch processing, significantly improving detection efficiency

[0210] The system supports automatic and batch processing of large amounts of medical paper image data without human intervention.

[0211] The image processing speed can reach 20 images per minute, greatly shortening the image review time period and significantly improving the efficiency of duplicate checking and quality control.

[0212] 3. Strong robustness, adapt to complex image changes

[0213] Through systematic image preprocessing (such as scale normalization, grayscale conversion, brightness balance, etc.) and post-processing (such as feature point labeling, region visualization), effectively deal with the problems of multi-scale, rotation, noise interference, etc. in medical images.

[0214] Improve the adaptability of the algorithm to different imaging conditions and image transformations, enhance the stability and robustness of the system.

[0215] 4. Intuitive visual output, auxiliary manual review

[0216] For detected similar image pairs, the system can automatically generate visual result images containing feature matching points and corresponding region labels.

[0217] Visual information is clear and intuitive, which helps researchers or reviewers quickly locate similar content, providing strong support for the authenticity review of medical papers.

[0218] 5. Low cost, high security, protect data privacy

[0219] Compared with traditional image duplicate checking methods that rely on third-party services, this invention can be deployed on a hospital's local server, avoiding the uploading of sensitive images to external platforms.

[0220] This significantly reduces the cost of image duplicate checking for hospitals, while greatly improving the privacy protection level and security of medical data, preventing data leakage risks.

[0221] 6. Modular design, easy to extend and maintain

[0222] The system adopts a modular architecture design, and the functions of various functional modules are clear and the coupling degree is low, so that flexible deployment, function expansion and system upgrading can be carried out according to the actual needs of the hospital.

[0223] High availability and long-term maintainability are supported, and the system is suitable for popularization and application in various medical institutions.

[0224] In summary, the application not only performs well in image recognition accuracy, processing efficiency and robustness, but also takes into account data security, cost control and system scalability, and has wide application prospect and promotion value.

[0225] The embodiment of the application also provides an electronic device, including a processor, a memory and a computer program which can run on the processor, and the processor executes the program to realize the method as described above.

[0226] The embodiment of the application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method as described above.

[0227] The embodiment of the application also provides a computer product, including a computer program / instruction, and the computer program / instruction is executed by a processor to realize the method as described above.

[0228] Those skilled in the art should understand that the embodiments of the application can be provided as a method, a system or a computer program product. Therefore, the application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0229] For the convenience of description, the above device is described as various units divided by functions. Of course, the functions of each unit can be realized in the same or more software and / or hardware in the implementation of the application.

[0230] The application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a machine that implements the function described in the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus with the functionality to achieve the specified functions in one or more blocks

[0231] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 one or more processes and / or blocks Figure 1 an apparatus with the functionality to achieve the specified functions in one or more blocks

[0232] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 one or more processes and / or blocks Figure 1 an apparatus with the functionality to achieve the specified functions in one or more blocks

[0233] It is also noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0234] Those skilled in the art will appreciate that embodiments of the present application can be devised for a variety of computer-implemented processes. Accordingly, embodiments of the present application can be embodied in a variety of different forms.

[0235] The present application can be described in the general context of computer- executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0236] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0237] The above only describes the embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A medical image similarity detection method employing neural network feature matching, comprising: The trained convolutional neural network model was used to perform type recognition on the input medical paper images, classifying them into four categories: streaming images, fluorescence images, particle fluorescence images, and other images. Image preprocessing is performed on four types of images. Preprocessing for streaming images includes edge trimming, removing coordinate axes, increasing resolution, and adjusting grayscale. Preprocessing for fluorescence images / particle fluorescence images and other images includes reducing resolution, adjusting grayscale, and balancing brightness. Feature extraction is performed on the preprocessed images of different types, and the extracted features are matched. For streaming images, the SIFT algorithm is used for feature extraction and matching; for fluorescence images / particle fluorescence images and other images, the AKAZE algorithm is used. The similarity between two images is determined based on the number of matched feature points and the proportion of matched points to the total number of points. If similar, their transformation parameters are further analyzed. For detected similar image pairs, feature matching points are plotted, similar regions are labeled, and the resulting images are visualized and output. The trained convolutional neural network model structure includes three convolutional layers, three pooling layers, and two fully connected layers. The convolutional layers extract local texture and structural features of the image, the pooling layers reduce image dimensionality and enhance scale invariance, and the fully connected layers are responsible for the final image classification decision. The AKAZE algorithm is used for feature extraction and matching of fluorescence images / particle fluorescence images, with a matching threshold set to 20. The AKAZE algorithm is also used for feature extraction and matching of other images, with a matching threshold set to 40. After feature extraction and matching of fluorescence images / particle fluorescence images and other images using the AKAZE algorithm, the model further includes using a random sampling consensus algorithm to remove mismatched points.

2. The method as described in claim 1, wherein the transformation parameters include scale, rotation, and flip.

3. The method as described in claim 1, wherein when the system implementing the method consists of multiple modules, data transmission between the modules is performed through an encrypted channel.

4. An electronic device comprising a processor, a memory, and a computer program executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 3.

5. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-3.

6. A computer software product comprising a computer program / instructions that, when executed by a processor, implement the method as described in any one of claims 1-3.

Images