A biomedical image copy detection method

Abstract

The present application relates to a biomedical image copy detection method, belonging to the technical field of digital information security, which can solve the problem of low accuracy, great detection difficulty and low efficiency of the image copy detection method in the prior art when used for biomedical image copy detection. The method steps of the present application are: S1, input the path of the query image set and the path of the reference image set; S2, input the query image set and the reference image set into the self-supervised image copy detection model to generate image feature vectors; S3, use the faiss library to construct the index of the image feature vectors in the reference image set; S4, traverse and calculate the minimum Euclidean distance between the image feature vectors of the query image set and the image feature vectors in the reference image set according to the index of S3; S5, obtain the suspected copy image pair based on the minimum Euclidean distance obtained in step S4; S6, judge whether the biomedical image is a copy image according to each suspected copy image pair.

Description

Technical Field

[0001] This invention belongs to the field of digital information security technology and relates to a method for detecting copying of biomedical images. Background Technology

[0002] With the development of digital technology, people can access various information and data more conveniently and quickly. At the same time, this has also made piracy, illegal copying, and tampering with digital information easier. Therefore, the management and protection of digital content has become an urgent problem for the industry. Digital images, with their ease of storage and dissemination, are widely used, resulting in a large number of copied images on the internet. To identify the originality of digital images, image copy detection methods have emerged.

[0003] Especially in the biomedical field, experimental results and processes often require visual representation. Images provide a clear understanding of the pathological information of tissues and organs, as well as how substances act on organisms. These images offer crucial guidance for biological experiments, pharmaceutical experiments, and clinical medicine. However, due to the long experimental cycles and high costs, instances of a single image being used for multiple purposes frequently occur in the biomedical field.

[0004] However, most existing image copy detection methods are designed for natural images, and there are currently no copy detection methods specifically for biomedical images. The characteristics of natural images and biomedical images differ significantly. Natural images often have complex textures and a large number of salient semantic objects, while biomedical images are mostly composed of points and lines, do not contain identifiable objects, have smooth textures, fewer feature points, and contain many semantic objects with similar shapes and colors. Furthermore, to avoid detection, copyists often perform image coloring, rotation, flipping, scaling, deformation, size transformation, and various combinations of transformations after copying the original image, making existing image copy detection methods unsuitable for biomedical image copy detection. In addition, the existing biomedical academic images are massive in volume. The MEDLINE database alone contains more than 6.6 million biomedical journal articles published in the past five years, and the number of articles published in 2021 alone has exceeded 1.66 million. Moreover, the number of articles published each year shows a clear upward trend. Traditional image copy detection methods are time-consuming to process millions of image copies, and their detection accuracy drops significantly when dealing with a large number of biomedical images. Summary of the Invention

[0005] In view of the above problems, the present invention provides a biomedical image copy detection method, which can solve the problems of low accuracy, high detection difficulty and low efficiency of existing image copy detection methods when used for biomedical image copy detection.

[0006] This invention provides a method for detecting copies in biomedical images, the specific steps of which include:

[0007] S1. Input the path to the query image set and the path to the reference image set for biomedical images;

[0008] S2. Input the query image set and the reference image set into the trained self-supervised image copy detection model to generate image feature vectors for the query image set and the reference image set;

[0009] S3. Use the Faiss library to construct an index of image feature vectors in the reference image set;

[0010] S4. Calculate the minimum Euclidean distance between the image feature vectors of the query image set and the image feature vectors of the reference image set based on the index traversal in S3.

[0011] S5. Determine whether the minimum Euclidean distance obtained in step S4 is greater than 0. If it is greater than 0, the detection ends; if it is equal to 0, the image is regarded as a suspected copy image pair.

[0012] S6. Determine whether a biomedical image is a copy image based on each suspected copy image.

[0013] Optionally, the pre-trained self-supervised image copy detection model includes a positive example generation module, a set of data augmentation strategies, and an image coding module.

[0014] Optionally, the specific steps in step S2 for inputting the query image set and reference image set into the trained self-supervised image copy detection model to generate image feature vectors for the query image set and reference image set are as follows: input the source images from the query image set and reference image set into the trained self-supervised image copy detection model respectively; the positive example generation module of the trained self-supervised image copy detection model and a set of data augmentation strategies perform data augmentation transformation operations on the given image to generate two positive example samples, and send the two positive example samples into the image encoding module respectively to extract the real-value features of the image; after pooling the ground-value features of the image through a pooling layer, a two-dimensional feature map is obtained; the two-dimensional feature map is input into the feature representation layer, and then passes through the fully connected layer and the normalization layer of the feature representation layer to generate image feature vectors for the query image set and reference image set.

[0015] Optionally, the image encoding module includes an encoder preprocessor; the encoder preprocessor includes a 1*1 convolutional layer, a position attention module, and a 3*3 convolutional layer.

[0016] Optionally, the positional attention module includes feature pooling, feature fusion and transformation, standardization and activation, feature decomposition and processing, and feature reweighting. The positional attention module uses two one-dimensional global pooling operations to aggregate the input 1*1 convolutional layer output feature maps along the vertical and horizontal directions into two feature maps in different directions; and encodes the two feature maps in different directions with embedded information in specific directions into a weighted attention feature map.

[0017] Optionally, the feature pooling operation includes vertical average pooling and horizontal average pooling. Vertical average pooling performs average pooling on the input feature map along the vertical direction, compressing the feature map into feature lines in the width dimension. Horizontal average pooling performs average pooling on the input feature map along the horizontal direction, compressing the feature map into feature lines in the height dimension.

[0018] Optionally, feature fusion and transformation are used to concatenate feature lines in the width dimension and feature lines in the height dimension along the channel dimension, and then perform feature fusion through a 2D convolutional layer to obtain global information.

[0019] Optionally, standardization and activation use batch standardization to standardize the global information of the convolution to obtain standardized global information, and then use a nonlinear activation function to increase the nonlinear expressive power of the standardized global information.

[0020] Optionally, the feature decomposition and processing includes decomposition, transformation, and activation operations; wherein, the decomposition operation decomposes the standardized global information into two parts of features, corresponding to the original vertical and horizontal information respectively; the transformation operation performs a convolution process on each part of the decomposed features; and the activation operation generates weight coefficients through an activation function.

[0021] Optionally, feature reweighting uses weight coefficients obtained from the activation operation to weight the original input feature map.

[0022] Compared with the prior art, the present invention can achieve at least the following beneficial effects:

[0023] (1) The copy detection method of the present invention can detect various transformations that are common when biomedical images are copied and reused, such as: changing the color of the original image, changing the geometric shape of the image, scaling the image, and mixed transformations that combine multiple single transformations.

[0024] (2) The copy detection method of the present invention converts the image to be detected into a shorter code and uses dense vector matching and other methods to construct an index structure for the existing image code, which can actively and in batches complete the detection of a large number of images and greatly improve the speed of image copy detection.

[0025] (3) The copy detection method of the present invention can accurately detect images that have undergone coloring, rotation, flipping, scaling, deformation, size transformation, and various combinations of transformations. Attached Figure Description

[0026] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0027] Figure 1 This is a schematic diagram of the pre-trained self-supervised image copy detection model of the present invention;

[0028] Figure 2 This is the image encoding module of the pre-trained self-supervised image copy detection model of the present invention;

[0029] Figure 3 This is a schematic diagram of the position attention module of the present invention;

[0030] Figure 4 This is a flowchart of the biomedical image copy detection method of the present invention. Detailed Implementation

[0031] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0032] A specific embodiment of the present invention, such as Figure 1-4 A method for detecting copies in biomedical images is disclosed, the specific steps of which include:

[0033] S1. Input the path to the query image set and the path to the reference image set for biomedical images;

[0034] S2. Input the query image set and the reference image set into the trained self-supervised image copy detection model to generate image feature vectors for the query image set and the reference image set;

[0035] See Figure 1 The pre-trained self-supervised image copy detection model includes a positive example generation module, a set of data augmentation strategies, and an image coding module.

[0036] In practice, source images from the query image set and the reference image set are input into the pre-trained self-supervised image copy detection model. The positive example generation module of the pre-trained self-supervised image copy detection model and a set of data augmentation strategies perform data augmentation transformation operations on the given image to generate two positive examples. The two positive examples are then fed into the image encoding module to extract the real-valued features of the image. The ground-valued features of the image are pooled by the Gem pooling layer to obtain a two-dimensional feature map. The two-dimensional feature map is input into the feature representation layer, and then passes through the fully connected layer FC and the normalization layer L2 in sequence to generate image feature vectors for the query image set and the reference image set. The fully connected layer FC transforms the two-dimensional feature map output by the Gem pooling layer into a one-dimensional feature vector, which is then normalized by the L2 normalization layer and then similarity calculation is performed to generate image feature vectors for the query image set and the reference image set.

[0037] Furthermore, the positive example generation module includes a reinforcement learning network. When an image is input into the reinforcement learning network, two positive examples are obtained through the positive example generation module and a set of data augmentation strategies. These two positive examples are then fed into the image encoding module for training. The micro-average precision μAP measured on the validation set is used as a reward on the reinforcement learning network.

[0038] Furthermore, the validation set is used for validation during the training of the self-supervised image copy detection model. For example, the training set is input into the positive example generation module, undergoes data augmentation, and then fed into the image encoding module. Validation is then performed on the validation set to obtain the micro-average precision (μAP). This μAP metric is used as a reward on the reinforcement learning network. The training and validation sets are obtained from the dataset used to train the self-supervised image copy detection model.

[0039] Furthermore, data augmentation transformation operations include image rotation and / or color transformation, etc.

[0040] Furthermore, sub-strategies are searched within the data augmentation strategy space as positive example generation methods for the source image. The data augmentation transformation space includes various data augmentation transformations for biomedical image copy detection, preferably including color transformations and geometric transformations. Each augmentation strategy consists of multiple sub-strategies, and each sub-strategy consists of two data augmentation transformations, where each data augmentation transformation includes the operation magnitude and the probability of implementation. One search generates one augmentation strategy, and two different sub-strategies are randomly selected from the multiple sub-strategies of the augmentation strategy and applied to a source image to generate a positive example image of the source image. This positive example image is trained on a self-supervised copy model structure, and the metric uAP measured on the validation set is used as a reward on the reinforcement learning network. The method for generating positive example images in this invention not only ensures the visibility of image information but also learns the inherent structural consistency information of the image.

[0041] Furthermore, similarity is calculated using cosine similarity to measure the similarity between features. The similarity is evaluated by measuring the cosine of the angle between the feature vectors of the query image set and the reference image set. The closer the directions of the two vectors are, the higher their cosine similarity.

[0042] Further, see Figure 2-3 The encoder preprocessor Encoder_pre of the image encoding module includes a 1*1 convolutional layer, a positional attention module, and a 3*3 convolutional layer;

[0043] The positional attention module includes feature pooling, feature fusion and transformation, standardization and activation, feature decomposition and processing, and feature reweighting. The positional attention module uses two one-dimensional global pooling operations to aggregate the input 1*1 convolutional layer output feature maps along the vertical and horizontal directions into two feature maps in different directions. The two feature maps with embedded information in specific directions are encoded into weighted attention feature maps.

[0044] Furthermore, the feature pooling operations include vertical average pooling (X Avg Pool) and horizontal average pooling (Y Avg Pool). Vertical average pooling (X Avg Pool) performs average pooling on the input feature map along the vertical direction (height), compressing the feature map into feature lines in the width dimension, which helps to capture horizontal global information. Horizontal average pooling (Y Avg Pool) performs average pooling on the input feature map along the horizontal direction (width), compressing the feature map into feature lines in the height dimension, which helps to capture vertical global information.

[0045] Furthermore, the feature fusion and transformation Concat+Conv2d is used to concatenate the average pooling results of the two directions (i.e., feature lines in the width dimension and feature lines in the height dimension) along the channel dimension, and then perform feature fusion through a 2D convolutional layer to obtain global information that integrates different directions.

[0046] Furthermore, BatchNorm+Non-linear standardization uses batch standardization to standardize the results of convolution (i.e., global information) to obtain standardized global information, and then uses a non-linear activation function (such as ReLU) to increase the non-linear expressive power of the standardized global information.

[0047] Furthermore, the feature decomposition and processing includes the decomposition operation Split, the transformation operation Conv2d, and the activation operation Sigmoid. The Split operation decomposes the standardized global information into two parts, corresponding to the original vertical and horizontal information, respectively. The Conv2d operation performs a convolution process on each decomposed feature part to further refine the features. The Sigmoid operation generates weight coefficients through the Sigmoid activation function; these weights are used to highlight important features and suppress unimportant features.

[0048] Furthermore, the feature reweighting (Re-weight) uses weight coefficients obtained from the sigmoid activation operation to weight the original input feature map (element-wise multiplication), which enhances the attention of the trained self-supervised image copy detection model to important spatial locations.

[0049] The position attention module of this invention can adaptively calculate previously extracted features, thereby enhancing the ability to extract image features.

[0050] S3. Use the Faiss library to construct an index of image feature vectors in the reference image set;

[0051] Specifically, an index of image feature vectors in a reference image set is constructed using the open-source method of the Faiss library. During the search, a brute-force search index based on the normalized L2 distance (i.e., Euclidean distance) is used to achieve accurate nearest neighbor search. The index of this invention can quickly retrieve image feature vectors, facilitate the calculation of Euclidean distance, and quickly find image pairs with an Euclidean distance of 0.

[0052] S4. Calculate the minimum Euclidean distance between the image feature vectors of the query image set and the image feature vectors of the reference image set based on the index traversal in S3.

[0053] The Euclidean distance calculation method is as follows:

[0054]

[0055] Where D(.) represents the Euclidean distance between two eigenvectors; A and B represent eigenvectors, each of which is a point in d-dimensional space; A i and B i Let represent the coordinates of eigenvectors A and B in the i-th dimension, respectively; d represents the total dimension of the space.

[0056] Based on the Euclidean distance between all two feature vectors, obtain the minimum Euclidean distance between the image feature vectors of the query image set and the image feature vectors of the reference image set.

[0057] S5. Determine whether the minimum Euclidean distance obtained in step S4 is greater than 0. If it is greater than 0, the detection ends; if it is equal to 0, the image is regarded as a suspected copy image pair.

[0058] Specifically, steps S4 and S5 are the process of finding images in the query image set and the reference image set with a minimum Euclidean distance of 0. The specific steps are as follows: obtain the array index (subscript) of the image feature quantity in the reference image set in S3, and establish a mapping between the array index and the image path; obtain the minimum Euclidean distance in step S4 according to the mapping, where the index is the array index corresponding to the feature vector; find the corresponding image path according to the mapping between the array index and the image path; obtain the image in the query image set and the reference image set with a minimum Euclidean distance of 0 according to the image path, and take the image with a minimum Euclidean distance of 0 as a suspected copy image pair.

[0059] S6. Determine whether a biomedical image is a copy image based on each suspected copy image.

[0060] To illustrate the effectiveness of the method proposed in this invention, a specific embodiment is provided below to describe the above technical solution in detail. In a biomedical image copyright inspection system, it is necessary to confirm whether the biomedical images uploaded to the platform already exist in the reference database to avoid copyright infringement. This system can help content creators protect their works from unauthorized copying or reuse. The copyright inspection system uses the biomedical image copy detection method, and the specific implementation steps are as follows:

[0061] S1: Input path;

[0062] Input: The system administrator inputs the folder path containing the images to be detected (query image set) and the folder path containing images with known copyright (reference image set).

[0063] S2: Image feature extraction;

[0064] Processing: The system extracts feature vectors for each image in both the query and reference sets by invoking a self-supervised image copy detection model. This model has been trained to effectively extract features that express the content of images.

[0065] S3: Construct the FAISS index;

[0066] Index building: Using the FAISS library, the system creates an index for the image feature vectors extracted from the reference image set. This step is to enable rapid searching and matching of feature vectors of query images within the reference set.

[0067] S4: Calculate Euclidean distance;

[0068] Search and comparison: The system iterates through each image feature vector in the query image set and uses the FAISS index to quickly calculate its Euclidean distance with all image feature vectors in the reference image set.

[0069] S5: Detect copied image pairs;

[0070] Comparison results: The system checks the calculated Euclidean distance. If any pair of images has a Euclidean distance of 0, it means that the two images are completely identical at the feature level, and are very likely to be copies.

[0071] Output: Mark all image pairs with a Euclidean distance of 0 as suspected copy image pairs.

[0072] S6: Record suspected copied image pairs;

[0073] Log: The system saves information on all suspected copied image pairs in a text file, including the image filenames and paths. This file can be used for further review and action.

[0074] The system administrator checked the log files, further analyzed these suspected copied image pairs, and took measures to protect copyright if necessary.

[0075] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting copies in biomedical images, characterized in that, The specific steps include: S1. Input the path to the query image set and the path to the reference image set for biomedical images; S2. Input the query image set and the reference image set into the trained self-supervised image copy detection model to generate image feature vectors for the query image set and the reference image set; S3. Use the Faiss library to construct an index of image feature vectors in the reference image set; S4. Calculate the minimum Euclidean distance between the image feature vectors of the query image set and the image feature vectors of the reference image set based on the index traversal in S3. S5. Determine whether the minimum Euclidean distance obtained in step S4 is greater than 0. If it is greater than 0, the detection ends; if it is equal to 0, the biomedical image is regarded as a suspected copy image pair. S6. Determine whether a biomedical image is a copy image based on each suspected copy image pair; The pre-trained self-supervised image copy detection model includes a positive example generation module, a set of data augmentation strategies, and an image coding module. The specific steps in step S2 for inputting the query image set and reference image set into the pre-trained self-supervised image copy detection model to generate image feature vectors for the query image set and reference image set are as follows: The source images from the query image set and reference image set are respectively input into the pre-trained self-supervised image copy detection model; the positive example generation module of the pre-trained self-supervised image copy detection model and a set of data augmentation strategies perform data augmentation transformation operations on the given image to generate two positive example samples; the two positive example samples are respectively sent into the image encoding module to extract the real-valued features of the image; the real-valued features of the image are pooled through a pooling layer to obtain a two-dimensional feature map; the two-dimensional feature map is input into the feature representation layer, and sequentially passes through the fully connected layer and normalization layer of the feature representation layer to generate image feature vectors for the query image set and reference image set; The image encoding module includes an encoder preprocessor; the encoder preprocessor includes a 1*1 convolutional layer, a position attention module, and a 3*3 convolutional layer; The positional attention module includes feature pooling, feature fusion and transformation, standardization and activation, feature decomposition and processing, and feature reweighting. The positional attention module uses two one-dimensional global pooling operations to aggregate the input 1*1 convolutional layer output feature maps along the vertical and horizontal directions into two feature maps in different directions. The two feature maps with embedded information in specific directions are encoded into weighted attention feature maps.

2. The copy detection method according to claim 1, characterized in that, Feature pooling operations include vertical average pooling and horizontal average pooling. Vertical average pooling averages the input feature map along the vertical direction, compressing the feature map into feature lines in the width dimension. Horizontal average pooling averages the input feature map along the horizontal direction, compressing the feature map into feature lines in the height dimension.

3. The copy detection method according to claim 2, characterized in that, Feature fusion and transformation are used to concatenate feature lines in the width and height dimensions along the channel dimension, and then perform feature fusion through a 2D convolutional layer to obtain global information.

4. The copy detection method according to claim 3, characterized in that, Standardization and activation are performed by batch standardization to standardize the global information of the convolution, and then a nonlinear activation function is used to increase the nonlinear expressive power of the standardized global information.

5. The copy detection method according to claim 4, characterized in that, Feature decomposition and processing includes decomposition, transformation, and activation operations. The decomposition operation further decomposes the standardized global information into two parts, corresponding to the original vertical and horizontal information, respectively. The transformation operation performs a convolution process on each decomposed feature. The activation operation generates weight coefficients through an activation function.

6. The copy detection method according to claim 5, characterized in that, Feature reweighting uses weight coefficients obtained from activation operations to weight the original input feature map.

Images