Biomedical image-based academic misconduct detection method and system

Abstract

The present application belongs to the field of computer vision, and particularly relates to a biomedical image-based academic misconduct detection method and system, aiming to solve the problem that the existing technology is not comprehensive enough in detecting picture tampering misconduct in the biomedical field, affecting the detection effect. The method comprises: obtaining a target picture to be detected; using a preset contour extraction algorithm to perform image segmentation on the target picture to obtain a plurality of sub-picture sets; based on a pre-trained biomedical classification model, dividing the plurality of sub-pictures into a plurality of sub-picture sets; performing reuse detection on the same sub-pictures in each sub-picture set to obtain a first detection result; based on a pre-trained multi-modal anomaly detection model, performing anomaly detection on the plurality of sub-pictures to obtain a second detection result; determining a marking area and outputting the detection result. The method can effectively improve the detection effect of biomedical pictures.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method and system for detecting academic misconduct based on biomedical images. Background Technology

[0002] With the widespread use of the internet and the popularity of various image editing software, people can easily edit and publish images. However, this makes the authenticity of images very easy to compromise, and maliciously altered images are widely disseminated in today's digital media age, disrupting normal social order and causing serious negative impacts. In particular, image manipulation in academic research has become increasingly prevalent in recent years, especially in biomedical papers, causing adverse effects.

[0003] Image tampering misconduct refers to the act of artificially modifying original image data through technical means. Its main types include: image reuse, which involves completely or partially reusing other images or using fragments of the same image, possibly through direct copying or image editing techniques such as scaling, rotation, and brightness adjustment; splicing and combining, which involves directly or after scaling and rotating fragments of other images and then splicing them onto a target image to create a new image; and deletion, which involves removing certain fragments from an image to achieve the purpose of tampering. Currently, many image tampering techniques based on traditional and deep learning methods are applied to natural images, and rarely used for tampering detection in the biomedical field. Some methods used in the biomedical field only detect duplicate images shared between them, without detecting tampering of the image itself. These methods only provide a partial detection of certain types of image tampering misconduct, lacking comprehensiveness and systematicity, which affects the detection results. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, namely, the inadequacy of existing technologies in detecting image tampering misconduct in the biomedical field, which affects detection effectiveness, the first aspect of this invention proposes a method for detecting academic misconduct based on biomedical images, comprising:

[0005] Obtain the target image to be detected;

[0006] Using a preset contour extraction algorithm, the target image is segmented to obtain multiple sub-images. The contour extraction algorithm converts the target image into a binary image and searches for a continuous set of points in the binary image to determine the contour boundary.

[0007] Based on a pre-trained biomedical classification model, fine-grained classification is performed on multiple sub-images, and based on the results of the fine-grained classification, the multiple sub-images are divided into multiple sub-image sets, each of which includes at least one type of sub-image.

[0008] Based on the multiple sub-image sets, the feature point reuse detection rules corresponding to each sub-image set are determined, and the reuse detection of similar sub-images within each sub-image set is performed according to the feature point reuse detection rules to obtain the first detection result.

[0009] Based on a pre-trained multimodal anomaly detection model, anomaly detection is performed on multiple sub-images sequentially to obtain a second detection result; the multimodal anomaly detection model includes a dual-branch structure for anomaly detection and for feature point reuse detection, the multimodal anomaly detection is used to determine the copied region and splicing deletion region of the sub-image, and the feature point reuse detection is used to determine the reuse region of the sub-image;

[0010] Based on the first detection result and the second detection result, the marked region in the target image is determined, and the detection result of the target image is output based on the marked region.

[0011] In some preferred embodiments, the training process of the biomedical classification model includes:

[0012] Acquire biomedical image data of various categories;

[0013] Category labels are added to the biomedical image data, and data augmentation techniques are used to supplement the biomedical image data to construct a dataset;

[0014] Based on the ResNet50 architecture and the preset number of classes, the output dimension of the fully connected layer of the initial ResNet model is modified, and pre-trained weights are loaded to accelerate convergence. The optimizer and cross-entropy loss function are set.

[0015] Forward and backward propagation are performed using the training set, and the parameters of the initial ResNet model are iteratively adjusted using the dataset and the optimizer until the cross-entropy loss function converges.

[0016] In some preferred embodiments, the step of determining the feature point reuse detection rule corresponding to each of the multiple sub-image sets, and performing reuse detection on similar sub-image images within each sub-image set according to the feature point reuse detection rule, includes:

[0017] Determine the image features corresponding to each sub-image set;

[0018] Based on the image features and the preset detection rules, determine multiple feature point detection methods for each sub-image set;

[0019] The first detection result is obtained by performing weighted fusion detection on each sub-image set using multiple feature point detection methods corresponding to each sub-image set.

[0020] In some preferred embodiments, the weighted fusion detection of each sub-image set according to multiple feature point detection methods corresponding to each sub-image set satisfies:

[0021] ;

[0022] In the formula, The set of matching points after fusion. The first set of matching points between two images represents the combination of the first matching points of the first image and the second image. For the first of the two images There are n matching points, where n is the number of feature point detection methods. For the first The set of matching points corresponding to each detection method, each detection method has Matching points, Indicates corresponding to The weighting coefficients satisfy:

[0023] .

[0024] In some preferred embodiments, the weighting coefficient The weight coefficients are obtained through data learning based on a preset sample set. The learning process includes:

[0025] Initialize weight coefficients ;

[0026] A weighted fusion calculation is performed on each pair of images in the preset sample set to obtain a weighted matching point set;

[0027] Calculate the overlap score between the weighted matching point set and the corresponding matching point set in the preset sample set;

[0028] With the goal of maximizing the overlap score, the initialized weight coefficients are iteratively updated. This continues until the overlap score is less than a preset threshold, and / or the current iteration number reaches the preset iteration number.

[0029] In some preferred embodiments, the formula for calculating the overlap score is:

[0030] ;

[0031] In the formula, The overlap score is calculated as follows: For recall rate, Used to indicate the proportion of the number of weighted matching points that overlap with the total number of actual matching points. For accuracy, Used to indicate the proportion of the number of points in the weighted matching point set that overlap with the actual matching points out of the total number of weighted matching points.

[0032] In some preferred embodiments, the pre-trained multimodal anomaly detection model sequentially detects multiple sub-images to obtain a second detection result, including:

[0033] The images of each sub-image are input into the multimodal anomaly detection model in sequence;

[0034] The multimodal anomaly detection branch is used to perform multimodal anomaly detection on the sub-image to obtain anomaly detection results. The multimodal anomaly detection branch includes multiple sub-models, and each sub-model corresponds to the image category of each sub-image set.

[0035] The feature point reuse detection branch performs feature point reuse detection on the sub-image to obtain reuse detection results. The feature point reuse detection branch divides the sub-image into multiple blocks, and sequentially determines the feature point matching results of any current block with other blocks to obtain a fused set of matching points as the reuse detection results.

[0036] Based on the anomaly detection result and the reuse detection result, the second detection result is determined. The second detection result includes a binarized matrix obtained through multimodal anomaly detection and a set of matching points obtained through feature point reuse detection.

[0037] In some preferred embodiments, the marked region includes a marked anomaly region, which includes a mark for the reuse detection result between two sub-images. The marking process for the reuse detection result between the two sub-images includes:

[0038] Using a pre-defined density clustering algorithm, density clustering is performed on two sub-images to obtain clusters. and Each cluster contains several matching points and bounding box parameters;

[0039] Within each cluster, a filter is performed to remove rectangles that do not meet the preset filter rules, and the filtered clusters are marked with a preset rectangle color so that the rectangle colors of corresponding matching point clusters in the two sub-images are the same.

[0040] Arrange the rectangles in each cluster from largest to smallest, and then sort the clusters sequentially. Each rectangle in the data corresponds to a cluster. Each bounding box is compared individually. If the bounding box contains a matching point, then the current cluster is considered. and clusters All rectangles are marked with the same color until all rectangles are marked.

[0041] Clustering in sequence Each rectangle in the data corresponds to a cluster. Each bounding box is compared individually. If the bounding box contains a matching point, then the current cluster is considered. and clusters All rectangles are marked with the same color until all rectangles are marked, then the clusters are removed. and clusters The unmarked rectangle in the middle.

[0042] In some preferred embodiments, the method further includes:

[0043] Based on the marked area, the type of image tampering is determined, specifically including:

[0044] Based on the marked regions corresponding to the sub-images after anomaly detection by the multimodal anomaly detection model, a first set of rectangular boxes is determined;

[0045] Based on the marked regions corresponding to the sub-images after feature point reuse detection by the multimodal anomaly detection model, a second set of rectangular boxes is determined;

[0046] Determine whether the areas of the first set of rectangles and the second set of rectangles are greater than a preset threshold.

[0047] If so, then the image tampering type is determined to be a reuse type;

[0048] If not, then the image tampering type is determined to be a splicing deletion type.

[0049] In a second aspect, the present invention also proposes an academic misconduct detection system based on biomedical images, the system comprising:

[0050] The data acquisition module is configured to acquire the target image to be detected.

[0051] The image segmentation module is configured to segment the target image using a preset contour extraction algorithm to obtain multiple sub-images. The contour extraction algorithm converts the target image into a binary image and searches for a continuous set of points in the binary image to determine the contour boundary.

[0052] The image classification module is configured to perform fine-grained classification on multiple sub-images based on a pre-trained biomedical classification model, and divide the multiple sub-images into multiple sub-image sets based on the results of the fine-grained classification, wherein each sub-image set includes at least one class of sub-images.

[0053] The reuse detection module is configured to determine the feature point reuse detection rules corresponding to each of the multiple sub-image sets, and perform reuse detection on the same type of sub-images within each sub-image set according to the feature point reuse detection rules to obtain a first detection result;

[0054] An anomaly detection module is configured to perform anomaly detection on multiple sub-images sequentially based on a pre-trained multimodal anomaly detection model to obtain a second detection result. The multimodal anomaly detection model includes a dual-branch structure for anomaly detection and for feature point reuse detection. The multimodal anomaly detection is used to determine the copied region and spliced ​​deletion region of the sub-image, and the feature point reuse detection is used to determine the reuse region of the sub-image.

[0055] The marker output module is configured to determine the marker region in the target image based on the first detection result and the second detection result, and output the detection result of the target image based on the marker region.

[0056] The beneficial effects of this invention are:

[0057] (1) The present invention proposes a solution for detecting academic misconduct in biomedical images, which can perform batch detection on a set of images, including single image anomaly (reuse, splicing and deletion) detection and multi-image reuse detection. It can detect various academic misconduct behaviors more comprehensively and systematically, and effectively solves the technical problem that the existing technology has poor effect due to the single and one-sided detection method.

[0058] (2) The present invention proposes to classify biomedical sub-images in a fine-grained manner and apply them to the detection of image anomalies and reuse. Specifically, the classification can refine the detection method, improve the accuracy of fine-classified image detection, and reduce unnecessary detection between unrelated images, thereby effectively improving the overall performance of the solution.

[0059] (3) The present invention proposes a multimodal anomaly feature representation method, which uses large model technology to learn academic misconduct in images of multiple fine-grained categories, and uses the multimodal few-sample anomaly feature detection mode to learn the anomaly feature representation of fine-grained classification images based on biomedical sub-images, while considering both visual and text features.

[0060] (4) This invention proposes a biomedical image reuse detection method that integrates multiple feature point detection methods, including multi-image reuse detection based on biomedical fine-classification images and single-image reuse detection in image anomaly detection. Simultaneously, single-image reuse detection and image anomaly detection are combined in a dual-branch approach to determine the type of image misconduct (reuse or splicing deletion). Furthermore, the feature point detection method can be trained and adjusted based on fine-classification image data, thereby enhancing the biomedical image reuse detection capability in multiple dimensions. Attached Figure Description

[0061] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0062] Figure 1 This is a flowchart illustrating a method for detecting academic misconduct in biomedical images, as proposed in an embodiment of the present invention.

[0063] Figure 2 This is a schematic diagram of the processing procedure for a feature point detection method proposed in an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of the processing procedure of a multimodal anomaly detection model proposed in an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of a feature point reuse detection process proposed in an embodiment of the present invention;

[0066] Figure 5 This is a schematic diagram of the structure of a computer system proposed in an embodiment of the present invention. Detailed Implementation

[0067] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0068] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0069] To facilitate a better understanding of the technical solutions in the embodiments of the present invention, the relevant technologies involved in the present invention will first be introduced:

[0070] Image tampering techniques can be broadly categorized into traditional feature extraction-based detection techniques and deep learning network-based detection techniques. Feature point-based methods are a crucial aspect of traditional detection techniques; SIFT is one of the most classic and commonly used feature point extraction techniques. Since 2016, deep learning-based image tampering detection techniques have developed rapidly, enabling end-to-end learning and detection of various tampering issues inherent in the image itself. ManTra-Net, released in 2019, uses self-supervised learning to learn tampering features from 385 tampering types, treating tampering localization as an anomaly detection problem. Its training and testing data include commonly used tampering datasets such as NIST16 and CASIA. Tampering with biomedical images can be considered an image anomaly detection problem; however, obtaining truly tampered images is difficult, and the number is limited, hindering large-scale training. In 2024, MVFA (Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images) was proposed. This lightweight, multi-level adaptation and comparison framework transfers the powerful generalization capabilities of large-scale visual-language pre-trained models (CLIPs) to medical image anomaly detection tasks, improving the model's generalization ability across different medical image modalities and anatomical regions. The core idea of ​​this framework is to fine-tune the pre-trained visual encoder through multi-level feature adapters, enabling it to better identify anomalies in medical images. Specifically, the Multi-Level Feature Adapter (MVFA) method progressively enhances visual features at different levels by inserting multiple residual adapters into the visual encoder of the CLIP. These adapters are guided by a multi-level pixel-level visual-language feature alignment loss function, shifting the model's attention from the semantics of objects in natural images to anomaly detection in medical images. Visual-language feature alignment involves generating textual features using textual cues (such as "normal" and "abnormal") and aligning them with adapted visual features to optimize the model for better anomaly identification. During testing, multi-level anomaly scoring maps are generated by comparing the adapted visual features with textual features (and optional reference image features) for image-level (classification) and pixel-level (segmentation) anomaly detection. MVFA is the first attempt to adapt a pre-trained visual-language model to zero-shot and few-shot scenarios for medical image anomaly detection, demonstrating excellent performance in anomaly classification (AC) and anomaly segmentation (AS) tasks. This provides a novel method and approach for anomaly detection in medical images with limited data.

[0071] Regarding image reuse, feature point detection and matching can be performed. If a feature point match is found, the image is considered reused. This part involves techniques from the field of image registration. This detection technique typically consists of two parts: feature point extraction and feature point matching. With technological advancements, some deep learning methods in recent years no longer distinguish between feature point extraction and matching, but treat them as a whole, truly achieving end-to-end matching. The traditional classic feature point detection algorithm SIFT (Scale-Invariant Feature Transform) includes feature points and feature point descriptions. Its basic principle is to determine extreme points in a series of images at different scales by comparing them with surrounding pixels, thereby identifying keypoints (feature points). Each keypoint is assigned one or more orientations, and finally, a descriptor is generated for each keypoint to describe the local image region surrounding it. The descriptor is typically a 128-dimensional vector. The advantage of SIFT is its ability to detect and describe keypoints in images under different scales, rotations, and lighting conditions. Its disadvantages include high computational complexity and limited ability to detect smooth texture regions.

[0072] Superpoint, proposed in 2018, is a classic image feature point algorithm based on deep learning. It's a fully convolutional neural network (CNN) with an architecture similar to VGG, and employs a self-supervised network framework for model training. This allows it to simultaneously extract feature points and descriptors with pixel-level precision. Specifically, it first trains a basic interest point detector on a synthetic dataset (Synthetic Shapes, containing simple geometric shapes such as triangles, quadrilaterals, and lines). Then, it enhances the synthetic dataset images using Homographic Adaptation (multiple transformations such as translation, scaling, and rotation on the input image, similar to data augmentation) to improve the detector's adaptability to different viewpoints and scales, further enhancing its performance on real images. Results show that compared to traditional features, Superpoint generates denser and more accurate matching points and exhibits strong robustness to changes in lighting and viewpoint. A drawback is its poor support for rotation invariance; when the image rotation angle is too large (90 degrees or more), the number of feature points detected by Superpoint significantly decreases.

[0073] Proposed in 2024, xFeat differs from previous deep learning methods by no longer distinguishing between feature point extraction and matching. Instead, it employs end-to-end learning and matching. It's a lightweight and accurate CNN (Convolutional Neural Network) architecture that meets the demands of fast and robust algorithms, enabling efficient operation on resource-constrained devices. Simply put, the main reason for network latency is high resolution. xFeat's network structure rapidly reduces image resolution in the early stages through convolutional operations, decreasing computation. Simultaneously, it gradually increases the number of channels as resolution decreases, significantly reducing computation while maintaining comparable performance. xFeat's training is based on pixel-level supervised learning, using synthetic distorted images from the Megadepth and COCO datasets (two image datasets) for mixed training. During training, it focuses on low-level image structures, making it robust to lower-quality, smooth-textured images and achieving end-to-end semi-dense matching.

[0074] The above three methods are all feature point extraction algorithms. Early matching can employ methods like nearest neighbor matching, exhaustively searching for the nearest matching feature point. However, this method has high computational complexity for matching high-dimensional feature points and requires complex post-processing to remove outliers. With the development of deep learning technology, deep matchers have made significant progress by simultaneously considering two images to jointly match sparse points and reject outliers. LightGlue is a novel deep matching network using a Transformer architecture, containing multiple layers, each consisting of self-attention units and cross-attention units. These units update the confidence representation of each feature point by aggregating information within the images. During the training process, matching prediction is performed, predicting a matching matrix after each layer of the above architecture. This combines a similarity score and a matchability score. The similarity score measures whether two feature points are visually similar, i.e., whether they might be projections of the same 3D point in the two images. The matchability score evaluates whether a feature point has a corresponding matching point in another image. These two scores are combined to generate the final matching result. By combining these two scores, LightGlue can predict matching points more accurately while reducing the impact of false matches and outliers. This design combines traditional nearest neighbor matching with outlier filtering, achieved through deep learning, making it more efficient and robust.

[0075] se2-loftr addresses the common weakness of many deep learning methods in supporting image rotation invariance. This method replaces ordinary CNNs with steerable convolutional networks to extract features, significantly improving robustness to rotational changes without compromising performance in matching under normal lighting and viewpoint conditions. Specifically, a steerable CNN is a special type of convolutional neural network where both input and output are normal image representations, with intermediate layers allowing rotational representations. At each layer, the transformation of the feature map is consistent with the transformation of the input image. This allows the trained network to maintain high performance for ordinary image matching tasks while preserving rotation invariance.

[0076] The above provides a detailed introduction to some image detection techniques that may be involved in the patent. Besides the methods mentioned above, there are many other detection methods such as disk, aliked, dog_hardnet, r2d2, SuperGLUE, etc., which can all be used for feature point detection and matching in images.

[0077] Specifically, please refer to Figure 1 The first embodiment of the present invention provides a method for detecting academic misconduct based on biomedical images, the method comprising:

[0078] Step S10: Obtain the target image to be detected;

[0079] Step S20: Using a preset contour extraction algorithm, the target image is segmented to obtain multiple sub-images. The contour extraction algorithm converts the target image into a binary image and searches for a continuous set of points in the binary image to determine the contour boundary.

[0080] Step S30: Based on the pre-trained biomedical classification model, perform fine-grained classification on the multiple sub-images, and divide the multiple sub-images into multiple sub-image sets according to the results of the fine-grained classification, with each sub-image set including at least one type of sub-image;

[0081] Step S40: Based on the multiple sub-image sets, determine the feature point reuse detection rules corresponding to each sub-image set, and perform reuse detection on the same type of sub-images within each sub-image set according to the feature point reuse detection rules to obtain the first detection result;

[0082] Step S50: Based on the pre-trained multimodal anomaly detection model, anomaly detection is performed on multiple sub-images sequentially to obtain a second detection result; the multimodal anomaly detection model includes a dual-branch structure for anomaly detection and for feature point reuse detection, the multimodal anomaly detection is used to determine the copied region and splicing deletion region of the sub-image, and the feature point reuse detection is used to determine the reuse region of the sub-image;

[0083] Step S60: Based on the first detection result and the second detection result, determine the marked region in the target image, and output the detection result of the target image based on the marked region.

[0084] It's easy to understand that biomedical images exhibit significant content differences across categories. For example, protein blots primarily provide information on band intensity and width, while tissue staining contains cellular and tissue structural morphology information. This data variability places different demands on image tampering detection techniques, preventing a one-size-fits-all approach. Furthermore, real-world biomedical tampering images show that tampering typically occurs within the same category, highlighting the significant differences between different categories. Detecting tampering within the same category not only improves the accuracy of detection methods or models but also effectively reduces useless detections between different categories, greatly enhancing the overall performance of the detection system. Therefore, fine-grained classification of biomedical images is necessary. Biomedical images in academic papers are often composite images; for accurate fine-grained classification, sub-image segmentation is a prerequisite.

[0085] Image tampering detection typically uses datasets such as CASIA, NIST16, and MS COCO, which contain tens of thousands of images. However, these datasets often lack biomedical images, as a large number of images are used for training tampering models. Real biomedical tampered images are often difficult to obtain in large quantities. Multimodal image anomaly detection models improve upon the visual multimodal model clip by combining it with text prompts on various medical image datasets. This allows for training with fewer samples, which is advantageous for biomedical images with limited sample sizes. Combining image tampering detection with image anomaly detection, and training and adjusting the multimodal anomaly detection model using a small sample of biomedical image data and corresponding category text prompts based on the characteristics of each image class, enables tampering detection of finely classified images, improving detection accuracy.

[0086] Biomedical image reuse is a significant aspect of academic misconduct. Feature point detection methods from the field of image registration are used to detect suspicious reuse regions in images. These methods include traditional approaches and deep learning methods. Depending on the image category and detection characteristics, different methods can be combined, and the model can be fine-tuned using fine-grained image data to better adapt to the detection task of that image category. For example, the traditional method SIFT supports rotation invariance, different viewpoints, and scaling, but its ability to detect regions with smooth textures is limited. Superpoint and XFeat have limited support for rotation invariance, but their feature point extraction is denser than traditional methods. XFeat has good support for regions with smooth textures, and it has low device requirements and high speed. SE2-Loftr has excellent support for rotation invariance, but its support for scaling is slightly insufficient. WestornBlot-XFeat is a detection model tailored for protein blot images, which can enhance the detection ability for different image categories. These methods each have their advantages and disadvantages, complementing each other, and their integration can better detect the reuse problem in biomedical images. By setting empirical values ​​or using learnable weighting coefficients to dynamically adjust the feature point detection method for each type of image, the methods can be more fully integrated, resulting in a better fused detection method.

[0087] Anomaly detection of the image itself cannot distinguish between reusing and splicing deletion. A two-branch structure is adopted to apply image reusing detection to the image itself. By checking whether the reusing area and the anomaly detection area overlap, the tampering type of the image itself can be determined: if they overlap, it is reusing; otherwise, it is splicing deletion.

[0088] Furthermore, in this embodiment, the training process of the biomedical classification model includes: acquiring biomedical image data of each category; adding category labels to the biomedical image data and supplementing the biomedical image data using data augmentation techniques to construct a dataset; dividing the dataset into a training set, a test set, and a validation set; modifying the output dimension of the fully connected layer of the initial ResNet model according to the ResNet50 architecture and the preset number of categories; loading pre-trained weights to accelerate convergence and setting the optimizer (Adam) and cross-entropy loss function; performing forward and backward propagation through the training set, and iteratively adjusting the parameters of the initial ResNet model using the validation set until the loss function converges.

[0089] Specifically, we prepared biomedical image data for various categories, added category labels to the images, and supplemented the dataset using data augmentation techniques (such as rotation, flipping, cropping, scaling, etc.). The dataset was divided into training, testing, and validation sets. We used the ResNet50 architecture, modifying the output dimension of the fully connected layers of the ResNet model according to the number of categories in the medical image classification task. We loaded pre-trained weights to accelerate convergence, set the optimizer (Adam) and loss function (cross-entropy loss), performed forward and backward propagation using the training set, and adjusted the parameters using the validation set and saved the optimal model parameters.

[0090] After training a biomedical classification model, n subgraphs can be used to obtain a set of p categories of biomedical subgraph images.

[0091] Further, the step of determining the feature point reuse detection rules corresponding to each of the multiple sub-image sets, and performing reuse detection on similar sub-images within each sub-image set according to the feature point reuse detection rules, includes: determining the image features corresponding to each sub-image set; determining multiple feature point detection methods corresponding to each sub-image set according to the image features and preset detection rules; and performing weighted fusion detection on the multiple feature point detection methods corresponding to each sub-image set to obtain the first result.

[0092] Specifically, the process of weighted fusion detection of multiple feature point detection methods corresponding to each sub-image image set according to the multiple feature point detection methods corresponding to each sub-image image set satisfies:

[0093] ;

[0094] In the formula, The set of matching points after fusion. The first set of matching points between two images represents the combination of the first matching points of the first image and the second image. For the first of the two images There are n matching points, where n is the number of feature point detection methods. For the first The set of matching points corresponding to each detection method, each detection method has Matching points, Indicates corresponding to The weighting coefficients satisfy:

[0095] .

[0096] More specifically, the weighting coefficients The weight coefficients are obtained through data learning based on a preset sample set. The learning process includes: initializing weight coefficients. For each pair of images in the sample set, a weighted fusion calculation is performed to obtain a weighted matching point set. ; Calculate the weighted matching point set The set of matching points corresponding to the sample set The overlap score; to maximize the overlap score. To achieve the goal, iteratively update the initialized weight coefficients. Until the overlap score is obtained. The current iteration number is less than the preset threshold, and / or the current iteration number has reached the preset iteration number.

[0097] For the p-seed image set, one or more suitable feature point reuse detection methods are selected according to the characteristics of each image class to perform reuse detection among sub-images within the same class. Since there are significant differences between categories of biomedical images, image reuse typically occurs between images within the same category. Therefore, this module performs reuse detection among multiple images within each category of the classified biomedical sub-images, reducing unnecessary image detection and effectively improving detection efficiency. Furthermore, the feature point detection method can also be a fusion of multiple methods.

[0098] Specifically, feature point detection is performed separately for multiple images within the same category. Different image categories may employ different feature point detection methods, each with its own advantages and disadvantages. Depending on the image characteristics, existing feature point detection methods that match the image category can be selected, or the model can be specifically trained and adjusted using biomedical sub-image data of the category to better suit the feature point detection of the current category of biomedical images. For example, XFeat is an effective end-to-end detection method for low-quality, smooth-textured images. To better apply it to protein blot image detection, the XFeat pre-trained model can be fine-tuned using a protein blot image dataset to generate a new feature point detection model, westornBlot-xfeat. These feature point detection methods are then fused together, complementing each other to achieve better matching results when performing feature point detection on images.

[0099] Please refer to Figure 2 Feature point detection methods can be selected and combined in different ways according to the characteristics of protein images, including traditional, deep learning-based, and category data-based fine-tuning methods such as westornBlot-xfeat. Multiple combinations can maintain the original detection capabilities and enhance the detection capabilities of protein imprint images.

[0100] exist Figure 2In the processing flow shown, the input images refer to two biomedical sub-images of the same category. The feature point detection method can be selected differently depending on the characteristics of the biomedical image. Figure 2 This study selected several feature point detection methods, including SIFT, Superpoint, XFeat, WestornBlot-XFeat (fine-tuned using protein blot images), and SE2-Loftr. SIFT is a classic detection method widely used in various fields, supporting rotation invariance and scaling, but its detection capability is limited for smooth-textured images such as protein blots. Superpoint is a classic deep learning-based detection method that extracts more densely more image feature points than traditional methods, compensating for the shortcomings of traditional methods and forming denser feature point matches; however, its support for rotation invariance is relatively poor. XFeat is mainly used to compensate for its detection capability on smooth-textured images, while WestornBlot-XFeat can enhance the detection of protein blot images. SE2-Loftr is used to compensate for the rotation invariance detection capability of deep learning methods, and this method supports rotation invariance well. The matching methods also vary. SIFT and Superpoint can use the LightGlue method for feature point matching, XFeat can use both end-to-end and LightGlue matching, and SE2-Loftr uses its end-to-end matching. After using these five feature point detection methods, five matching results will be obtained.

[0101] In this embodiment, the aforementioned weighting coefficients can be set based on practical experience, such as by setting the mean, or they can be learned using data. This allows for dynamic adjustment of each type of image data, improving the fusion capability of the reuse detection method. Learning the weighting coefficients requires a set of image pairs that have matching relationships. The learning objective is to obtain the set of matching points after weighted image fusion. Set of matching points with the sample set The higher the degree of overlap, the better.

[0102] Furthermore, in this embodiment, the formula for calculating the overlap score is:

[0103] ;

[0104] In the formula, The overlap score is calculated as follows: For recall rate, Used to indicate the proportion of the number of weighted matching points that overlap with the total number of actual matching points. For accuracy, Used to indicate the proportion of the number of points in the weighted matching point set that overlap with the actual matching points out of the total number of weighted matching points.

[0105] Furthermore, in this embodiment, multiple sub-images are sequentially detected to obtain a second detection result, including:

[0106] The images of each sub-image are input into the multimodal anomaly detection model in sequence;

[0107] The multimodal anomaly detection branch is used to perform multimodal anomaly detection on the sub-image to obtain anomaly detection results. The multimodal anomaly detection branch includes multiple sub-models, and each sub-model corresponds to the image category of each sub-image set.

[0108] The feature point reuse detection branch performs feature point reuse detection on the sub-image to obtain reuse detection results. The feature point reuse detection branch divides the sub-image into multiple blocks, and sequentially determines the feature point matching results of any current block with other blocks to obtain a fused set of matching points as the reuse detection results.

[0109] Based on the anomaly detection result and the reuse detection result, the second detection result is determined. The second detection result includes a binarized matrix obtained through multimodal anomaly detection and a set of matching points obtained through feature point reuse detection.

[0110] It should be noted that this step focuses on a single image and employs a dual-branch structure of anomaly detection and reuse detection. At this stage, anomaly detection and reuse detection are performed simultaneously on each biomedical image after fine-classification. Anomaly detection can detect suspicious copy-moves and spliced ​​deletion regions, but it cannot distinguish the anomaly type. Reuse detection can detect reused regions, which overlap with the copy-move detection of anomalies, thus allowing for the determination of reuse and spliced ​​deletion types in subsequent processes.

[0111] Specifically, please refer to Figure 3 The specific method and process for this step are as follows: Figure 3 As shown, the input biomedical image refers to a biomedical sub-image.

[0112] The content of biomedical images varies significantly across different categories. For example, protein blots primarily provide information on band intensity and width, while tissue staining contains cellular and tissue structural morphology information. To better detect anomalies in each image category, a multimodal anomaly detection model can be trained for each category, resulting in a set of models for anomaly detection in different biomedical image categories. Specifically, for each image category: a pre-trained model using the MVFA (Multi-level Visual Feature Adapter) framework can be employed. The parameters of this pre-trained model are then adjusted using a small number of samples and the corresponding text prompt for that category to obtain the anomaly detection model for that category. The specific process for adjusting and training the model is as follows:

[0113] Data Preparation: The dataset samples include both normal and abnormal images. Both normal and abnormal images contain the original images and anomaly-marking mask images (the mask is a binary image of the same size as the sample images; normal values ​​are black (0), and abnormal values ​​are white (255)). The abnormal samples include copied-and-pasted and spliced-and-deleted images; the abnormal areas of these images are marked in the corresponding mask image regions. Abnormal data is obtained from both real manipulated images and user-created manipulated images to ensure a sufficient sample size. User-created manipulated images can be repaired using image inpainting and fusion techniques to fill in and restore the manipulated areas or edges, making the fusion with the original image more natural and closer to real image manipulation. For example, image segmentation techniques can be used to select areas to be cut or spliced ​​for deletion. Then, Poisson blending can be applied to the pasted or spliced ​​edge areas to seamlessly embed a region from the original image into the target region, creating a natural, seamless altered image. For deleted areas, image inpainting models such as DeepFill can be used to generate content consistent with the surrounding image areas, filling in the deleted areas and making the repaired areas look natural and seamless with the original image. Regarding text prompts, in addition to using regular text prompts, the prompts can be appropriately adjusted for each category. This strengthens the consistency between the image and text, enhancing the model's understanding and accuracy, and improving its few-shot learning ability.

[0114] Model initialization: MVFA uses the pre-trained visual language model CLIP as its infrastructure and inserts multiple multi-level feature adapters into CLIP's visual encoder to progressively enhance visual features to adapt to the medical image anomaly detection task. The pre-trained model is loaded, and the initialization prompt is executed.

[0115] Model training: Few-shot training and tuning, randomly selecting normal and abnormal samples from the dataset for training. After multiple training sessions and shot adjustments, the optimal training result is selected and the model parameters are saved.

[0116] Biomedical images are processed by a multimodal anomaly detection model, which can detect and mark abnormal regions (similar to the mask in the sample; the closer the pixel value is to 255, the higher the anomaly rate). The mask is usually binarized. The binarization process is that if the pixel value is greater than a certain threshold (such as 200), it is set to 255 to indicate an anomaly; otherwise, it is 0 to indicate normal. These anomalies may be cop-move or splicing deletion types.

[0117] Feature point reuse detection: This is used to detect reuse issues within the image itself. Compared to the fusion feature point detection in section 4), the selection and fusion of feature point detection methods are the same. The difference is that since this is a reuse detection of the image itself (two identical images), the matching process involves removing the original feature points and searching for other matching feature points. Therefore, in practice, the image is divided into several blocks (e.g., 64*n), and feature points in the current block are matched with those in other blocks.

[0118] Specifically, please refer to Figure 4 The input consists of two identical subgraphs (i.e., one graph).

[0119] Feature point extraction methods include using SIFT, Superpoint, XFeat, and the self-trained model WestornBlot-XFeat for feature point detection. The difference from the aforementioned methods is that only feature point extraction is performed; matching methods such as LightGlue are not used in this step. After this operation, the image yields four feature point sets.

[0120] Matching feature points includes: dividing the graph into n blocks. Each block contains several feature points. During matching, matching algorithms such as LightGlue can be used to find matching feature points in other blocks for the feature points of each block. If a match is found, then... Find matching points in The middle one is in The process of finding matching points continues until all feature points of all blocks have been found, ultimately yielding the set of matching points for the current image. By applying the feature point matching strategy described above to each of the four feature sets, four matching point sets can be obtained.

[0121] The fusion of matching feature points includes: weighted fusion of four matching point sets to obtain the final matching point set of the image itself.

[0122] The output includes two types: one is a binary matrix mask of the image size after anomaly detection; the other is the result of image reuse detection, with a set of matching points as shown below. .

[0123] Furthermore, in this embodiment, the marked region includes a marked anomalous region, which includes a mark for the reuse detection result between two sub-images. The marking process for the reuse detection result between the two sub-images includes:

[0124] Using a pre-defined density clustering algorithm, density clustering is performed on the two sub-images to obtain clusters. and ,in, It is the kth cluster in image 1, and each cluster contains several matching points and bounding box parameters;

[0125] Within each cluster, a filter is performed to remove rectangles that do not meet the preset filter rules, and the filtered clusters are marked with a preset rectangle color so that the rectangle colors of corresponding matching point clusters in the two sub-images are the same.

[0126] Arrange the rectangles in each cluster from largest to smallest, and then... Perform the following operations in sequence: Each rectangle in Each bounding box is compared individually; if the contained point matches the corresponding point, then the current bounding box is... and All rectangles are marked with the same color until all rectangles are marked.

[0127] right Perform the following operations in sequence: Each rectangle in Each bounding box is compared individually; if the contained point matches the corresponding point, then the current bounding box is... and Mark all rectangles with the same color until all rectangles are marked; remove and The unmarked rectangle in the middle.

[0128] In this embodiment, the density-based clustering algorithm DBSCAN can be used for clustering. DBSCAN can divide data points in high-density areas into clusters without pre-specifying the number of clusters, thus enabling the identification of clusters of arbitrary shapes and effectively handling noisy data. This method is suitable for labeling the coordinates of matching points. The DBSCAN algorithm has two main parameters: the maximum radius of the neighborhood. and the minimum number of points in the neighborhood A cluster is an object that contains at least one cluster within its neighborhood. Points of an object. For example, to match points, calculate the distance between two points. If the number of points in a given class is greater than a certain value, then group these two points into one class. Recursively calculate and group all such points into one class. If the number of points in that class is greater than a certain value, then group the two points into one class. If a point is smaller than a certain value, it is considered part of a cluster; otherwise, it is considered noise. This process is repeated until all points have been calculated and classified, thus completing density classification. Density clustering is then performed on the two images to obtain the respective clusters. and ,in It is the kth cluster in image 1, and each cluster contains several matching points and rectangle parameters.

[0129] The purpose of filtering within each cluster is to remove rectangles that are too small, as well as rectangles with a large proportion of text within them. In this embodiment, text detection can be implemented using EAST, an open-source implementation that quickly predicts the geometry (e.g., rectangles) of text lines. If the detected text area is too large (e.g., exceeding 1 / 3 of the original rectangle area), it should be filtered out and deleted.

[0130] Cluster and Mark the corresponding bounding box color so that matching point clusters in two images use the same bounding box color. Sort the bounding boxes in the cluster by size, from largest to smallest. Perform the following operations: Each rectangle and Each bounding box is compared individually; if the contained point matches the corresponding point, then the current bounding box is... and All rectangles are marked with the same color until all rectangles are marked with color. For Perform the same operation on the rectangles that are not marked with color, and finally remove them. and An unmarked rectangle.

[0131] Furthermore, the method in this embodiment also includes: determining the image tampering type based on the marked area, specifically including:

[0132] Based on the marked regions corresponding to the sub-image after anomaly detection by the multimodal anomaly detection model, a first set of bounding boxes is determined; based on the marked regions corresponding to the sub-image after feature point reuse detection by the multimodal anomaly detection model, a second set of bounding boxes is determined; it is then determined whether the areas of the first set of bounding boxes and the second set of bounding boxes are greater than a preset threshold; if yes, the image tampering type is determined to be a reuse type; if no, the image tampering type is determined to be a splicing and deletion type.

[0133] The second embodiment of the present invention also proposes an academic misconduct detection system based on biomedical images, the system comprising:

[0134] The data acquisition module is configured to acquire the target image to be detected.

[0135] The image segmentation module is configured to segment the target image using a preset contour extraction algorithm to obtain multiple sub-images. The contour extraction algorithm converts the target image into a binary image and searches for a continuous set of points in the binary image to determine the contour boundary.

[0136] The image classification module is configured to perform fine-grained classification on multiple sub-images based on a pre-trained biomedical classification model, and divide the multiple sub-images into multiple sub-image sets based on the results of the fine-grained classification, wherein each sub-image set includes at least one class of sub-images.

[0137] The reuse detection module is configured to determine the feature point reuse detection rules corresponding to each of the multiple sub-image sets, and perform reuse detection on the same type of sub-images within each sub-image set according to the feature point reuse detection rules to obtain a first detection result;

[0138] An anomaly detection module is configured to perform anomaly detection on multiple sub-images sequentially based on a pre-trained multimodal anomaly detection model to obtain a second detection result. The multimodal anomaly detection model includes a dual-branch structure for anomaly detection and for feature point reuse detection. The multimodal anomaly detection is used to determine the copied region and spliced ​​deletion region of the sub-image, and the feature point reuse detection is used to determine the reuse region of the sub-image.

[0139] The marker output module is configured to determine the marker region in the target image based on the first detection result and the second detection result, and output the detection result of the target image based on the marker region.

[0140] For the principles and specific implementation processes of each module, please refer to the explanations in the above method embodiments. This embodiment will not repeat them here.

[0141] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system suitable for implementing embodiments of the methods, systems, and apparatus of the present invention. Figure 5 The server shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0142] like Figure 5 As shown, the computer system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 302 or programs loaded from storage section 308 into Random Access Memory (RAM) 303. RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0143] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0144] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of the present invention. It should be noted that the computer-readable medium described above in the present invention can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.

[0145] More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0146] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0148] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.

[0149] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.

[0150] The technical solution of the present invention has now been described in conjunction with the preferred embodiments shown in the accompanying drawings.

[0151] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A biomedical image-based academic misconduct detection method for detecting pictures in biomedical literature, comprising: The method comprises: acquiring a target picture to be detected; performing image segmentation on the target picture by using a preset contour extraction algorithm to obtain a plurality of sub-picture sets, the contour extraction algorithm being to convert the target picture into a binary image and find a continuous point set in the binary image to determine a contour boundary; performing fine-grained classification on the plurality of sub-pictures based on a pre-trained biomedical classification model, and dividing the plurality of sub-pictures into a plurality of sub-picture sets according to the result of fine-grained classification; determining a feature point multiplexing detection rule corresponding to each sub-picture set according to the plurality of sub-picture sets, and performing multiplexing detection on the same sub-pictures in each sub-picture set according to the feature point multiplexing detection rule to obtain a first detection result; performing abnormal detection on the plurality of sub-pictures in sequence based on a pre-trained multi-modal abnormal detection model to obtain a second detection result, the multi-modal abnormal detection model comprising a double-branch structure for abnormal detection and feature point multiplexing detection, the multi-modal abnormal detection being used to determine a replication area and a splicing deletion area of the sub-pictures, and the feature point multiplexing detection being used to determine a multiplexing area of the sub-pictures; determining a marked area in the target picture according to the first detection result and the second detection result, and outputting a detection result of the target picture according to the marked area.

2. The biomedical image-based academic misconduct detection method of claim 1, wherein, The training process of the biomedical classification model comprises: acquiring biomedical image data of each category; adding category labels to the biomedical image data and supplementing the biomedical image data by using a data enhancement technique to construct a data set; modifying the output dimension of the fully connected layer of an initial ResNet model according to the ResNet50 architecture and a preset category number, loading pre-trained weights to accelerate convergence, setting an optimizer and a cross-entropy loss function; performing forward propagation and backward propagation by using a training set, and iteratively adjusting the parameters of the initial ResNet model by using the data set and the optimizer until the cross-entropy loss function converges. 3.The biomedical image-based academic misconduct detection method of claim 1, wherein, The determination of the feature point multiplexing detection rule corresponding to each sub-picture set according to the plurality of sub-picture sets, and the multiplexing detection on the same sub-pictures in each sub-picture set according to the feature point multiplexing detection rule comprise: determining image features corresponding to each sub-picture set; determining a plurality of feature point detection methods corresponding to each sub-picture set according to the image features and a preset detection corresponding rule; performing weighted fusion detection on each sub-picture set according to the plurality of feature point detection methods corresponding to each sub-picture set to obtain the first detection result.

4. The biomedical image-based academic misconduct detection method of claim 3, wherein, The weighted fusion detection on each sub-picture set according to the plurality of feature point detection methods corresponding to each sub-picture set satisfies: ； In the formula, The set of matching points after fusion. The first set of matching points between two images represents the combination of the first matching points in the first image and the second image. For the first of the two images There are n matching points, where n is the number of feature point detection methods. For the first The set of matching points corresponding to each detection method, each detection method has Matching points, Indicates corresponding to The weighting coefficients satisfy: 。 5. The method of claim 4, wherein, The weight coefficient is obtained through data learning based on a preset sample set, and a learning process of the weight coefficient includes: Initializing weight coefficients ; performing weighted fusion calculation on each pair of images in the preset sample set to obtain a weighted matching point set; calculating a coincidence degree score of the weighted matching point set and a matching point set corresponding to the preset sample set; iteratively updating the initialized weight coefficients with a goal of maximizing the concordance score until the concordance score is less than a preset threshold and / or a current iteration number reaches a preset iteration number.

6. The method of claim 5, wherein, the calculation formula of the coincidence degree score is: ； In the formula, is a degree of overlap score, is a recall rate, is used to indicate the proportion of the number of weighted matching points that coincide with the true matching points in the total number of true matching points, is a precision rate, is used to indicate the proportion of the number of weighted matching points that coincide with the true matching points in the total number of weighted matching points.

7. The method of claim 1, wherein, The pre-trained multi-modal anomaly detection model sequentially detects the plurality of sub-picture images to obtain a second detection result, including: sequentially inputting each sub-picture image into the multi-modal anomaly detection model; performing multi-modal anomaly detection on the sub-picture image through a multi-modal anomaly detection branch to obtain an anomaly detection result, the multi-modal anomaly detection branch including a plurality of sub-models, each sub-model corresponding to a picture category of each sub-picture image set; performing feature point multiplexing detection on the sub-picture image through a feature point multiplexing detection branch to obtain a multiplexing detection result, the feature point multiplexing detection branch cutting the sub-picture image into a plurality of blocks, sequentially determining a result of feature point matching between any one current block and other blocks to obtain a fused matching point set as the multiplexing detection result; determining the second detection result according to the anomaly detection result and the multiplexing detection result, the second detection result including a binary matrix obtained through multi-modal anomaly detection and a matching point set obtained through feature point multiplexing detection.

8. The method of claim 1, wherein, The marked area includes a marked anomaly area, and a marking process of two sub-picture inter-multiplexing detection results includes: The two subgraph pictures are subjected to density clustering by using a preset density clustering algorithm to obtain clustering clusters and Each clustering cluster contains a plurality of matching points and rectangular frame parameters. filtering in each clustering cluster to remove rectangular boxes that do not meet a preset filtering rule, and marking the filtered clustering cluster with a preset rectangular box color so that the rectangular boxes of the corresponding matching point clusters of the two sub-pictures have the same color; Arrange the rectangles in each cluster from largest to smallest, and then sort the clusters sequentially. Each rectangle in the data corresponds to a cluster. Each bounding box is compared individually. If the bounding box contains a matching point, then the current cluster is considered. and clusters All rectangles are marked with the same color until all rectangles are marked. Each rectangular frame in the cluster group is compared with all rectangular frames in the cluster group respectively, if the contained points are the corresponding matching points, then the rectangular frames of the current cluster group and the cluster group are marked with the same color, until all rectangular frames are marked, the unmarked rectangular frames in the cluster group and the cluster group are removed.

9. The method of claim 1, wherein, The method further includes: determining a picture tampering type according to the marked area, specifically including: determining a first rectangular box set according to the marked area corresponding to the sub-picture after anomaly detection through the multi-modal anomaly detection model; determining a second rectangular box set according to the marked area corresponding to the sub-picture after feature point multiplexing detection through the multi-modal anomaly detection model; respectively determining whether the areas of the first rectangular box set and the second rectangular box set are greater than a preset threshold value; if yes, determining that the picture tampering type is a multiplexing type; if no, determining that the picture tampering type is a splicing and deletion type. 10.A biomedical image-based academic misconduct detection system, characterized by, The system includes: a data acquisition module configured to acquire a target picture to be detected; an image segmentation module configured to perform image segmentation on the target picture using a preset contour extraction algorithm to obtain a plurality of sub-picture images, the contour extraction algorithm converting the target picture into a binary image and finding a continuous point set in the binary image to determine a contour boundary; an image classification module configured to perform fine-grained classification on a plurality of the sub-picture images based on a pre-trained biomedical classification model, and divide the plurality of sub-pictures into a plurality of sub-picture sets according to the fine-grained classification result, each sub-picture set including at least one type of sub-picture; a multiplexing detection module configured to determine a feature point multiplexing detection rule corresponding to each sub-picture set according to a plurality of the sub-picture sets, and perform multiplexing detection on the same type of sub-picture within each sub-picture set according to the feature point multiplexing detection rule to obtain a first detection result; an anomaly detection module configured to sequentially perform anomaly detection on the plurality of sub-picture images based on a pre-trained multi-modal anomaly detection model to obtain a second detection result, wherein the multi-modal anomaly detection model comprises a double-branch structure for anomaly detection and for feature point multiplexing detection, the multi-modal anomaly detection is used to determine a replication region and a splicing deletion region of the sub-picture images, and the feature point multiplexing detection is used to determine a multiplexing region of the sub-picture images; a label output module configured to determine a label region in the target picture according to the first detection result and the second detection result, and output a detection result of the target picture according to the label region.

Images