Image duplicate checking method and device, electronic equipment, storage medium and product

Abstract

The present disclosure relates to the technical field of image processing, and particularly provides an image duplicate checking method and device, electronic equipment, storage medium and product. In the present disclosure, based on the image features of a to-be-processed image, duplicate checking retrieval is performed in a target database to obtain candidate images; multi-modal features of the to-be-processed image are acquired, the multi-modal features including at least two of text features, time-frequency features, color features or object features; based on the multi-modal features, duplicate checking analysis is performed on the to-be-processed image and the candidate images to obtain a duplicate checking result. In this way, by further combining the multi-modal features of the to-be-processed image and the candidate images for duplicate checking analysis on the basis of the candidate images retrieved, the recognition accuracy of repeated content in complex images or processed images is improved, and the accuracy of image duplicate checking is further improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image deduplication method, apparatus, electronic device, storage medium and product. Background Technology

[0002] Image plagiarism detection refers to the use of computer technology to analyze and understand information in images, including identifying objects, scenes, and faces in images. It is widely used in fields such as security monitoring, autonomous driving, and medical image analysis.

[0003] Related technologies provide an image deduplication technique that extracts the visual feature vector of an input image and compares it with a stored image visual feature library. It calculates the similarity distance between the input image's visual feature vector and each image visual feature stored in the library, and outputs the results in a sorted order. Images with similarity exceeding a threshold are identified as duplicate images, and the deduplication result is output. However, the image deduplication technique provided by this technology may not accurately identify duplicate content in processed or complex images, resulting in low accuracy. Summary of the Invention

[0004] This disclosure addresses some deficiencies mentioned in the background art by providing an image deduplication method, apparatus, electronic device, storage medium, and product that can improve the accuracy of image deduplication.

[0005] In a first aspect, embodiments of this disclosure provide an image deduplication method, comprising: Based on the image features of the image to be processed, a deduplication search is performed in the target database to obtain candidate images; Obtain the multimodal features of the image to be processed, wherein the multimodal features include at least two of the following: text features, time-frequency features, color features, or object features; Based on the multimodal features, a deduplication analysis is performed on the image to be processed and the candidate images to obtain the deduplication results.

[0006] Optionally, if the multimodal features include the text features, obtaining the multimodal features of the image to be processed includes: Detect text regions in the image to be processed; The text region is subjected to text recognition to obtain the text content; The text content is encoded to obtain the text features.

[0007] Optionally, if the multimodal features include the time-frequency features, obtaining the multimodal features of the image to be processed includes: The image to be processed is subjected to frequency domain transformation to obtain the transform coefficient matrix; A one-dimensional vector is determined based on the transformation coefficient matrix; The time-frequency feature is obtained by normalizing the one-dimensional vector.

[0008] Optionally, if the multimodal features include the color features, obtaining the multimodal features of the image to be processed includes: The image to be processed is spatially segmented to obtain multiple sub-regions; Color histogram quantization is performed on each of the sub-regions to obtain the color histogram corresponding to each of the sub-regions; The color features are obtained by performing feature fusion processing on multiple color histograms.

[0009] Optionally, if the multimodal features include the object features, obtaining the multimodal features of the image to be processed includes: The object features are determined based on the image to be processed using a pre-trained feature extraction model; wherein the pre-trained feature extraction model includes a spatial attention module and a channel attention module, and the pre-trained feature extraction model is obtained by optimizing the feature extraction model based on an attention mechanism.

[0010] Optionally, the step of performing deduplication retrieval in the target database based on the image features of the image to be processed to obtain candidate images includes: The image to be processed is hash-encoded to obtain a first hash value; The first hash value is used as a hash index to match and retrieve the second hash value corresponding to each image in the target database. For any image in the target database, if the hash similarity between the second hash value and the first hash value of the image is greater than or equal to the first threshold, the image is determined as the candidate image.

[0011] Optionally, the step of performing a deduplication analysis on the image to be processed and the candidate images based on the multimodal features to obtain the deduplication result includes: Obtain the vector similarity between the first feature vector and the second feature vector; wherein, the first feature vector is the feature vector corresponding to the multimodal features of the image to be processed, and the second feature vector is the feature vector corresponding to the multimodal features of the candidate image; If the vector similarity is greater than or equal to the second threshold, it is determined that the image to be processed and the candidate image are duplicates. If the vector similarity is less than the second threshold, it is determined that the image to be processed and the candidate image are not duplicates.

[0012] Optionally, the method further includes: Based on the first information, the similarity threshold is dynamically updated; wherein the similarity threshold includes at least one of the following: a first threshold and a second threshold; The first information includes at least one of the following: Distribution of similarity data; Historical plagiarism check results; The data source of the image to be processed; Historical plagiarism detection data; Data volume.

[0013] Optionally, the historical plagiarism detection results are obtained in the following ways: The confirmation results of the plagiarism detection are recorded through a feedback collection mechanism; wherein the confirmation results indicate whether the plagiarism detection results are true positive or false positive. The plagiarism check results, including the confirmation results, are used as the historical plagiarism check results.

[0014] In a second aspect, embodiments of this disclosure provide an image plagiarism detection device, comprising: The retrieval module is used to perform duplicate retrieval in the target database based on the image features of the image to be processed, and obtain candidate images; The acquisition module is used to acquire the multimodal features of the image to be processed, wherein the multimodal features include at least two of the following: text features, time-frequency features, color features, or object features; The plagiarism detection module is used to perform plagiarism analysis on the image to be processed and the candidate images based on the multimodal features, and obtain the plagiarism detection results.

[0015] In a third aspect, embodiments of this disclosure provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described image deduplication method.

[0016] In a fourth aspect, embodiments of this disclosure provide a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the above-described image deduplication method.

[0017] In a fifth aspect, embodiments of this disclosure provide a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described image deduplication method.

[0018] In this disclosure, based on the image features of the image to be processed, a deduplication search is performed in a target database to obtain candidate images; multimodal features of the image to be processed are acquired, including at least two of text features, time-frequency features, color features, or object features; based on the multimodal features, a deduplication analysis is performed between the image to be processed and the candidate images to obtain the deduplication results. Thus, by further combining the multimodal features of the image to be processed with the candidate images for deduplication analysis after retrieving the candidate images, the accuracy of identifying duplicate content in complex or processed images is improved, thereby enhancing the accuracy of image deduplication.

[0019] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0020] Figure 1 A schematic diagram of an image plagiarism detection technology provided for related technologies.

[0021] Figure 2 This is a flowchart illustrating an image deduplication method provided in an embodiment of the present disclosure.

[0022] Figure 3 This is a schematic diagram of a process for acquiring multimodal features according to an embodiment of the present disclosure.

[0023] Figure 4 This is a schematic flowchart of another image deduplication method provided in an embodiment of this disclosure.

[0024] Figure 5 This is a schematic diagram of the structure of an image plagiarism detection device provided in an embodiment of this disclosure.

[0025] Figure 6 This is a structural block diagram of another image deduplication device provided in an embodiment of this disclosure.

[0026] Figure 7 This is a hardware block diagram of an electronic device provided in an embodiment of the present disclosure.

[0027] Figure 8 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this disclosure. Detailed Implementation

[0028] To enable those skilled in the art to better understand the technical solution of this application, the application scenario of this application will be described first below.

[0029] Combination Figure 1As shown, the image plagiarism detection technologies provided by related technologies mainly rely on extracting the visual feature vectors of images and identifying duplicate or similar images through similarity retrieval. Image feature extraction is the first step in image plagiarism detection, and its purpose is to extract key information that can represent the content of the image. The following are some of the main image plagiarism detection technologies: Image feature extraction and deduplication based on wavelet transform refers to utilizing the multi-resolution analysis capabilities of wavelet transform to extract low-frequency features from images. These features have a good representational ability of the overall content of the image and also have a certain degree of noise resistance. For example, by using an illumination feature encoder, a local color histogram feature extraction module, a basic feature encoder, a frequency domain feature extraction module, a basic feature decoder, an illumination feature decoder, and a detail correction module, feature extraction, frequency domain feature reconstruction, illumination decoding, basic decoding, and detail correction are achieved for low-light images, ultimately generating an enhanced image. This method combines frequency domain information and local color histogram features, adaptively perceives changes in the illumination of the image, restores image details, and improves the visual quality of the enhanced image.

[0030] Image feature extraction and deduplication based on perceptual hashing refers to extracting frequency domain features from images using methods such as Discrete Cosine Transform (DCT) to form perceptual hash values, which are then used to quickly compare image similarity. For example, through the collaborative work of local feature extraction branches, global feature extraction branches, attention feature fusion modules, and recognition modules, efficient recognition of pedestrian images is achieved. The system utilizes a multi-scale channel attention mechanism to iteratively integrate features of different scales and semantics, enhancing the model's ability to perceive pedestrian body structures and thus improving the accuracy of cross-modal pedestrian re-identification. Furthermore, the system introduces a dense triplet loss function to further optimize the feature extraction process, ensuring that the model can better recognize pedestrian images under different modalities and pose changes.

[0031] Image feature extraction and deduplication based on deep learning refers to using deep learning models such as Convolutional Neural Networks (CNNs) and ResNet residual networks to extract deep features from images. These features can capture key visual elements and patterns in the image, generating a unique "fingerprint" for the image. For example, by obtaining the feature point matching set between the current frame image and the previous frame image, when the number of matching feature points is insufficient, the current frame image is reduced in size, and feature points in the reduced image are selected based on feature point scores to determine the corresponding points in the original image. Finally, these corresponding points are added to the feature point matching set, thereby enhancing the feature point matching set, reducing computational load, and improving the applicability of the SLAM algorithm on devices with lower performance.

[0032] Therefore, it can be seen that existing image plagiarism detection technologies have at least one of the following technical drawbacks: 1. Insufficient robustness to image editing operations: Traditional image feature extraction algorithms may fail to accurately identify duplicate content when processing images that have undergone editing operations such as rotation, scaling, cropping, or color adjustment. This is because many feature extraction methods, especially those based on traditional algorithms, are not robust enough to geometric and optical perturbations of images. For example, techniques based on wavelet transform and perceptual hashing rely on specific image properties; once these properties change, the feature vectors also change, thus limiting the effectiveness of plagiarism detection techniques.

[0033] 2. High computational cost of deep learning methods: Although deep learning-based feature extraction methods improve accuracy, they typically require significant computational resources and time, posing a substantial bottleneck for processing large-scale image datasets. For example, feature extraction using models like ResNet requires extensive matrix operations and backpropagation, usually necessitating high-performance GPUs and lengthy computation times. This high computational resource requirement limits the widespread adoption and application of image plagiarism detection technology, especially in resource-constrained environments.

[0034] 3. Difficulty in Setting Similarity Thresholds: Existing technologies often require manually setting a similarity threshold when determining whether images are duplicates. However, there is a lack of a unified standard for setting this threshold, which needs to be adjusted according to specific application scenarios. Due to the different definitions and tolerances for duplicate images in different scenarios, coupled with the limitations of image feature extraction methods, it is difficult to automatically determine a universally applicable similarity threshold. Furthermore, the choice of similarity calculation method (such as cosine similarity, Euclidean distance, etc.) also affects the threshold setting. The process of manually setting the threshold is both static and tedious, increasing workload and directly impacting the accuracy of the results. Improper setting may lead to missed detections or excessive false alarms, affecting the overall efficiency of the system and user experience.

[0035] To address the aforementioned technical problems, this disclosure provides an inventive concept: by further combining the multimodal features of the image to be processed with the candidate images after retrieving them, a deduplication analysis is performed, thereby improving the accuracy of identifying duplicate content in complex or processed images and thus enhancing the accuracy of image deduplication.

[0036] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present disclosure and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the drawings, not the entire structure.

[0037] Figure 2 This is a schematic flowchart illustrating an image deduplication method provided in an embodiment of this disclosure. Figure 2 As shown, the method includes: S201: Based on the image features of the image to be processed, perform deduplication retrieval in the target database to obtain candidate images.

[0038] The image to be processed refers to the image to be checked for duplicates. Image features refer to the features used for querying and retrieving data from the target database. The target database refers to a pre-set set of image data used for checking for duplicates with the image to be processed. The target database can include multiple images, and the image features corresponding to each image can serve as indexes for each image. Candidate images refer to images retrieved from the target database based on the image features of the image to be processed, which may be duplicates of the image to be processed.

[0039] S202: Obtain the multimodal features of the image to be processed.

[0040] Multimodal features refer to the features represented by different types of content contained in the image to be processed. In the embodiments of this application, multimodal features may include at least two of the following: text features, time-frequency features, color features, or object features.

[0041] Textual features refer to the textual information contained in an image to be processed. This textual information exists in the form of characters or symbols and is used to convey specific meanings or describe the content of the image. As an example, textual information may include, but is not limited to, keywords, annotations, titles, and tags in the image to be processed. This textual information is usually closely related to the content of the image to be processed, providing important clues for the understanding and analysis of the image.

[0042] Time-frequency characteristics refer to the features exhibited by the image to be processed in the time and frequency domains. Time-frequency characteristics reflect the characteristics of image signals changing over time and their frequency distribution.

[0043] Time-frequency features can include time-domain features and frequency-domain features. Time-domain features describe the changes of the image signal on the time axis, such as the mean, variance, and autocorrelation function of the image signal. These features are suitable for analyzing the instantaneous characteristics of the image, such as pulses and waveforms. Frequency-domain features are used to convert the image signal to the frequency domain for analysis through methods such as Fourier transform or wavelet transform. Frequency-domain features can represent the magnitude and relative position of different frequency components in the image to be processed, thereby understanding the frequency characteristics of the image, such as texture and noise.

[0044] Color features refer to the visual features of an image that are related to an object or scene. Color features describe the distribution and combination of colors in the image.

[0045] Color space refers to the appropriate color space that is usually chosen for description in order to quantify color characteristics, such as RGB color space, HSV color space, etc.

[0046] Description methods may include, but are not limited to, color histograms, color moments, color aggregation vectors, etc. Description methods are used to characterize color features by statistically analyzing color distribution or by using low-order statistics.

[0047] Object features refer to the characteristics of each object (such as a body or region) in the image to be processed. Object features describe the object's shape, size, position, texture, and other attributes.

[0048] Shape features describe the outline or boundary shape of an object, such as area, symmetry, boundary index, and aspect ratio. Position features describe the object's positional relationships within an image, such as its correlation with neighboring objects, upper-layer objects, and sub-layer objects. Texture features describe the texture patterns of an object's surface, such as roughness, smoothness, and directionality.

[0049] It should be understood that by extracting features in multiple dimensions, this method can more comprehensively capture the essential features of an image, thus accurately identifying duplicate content even after complex editing operations. This method is particularly suitable for applications requiring high-precision matching and recognition, such as copyright protection and content moderation.

[0050] S203: Based on multimodal features, perform deduplication analysis on the image to be processed and the candidate images to obtain the deduplication results.

[0051] The deduplication result refers to the result indicating whether the image to be processed is duplicated with the candidate image. The deduplication result can be either duplicated or not duplicated.

[0052] The image deduplication method according to embodiments of the present disclosure has been described above with reference to the accompanying drawings. This disclosure improves the accuracy of identifying duplicate content in complex or processed images by further combining the multimodal features of the image to be processed with the candidate images for deduplication analysis, based on the retrieved candidate images.

[0053] Based on the image deduplication method provided in the above embodiments, please refer to... Figure 3 , Figure 3 This is a schematic flowchart illustrating a method for acquiring multimodal features according to an embodiment of the present disclosure. The acquisition methods for each feature are described in detail below with reference to various embodiments.

[0054] In one implementation of the example, if the multimodal features include text features, step S22 may include: Step 11: Detect text regions in the image to be processed.

[0055] Text region refers to the area in the image to be processed that contains text.

[0056] In one possible implementation, to improve the accuracy and reliability of text recognition, the image to be processed can be preprocessed and text detected in order to determine the text region.

[0057] Preprocessing refers to the operations of grayscale conversion, binarization, and noise filtering on the input image to improve the clarity of text regions. Grayscale conversion converts a color image to a grayscale image, reducing computational complexity while retaining sufficient information for subsequent processing. Binarization converts a grayscale image to a black and white image by setting a threshold, creating a clear contrast between the background and foreground (i.e., the text), facilitating subsequent text detection and recognition. Noise filtering uses various filtering algorithms such as Gaussian filtering and median filtering to remove random noise from the image, further improving the clarity of text regions.

[0058] Text detection can use a model based on CTPN (Connectionist Text Proposal Network) to locate text regions in order to generate candidate text boxes. CTPN is a neural network architecture particularly suitable for scene text detection. It can effectively capture the features of text lines and has good adaptability to text regions of different scales and aspect ratios.

[0059] Step 12: Perform text recognition on the text region to obtain the text content.

[0060] Text recognition refers to extracting text sequences using a CRNN (Convolutional Recurrent Neural Network) model and decoding the output text content using a CTC (Connectionist Temporal Classification) loss function. The CRNN model consists of convolutional layers, recurrent layers, and transcription layers. Convolutional layers extract image features, recurrent layers (usually LSTM or GRU) are used for sequence modeling, and CTC allows the model to directly predict text sequences (i.e., text content) of variable length from the image being processed, without requiring precise labeling of the position of each character.

[0061] Step 13: Encode the text content to obtain text features.

[0062] Feature encoding refers to converting the identified text content into a vector representation. A pre-trained BERT (Bidirectional Encoder Representations from Transformers) model is used for semantic embedding, generating a 768-dimensional vector to capture contextual semantic information. The BERT model can deeply understand the contextual semantic information of the text and map it into a 768-dimensional vector space. This representation not only includes the meaning of the text content itself but also considers the position and role of the text content in the image being processed, thus providing a richer and more accurate feature representation.

[0063] Based on the image deduplication method provided in the above embodiments, in one implementation of the example, if the multimodal features include time-frequency features, step S22 may include: Step 21: Perform frequency domain transformation on the image to be processed to obtain the transformation coefficient matrix.

[0064] Frequency domain transformation refers to performing Discrete Cosine Transform (DCT) on the image to be processed, separating low-frequency (overall contour) and high-frequency (edge ​​details) components. DCT is a technique that converts spatial domain signals into frequency domain representations and is widely used in image processing. Through DCT, an image can be decomposed into different frequency components, thus achieving a frequency domain representation of image information. Specifically, this process can effectively separate low-frequency components and high-frequency components. Low-frequency components mainly contain the overall contour and basic tone of the image, while high-frequency components reflect more of the edge details and texture variations in the image.

[0065] Step 22: Determine the one-dimensional vector based on the transformation coefficient matrix.

[0066] By performing frequency domain transformation on the image to be processed, a transform coefficient matrix (DCT coefficient matrix) can be obtained. Further, the first 64×64 low-frequency components of the DCT coefficient matrix can be extracted. These low-frequency components represent the main structural information of the image to be processed and are crucial for describing its basic shape and color distribution. Then, the two-dimensional DCT coefficient matrix (i.e., the low-frequency components) can be converted into a one-dimensional vector using Zig-Zag scanning. Zig-Zag scanning traverses the matrix elements of the DCT coefficient matrix in a specific order, prioritizing the low-frequency coefficients, and reorganizes the two-dimensional data into a one-dimensional vector in this way, facilitating subsequent processing and analysis.

[0067] Step 23: Normalize the one-dimensional vector to obtain the time-frequency features.

[0068] Normalization refers to L2 normalization of a one-dimensional vector to eliminate the effects of differences in lighting and contrast. L2 normalization is a standardization technique that divides each element of the vector by its Euclidean norm, resulting in a vector of unit length. This method helps eliminate variations caused by lighting conditions and contrast differences, ensuring the consistency and stability of the feature vector under different conditions. The application of L2 normalization enhances the robustness of the algorithm, enabling it to provide reliable and consistent performance in various environments.

[0069] Based on the image deduplication method provided in the above embodiments, in one implementation of the example, if the multimodal features include color features, step S22 may include: Step 31: Perform spatial segmentation on the image to be processed to obtain multiple sub-regions.

[0070] Spatial segmentation refers to dividing the image to be processed into an 8×8 grid, with each sub-region having its color distribution calculated independently. Each sub-region is treated as an independent unit, and its own color distribution is calculated based on this unit. Spatial segmentation not only helps capture local color information in the image to be processed but also improves the algorithm's sensitivity and robustness to color changes. Spatial segmentation can effectively handle color inconsistencies caused by changes in lighting or shadows.

[0071] Step 32: Perform color histogram quantization on each sub-region to obtain the color histogram corresponding to each sub-region.

[0072] Color histogram quantization refers to the process of performing color histogram quantization on the color information of each sub-region in the HSV color space.

[0073] The HSV color model consists of three components: Hue (H), Saturation (S), and Value (V). In this embodiment, Hue (H) is divided into 16 levels, covering the entire color wheel, thus accurately representing different colors. Saturation (S) and Value (V) are each divided into 4 levels, used to describe the purity and brightness of the color, respectively. By quantizing the color distribution of each sub-region, a 16×4×4=256-dimensional color histogram is generated.

[0074] Color histogram quantization allows the system to consider not only the overall color distribution but also specific color attributes such as hue, saturation, and brightness variations. This enhances the algorithm's adaptability to color changes and improves its stability in complex environments.

[0075] Step 33: Perform feature fusion processing on multiple color histograms to obtain color features.

[0076] Feature fusion processing refers to concatenating the 256-dimensional color histograms generated from each sub-region to form a global color feature vector (i.e., color feature). This color feature vector integrates the color distribution information of all sub-regions in the image to be processed, providing a comprehensive and detailed description of the image's color. Through feature fusion processing, not only can the uniqueness of local color information be preserved, but the overall color composition of the image to be processed can also be grasped, providing strong support for subsequent image analysis tasks.

[0077] Based on the image deduplication method provided in the above embodiments, in one implementation of the example, if the multimodal features include object features, step S22 may include: using a pre-trained feature extraction model to determine the object features based on the image to be processed.

[0078] As one possible implementation, the pre-trained feature extraction model can employ an improved YOLOv8 model, and the process of obtaining object features can be as follows: Step 41: Model optimization.

[0079] In this embodiment, the model can be optimized using an attention mechanism to enhance its ability to detect small targets. The attention mechanism is a technique that mimics the human visual system, allowing the model to focus on the most relevant information regions in the image to be processed, thereby improving the accuracy of recognizing specific targets. Specifically, this embodiment introduces an improved Spatial Attention Module (SAM) and Channel Attention Module (CAM) on the YOLOv8 architecture.

[0080] The Spatial Attention Module (SAM) is used to highlight key spatial locations in an image. By evaluating and weighting the importance of each spatial location, the model can pay more attention to regions that contain important information.

[0081] The Channel Attention Module (CAM) is used to automatically enhance useful feature representations and suppress irrelevant or redundant information by learning the importance weights of each channel based on the interrelationships between different feature maps.

[0082] By combining spatial attention and channel attention modules, the model's sensitivity to small targets is improved, and its detection performance in complex backgrounds is also effectively enhanced, ensuring accurate identification of target objects even in the presence of a large number of interfering objects.

[0083] Step 42: Feature extraction.

[0084] The penultimate layer of the model outputs a 1024-dimensional feature vector, which can contain object category, location, and semantic information. This extraction of a 1024-dimensional feature vector from the penultimate layer is achieved through multi-level abstraction and integration, resulting in a final feature vector (i.e., object features) that contains rich semantic, object category, and location information.

[0085] Semantic information refers to conceptual descriptions of an image's content, such as a high-level understanding like "a person is standing under a tree." This is typically achieved through high-level feature maps in deep network structures, which capture global contextual information within the image.

[0086] Object category refers to the class to which the detected object belongs, such as person, car, animal, etc. This is achieved based on the feature representations of different categories learned during the training phase.

[0087] Location information refers to data that provides information about the exact location of an object in an image, including bounding box coordinates. This is crucial for accurately locating and identifying objects in an image.

[0088] As one possible implementation, Feature Pyramid Network (FPN) technology is also applied during feature extraction. FPN technology can construct feature hierarchies at multiple scales, allowing the model to handle targets of different sizes simultaneously. Furthermore, combined with an attention mechanism, FPN helps improve the detection performance of small targets because it allows the model to fuse information at different resolutions, ensuring that even small or partially occluded targets can be effectively identified and classified.

[0089] In one implementation of the image deduplication method provided in the above embodiments, step S21 may include: Step 51: Hash the image to be processed to obtain the first hash value.

[0090] One approach is to preprocess the image to be processed and perform DCT transformation, followed by hash encoding to obtain the first hash value.

[0091] Image preprocessing can include: first, resizing the input image to 32×32 pixels, and then converting the color image to a grayscale image through grayscale conversion. Grayscale conversion converts the RGB value of each pixel into a single value representing brightness, simplifying subsequent processing. Next, brightness normalization is performed on the grayscale image, adjusting the image's brightness level to reduce the impact of changes in lighting conditions. Brightness normalization can include calculating a global or local brightness mean and adjusting the brightness value of each pixel based on this mean, ensuring that images captured under different lighting conditions have a similar brightness distribution.

[0092] Discrete Cosine Transform (DCT) refers to the frequency domain transformation of a preprocessed grayscale image using the Discrete Cosine Transform (DCT). DCT is a technique that converts spatial domain signals into frequency domain representations and is widely used in image compression and feature extraction. During the DCT transformation, a 32×32 DCT coefficient matrix is ​​calculated, containing all information from low to high frequencies. Then, the top-left 8×8 low-frequency components of the 32×32 DCT coefficient matrix are retained. These low-frequency components primarily represent the overall structure and basic tone of the image, while the high-frequency components reflect more details and noise.

[0093] Hash encoding refers to calculating the mean value (meanValue) of all elements in a preserved 8×8 DCT coefficient matrix. Each DCT coefficient is compared to this mean value: if a coefficient is greater than or equal to the mean value, its corresponding binary hash value is set to 1; otherwise, it is set to 0. This generates a 64-bit binary hash value (i.e., the first hash value). This hash encoding method can effectively capture the main visual features of the image being processed while maintaining low information redundancy.

[0094] Step 52: Use the first hash value as a hash index to match and retrieve the second hash value corresponding to each image in the target database.

[0095] Hash index construction refers to using binary hash values ​​generated by database storage and their corresponding image IDs to form a hash index. A hash index is a highly efficient data structure that allows lookup operations to be completed in O(1) time complexity, greatly improving retrieval efficiency. As an example, inverted indexes or other techniques suitable for large-scale data storage and querying can be used to optimize performance.

[0096] Step 53: For any image in the target database, if the hash similarity between the second hash value and the first hash value of the image is greater than or equal to the first threshold, the image is determined as a candidate image.

[0097] In this embodiment, Hamming distance can be used to measure the difference between two hash values ​​(i.e., the first hash value and the second hash value). Hamming distance refers to the number of different characters at corresponding positions between two strings of equal length, which is the count of the number of different bits between two 64-bit binary hash values. If the Hamming distance between two images is less than or equal to 5 (an empirical threshold), then the two images are considered likely to be duplicates. This allows for control of the false positive rate while maintaining a high recall rate.

[0098] It should be noted that when the Hamming distance threshold is set to 5, approximately 98% of potential duplicate images can be recalled, while the false screening rate is controlled within 2%. Thus, high efficiency is maintained while ensuring high accuracy.

[0099] It should be understood that this disclosure, through the first-level filtering (i.e., steps 51 to 53 above), can eliminate more than 90% of irrelevant images, greatly reducing the number of images requiring precise matching. Furthermore, the computational cost of the second-level filtering (i.e., multimodal feature matching) can be reduced to about 10% of the original cost, significantly improving the overall processing speed.

[0100] In one implementation of the image deduplication method provided in the above embodiments, step S23 may include: Step 61: Obtain the vector similarity between the first feature vector and the second feature vector.

[0101] Wherein, the first feature vector is the feature vector corresponding to the multimodal features of the image to be processed, and the second feature vector is the feature vector corresponding to the multimodal features of the candidate image.

[0102] In this embodiment, a vectorized query can be performed, storing the multimodal feature vectors of candidate images obtained from the first-level filtering into a vector database. To accelerate the retrieval process, the IVF_PQ (Inverted File Product Quantization) indexing strategy is adopted. IVF_PQ combines the advantages of inverted files and product quantization, enabling efficient management and searching of large-scale vector data.

[0103] As an example, a set of the top 100 closest candidate images (referred to as the candidate set) can be initially screened using coarse quantization (PQ8), and then the vector similarity of these candidate sets can be further calculated using fine quantization (PQ16).

[0104] For the image to be processed and the candidate images, cosine similarity can be used to measure the distance between their feature vectors. The formula for cosine similarity is as follows: ; Where A represents the first feature vector corresponding to the image to be processed, and B represents the second feature vector corresponding to the candidate image.

[0105] It should be understood that the above method not only takes into account the directionality of vectors, but also effectively processes high-dimensional data and provides more accurate matching results.

[0106] Step 62: If the vector similarity is greater than or equal to the second threshold, it is determined that the image to be processed and the candidate image are duplicates.

[0107] It should be understood that if the vector similarity is greater than or equal to the second threshold, it indicates that the degree of repetition between the image to be processed and the candidate image exceeds the allowed degree of repetition, that is, the image to be processed and the candidate image are repetitive.

[0108] Step 63: If the vector similarity is less than the second threshold, determine that the image to be processed and the candidate image are not duplicates.

[0109] It should be understood that if the vector similarity is less than the second threshold, it indicates that the degree of repetition between the image to be processed and the candidate image does not exceed the allowed degree of repetition, that is, the image to be processed and the candidate image do not overlap.

[0110] In one implementation of the image deduplication method provided in the above embodiments, the method may further include: dynamically updating the similarity threshold based on the first information.

[0111] The similarity threshold may include at least one of a first threshold or a second threshold.

[0112] The first piece of information includes at least one of the following: similarity data distribution, historical plagiarism detection results, data source of the image to be processed, historical plagiarism detection data, or data volume.

[0113] In one possible implementation, the distribution of similarity data can be modeled based on the data distribution, and the similarity threshold can be updated according to the distribution of similarity data.

[0114] As an example, assuming similarity scores follow a normal (Gaussian) distribution, we can use historical plagiarism detection data to estimate the mean μ and variance σ² of the similarity distribution. This process typically involves statistical analysis of similarity scores in an existing dataset to obtain accurate parameter estimates. Specifically, maximum likelihood estimation (MLE) or Bayesian estimation methods can be used to infer these parameters.

[0115] After determining the distribution of similarity data, a threshold can be calculated. As an example, based on the 3σ principle (i.e., in a normal distribution, approximately 99.7% of the data lies between the mean and three standard deviations), the initial threshold is set to μ+2σ, which covers approximately 95% of positive samples (i.e., images considered duplicates). This strategy aims to ensure that most genuine duplicate images are correctly identified while minimizing the false positive rate.

[0116] In one possible implementation, historical plagiarism detection results can be obtained in the following way: Step 71: Record the confirmation results of the plagiarism check through the feedback collection mechanism.

[0117] The confirmation result indicates whether the plagiarism check result is a true positive or a false positive.

[0118] This application embodiment can employ incremental learning technology. Through a feedback collection mechanism, the system records the user's confirmation of the plagiarism detection results. The confirmation results include true positive (TP) and false positive (FP) cases, thus providing key data on the performance of the current threshold setting.

[0119] Step 72: Use the plagiarism check results, including the confirmation results, as historical plagiarism check results.

[0120] It should be understood that after recording the user's confirmation of the plagiarism check results, the plagiarism check results including the confirmation can be used as historical plagiarism check results, so as to facilitate subsequent adjustments to the similarity threshold.

[0121] In one possible implementation, based on historical plagiarism detection results, the mean μ and variance σ² can be dynamically adjusted using the Exponentially Weighted Moving Average (EWMA) method: ;

[0122] Here, α=0.9 is a decay factor used to balance the influence of new and old data, ensuring that the model can respond quickly to the latest changes without completely ignoring long-term accumulated experience.

[0123] In one possible implementation, the data source of the image to be processed can be obtained based on contextual feature extraction, meaning the system can automatically switch threshold strategies according to the data source of the image. For example, for highly standardized image types with relatively simple backgrounds, such as medical images, a stricter threshold can be applied; while for diverse images on social media, a more lenient standard may be needed to improve recall.

[0124] In one possible implementation, historical deduplication data can be determined using a sliding window mechanism, and the similarity threshold can be updated using this historical data. As an example, using data from the most recent 10,000 queries (i.e., historical deduplication data) to update the similarity threshold is an example of the sliding window mechanism. The sliding window mechanism helps ensure the freshness (i.e., timeliness) of the similarity threshold estimation, avoiding deviations in the similarity threshold due to outdated data.

[0125] One possible implementation is to use a cold start strategy to update the similarity threshold based on the amount of data available. For example, when there is insufficient data in the early stages of the system, a fixed threshold θ=0.8 can be used as a temporary solution. As more user feedback accumulates, the system gradually transitions to an adaptive mode based on real-time data analysis.

[0126] In one possible implementation, threshold boundary constraints can be set for the similarity threshold. Specifically, to prevent system failure due to improper threshold setting in extreme cases, upper and lower limits are set for the threshold (e.g., 0.6 ≤ θ ≤ 0.95). This ensures that the system's decisions remain reasonable and reliable even when the data distribution changes significantly.

[0127] It should be understood that the dynamic adjustment of the similarity threshold provided in the above embodiments, through A / B testing (i.e., A / B testing was conducted on a database containing more than 1 million images), showed that the F1-score of the deduplication scheme using the adaptive threshold reached 92.5%, which is 7.2 percentage points higher than the fixed threshold scheme. Simultaneously, the test results also indicate that the online parameter update process takes less than 1 millisecond, proving that the algorithm can support efficient operation in high-concurrency environments without affecting user experience.

[0128] It should be understood that the embodiments of this application use a method based on Bayesian statistics and online adaptive adjustment mechanism to dynamically set and adjust the image similarity threshold in order to improve the accuracy and robustness of the plagiarism detection results. In particular, incremental learning, parameter update formula, scene-aware optimization and sliding window mechanism are used to ensure the real-time response capability and stability of the system.

[0129] Based on the image deduplication method provided in the above embodiments, one implementation of the example combines... Figure 4 As shown in the illustration, this application also provides a flowchart of another image plagiarism detection method. Combined with... Figure 4 As shown, the image deduplication method provided in this application embodiment may include a first-level filter and a second-level filter.

[0130] The implementation process of the first-level filtering is as follows: First, image preprocessing is performed on the image to be processed; then, DCT transformation is performed on the preprocessed image to be processed; then, the DCT transformation matrix obtained by the DCT transformation is hashed to obtain the first hash value; further, a hash index is constructed on the first hash value to obtain the hash index; then, the similarity is calculated based on the hash index and the second hash value corresponding to each image in the target database; it is determined whether the similarity value is greater than or equal to the first threshold. If the similarity threshold is greater than or equal to the first threshold, the process ends; otherwise, the second-level filtering is entered.

[0131] The second-level filtering process is as follows: feature retrieval is performed on the target database to obtain the multimodal features corresponding to the candidate images; then, the similarity between the multimodal features corresponding to the candidate images and the multimodal features corresponding to the images to be processed is calculated (i.e., the similarity between the first feature vector and the second feature vector is calculated); then, it is determined whether the vector similarity is greater than or equal to the second threshold. If it is greater than or equal to the second threshold, it is determined that the two are duplicates; otherwise, they are not duplicates.

[0132] It should be understood that the embodiments of this application propose a dual-filtering mechanism, which significantly improves the accuracy and efficiency of image deduplication through a multi-level filtering strategy. The dual-filtering mechanism includes a first-level perceptual hash fast filtering and a second-level multimodal feature precise matching, each stage possessing unique technical advantages and value. Perceptual hash fast filtering can complete the initial screening in a very short time, greatly reducing the number of images requiring further processing. Experiments show that this step can eliminate more than 90% of irrelevant images, thereby significantly reducing the computational burden. Hamming distance measures the difference between two hash values ​​and can effectively identify structurally similar images, even if these images have undergone rotation, scaling, or slight color adjustments. This robustness allows the system to maintain high recognition accuracy even when faced with complexly edited images.

[0133] Based on the image deduplication method provided in the above embodiments, one example implementation is as follows: Figure 5 As shown in the diagram, this application also provides a structural schematic diagram of an image plagiarism detection device. See also... Figure 5 The image plagiarism detection device provided in this application includes six modules, as shown in the table below: Table 1: Functions and Business Logic of the Six Modules of the Image Plagiarism Detection Device

[0134] The retrieval request module serves as the user interface for the entire system, responsible for receiving image plagiarism checks from users. This module typically includes a web interface or API, allowing users to initiate plagiarism checks by uploading image files or providing image URLs. Internally, the retrieval request module contains three sub-modules: a video frame extractor, an image cropper, and an image enhancer, used for preprocessing the input file through video frame extraction, image cropping, and image enhancement.

[0135] The feature generation module is responsible for multimodal feature extraction from the input image. This module includes built-in text feature extractors, time-frequency feature extractors, color feature extractors, and object feature extractors. Using the image from the feature processing module, the feature generation module generates text features, time-frequency features, color features, and object features, which are then fed into the feature processing module for feature retrieval.

[0136] The feature processing module is responsible for comparing the image feature vector generated by the feature generation module with the feature library of the feature storage module, calculating the similarity between the input image vector and the stored vector, and determining whether the images are duplicates based on the image similarity threshold.

[0137] The feature processing module includes sub-modules such as a feature extractor, a feature retrieval unit, a feature threshold adjuster, and a retrieval result generator. The feature extractor acquires the photos sent by the retrieval request module, caches them, and sends them to the feature generation module to obtain the multimodal feature vectors of the images. The feature retrieval unit sends the feature vectors to the feature storage module to efficiently check whether the vectors have been matched within the feature storage module. The feature threshold adjuster is used to adjust the similarity threshold based on the initial information.

[0138] The feature storage module is responsible for storing and managing the feature vectors of the image. This module typically includes a feature database and a feature index, supporting efficient feature retrieval and management. The feature database can use a relational database (such as MySQL) or a NoSQL database (such as MongoDB), and the feature index can use an inverted index or a hash index.

[0139] The file storage module is responsible for storing and managing users' image files. It includes a file server and a file index, supporting efficient file retrieval and management. The file server can use a local file system or a cloud storage service (such as Amazon S3), and the file index can use file metadata or hash values.

[0140] The result generation module is responsible for generating and returning plagiarism detection results. It includes a result processor and a result cache, supporting efficient management and response of plagiarism detection results. The result processor formats and optimizes the plagiarism detection results generated by the feature generation module, while the result cache stores the most recent results, improving response speed.

[0141] Figure 6 This is a structural block diagram of an image plagiarism detection device provided in an embodiment of the present disclosure, such as... Figure 6 As shown, the image plagiarism detection device 600 includes: a retrieval module 610, an acquisition module 620, and a plagiarism detection module 630.

[0142] The retrieval module 610 is used to perform duplicate retrieval in the target database based on the image features of the image to be processed, and obtain candidate images; The acquisition module 620 is used to acquire multimodal features of the image to be processed, including at least two of the following: text features, time-frequency features, color features, or object features. The plagiarism detection module 630 is used to perform plagiarism analysis on the image to be processed and the candidate images based on multimodal features, and obtain the plagiarism detection results.

[0143] Optionally, if the multimodal features include the text features, the acquisition unit is used to: Detect text regions in the image to be processed; The text region is subjected to text recognition to obtain the text content; The text content is encoded to obtain the text features.

[0144] Optionally, if the multimodal features include the time-frequency features, the acquisition unit is used to: The image to be processed is subjected to frequency domain transformation to obtain the transform coefficient matrix; A one-dimensional vector is determined based on the transformation coefficient matrix; The time-frequency feature is obtained by normalizing the one-dimensional vector.

[0145] Optionally, if the multimodal features include the color features, the acquisition unit is used to: The image to be processed is spatially segmented to obtain multiple sub-regions; Color histogram quantization is performed on each of the sub-regions to obtain the color histogram corresponding to each of the sub-regions; The color features are obtained by performing feature fusion processing on multiple color histograms.

[0146] Optionally, if the multimodal features include the object features, the acquisition unit is used to: The object features are determined based on the image to be processed using a pre-trained feature extraction model; wherein the pre-trained feature extraction model includes a spatial attention module and a channel attention module, and the pre-trained feature extraction model is obtained by optimizing the feature extraction model based on an attention mechanism.

[0147] Optionally, the retrieval unit is used for: The image to be processed is hash-encoded to obtain a first hash value; The first hash value is used as a hash index to match and retrieve the second hash value corresponding to each image in the target database. For any image in the target database, if the hash similarity between the second hash value and the first hash value of the image is greater than or equal to the first threshold, the image is determined as the candidate image.

[0148] Optionally, the plagiarism detection module is used for: Obtain the vector similarity between the first feature vector and the second feature vector; wherein, the first feature vector is the feature vector corresponding to the multimodal features of the image to be processed, and the second feature vector is the feature vector corresponding to the multimodal features of the candidate image; If the vector similarity is greater than or equal to the second threshold, it is determined that the image to be processed and the candidate image are duplicates. If the vector similarity is less than the second threshold, it is determined that the image to be processed and the candidate image are not duplicates.

[0149] Optionally, the device further includes: An update module is used to dynamically update a similarity threshold based on first information; wherein the similarity threshold includes at least one of the following: a first threshold and a second threshold; The first information includes at least one of the following: Distribution of similarity data; Historical plagiarism check results; The data source of the image to be processed; Historical plagiarism detection data; Data volume.

[0150] Optionally, the update module is used to obtain historical plagiarism detection results in the following ways: The confirmation results of the plagiarism detection are recorded through a feedback collection mechanism; wherein the confirmation results indicate whether the plagiarism detection results are true positive or false positive. The plagiarism check results, including the confirmation results, are used as the historical plagiarism check results.

[0151] This application also provides an electronic device for performing the above-described image deduplication method. Please refer to... Figure 7 It illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 7 As shown, the electronic device 70 includes: a processor 700, a memory 701, a bus 702, and a communication interface 703. The processor 700, the communication interface 703, and the memory 701 are connected via the bus 702. The memory 701 stores a computer program that can run on the processor 700. When the processor 700 runs the computer program, it executes the image deduplication method provided in any of the foregoing embodiments of this application.

[0152] The memory 701 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between the device network element and at least one other network element is achieved through at least one communication interface 703 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0153] Bus 702 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory 701 is used to store programs. After receiving an execution instruction, the processor 700 executes the program. The image deduplication method disclosed in any of the foregoing embodiments of this application can be applied to the processor 700, or implemented by the processor 700.

[0154] The processor 700 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 700 or by instructions in software form. The processor 700 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 701. Processor 700 reads the information in memory 701 and, in conjunction with its hardware, completes the steps of the above method.

[0155] The electronic device provided in this application embodiment and the image deduplication method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0156] This application also provides a computer-readable storage medium corresponding to the image deduplication method provided in the foregoing embodiments. The computer-readable storage medium shown can be an optical disc, on which a computer program is stored. When the computer program is run by a processor, it executes the image deduplication method provided in any of the foregoing embodiments.

[0157] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0158] The computer-readable storage medium provided in the above embodiments of this application and the image deduplication method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0159] This application also provides a computer program product 800, such as... Figure 8 As shown. This computer program product carries a computer program 801, the instructions of which can be used to execute the steps of the image deduplication method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0160] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0161] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0162] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0163] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0164] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0165] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0166] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0167] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An image plagiarism detection method, characterized in that, include: Based on the image features of the image to be processed, a deduplication search is performed in the target database to obtain candidate images; Obtain the multimodal features of the image to be processed, wherein the multimodal features include at least two of the following: text features, time-frequency features, color features, or object features; Based on the multimodal features, a deduplication analysis is performed on the image to be processed and the candidate images to obtain the deduplication results.

2. The method according to claim 1, characterized in that, If the multimodal features include the text features, obtaining the multimodal features of the image to be processed includes: Detect text regions in the image to be processed; The text region is subjected to text recognition to obtain the text content; The text content is encoded to obtain the text features.

3. The method according to claim 1, characterized in that, If the multimodal features include the time-frequency features, obtaining the multimodal features of the image to be processed includes: The image to be processed is subjected to frequency domain transformation to obtain the transform coefficient matrix; A one-dimensional vector is determined based on the transformation coefficient matrix; The time-frequency feature is obtained by normalizing the one-dimensional vector.

4. The method according to claim 1, characterized in that, If the multimodal features include the color features, obtaining the multimodal features of the image to be processed includes: The image to be processed is spatially segmented to obtain multiple sub-regions; Color histogram quantization is performed on each of the sub-regions to obtain the color histogram corresponding to each of the sub-regions; The color features are obtained by performing feature fusion processing on multiple color histograms.

5. The method according to claim 1, characterized in that, If the multimodal features include the object features, obtaining the multimodal features of the image to be processed includes: The object features are determined based on the image to be processed using a pre-trained feature extraction model; wherein the pre-trained feature extraction model includes a spatial attention module and a channel attention module, and the pre-trained feature extraction model is obtained by optimizing the feature extraction model based on an attention mechanism.

6. The method according to claim 1, characterized in that, The image features of the image to be processed are used to perform deduplication retrieval in the target database to obtain candidate images, including: The image to be processed is hash-encoded to obtain a first hash value; The first hash value is used as a hash index to match and retrieve the second hash value corresponding to each image in the target database. For any image in the target database, if the hash similarity between the second hash value and the first hash value of the image is greater than or equal to the first threshold, the image is determined as the candidate image.

7. The method according to claim 1, characterized in that, The step of performing a deduplication analysis on the image to be processed and the candidate images based on the multimodal features to obtain the deduplication results includes: Obtain the vector similarity between the first feature vector and the second feature vector; wherein, the first feature vector is the feature vector corresponding to the multimodal features of the image to be processed, and the second feature vector is the feature vector corresponding to the multimodal features of the candidate image; If the vector similarity is greater than or equal to the second threshold, it is determined that the image to be processed and the candidate image are duplicates. If the vector similarity is less than the second threshold, it is determined that the image to be processed and the candidate image are not duplicates.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: Based on the first information, the similarity threshold is dynamically updated; wherein the similarity threshold includes at least one of the following: a first threshold and a second threshold; The first information includes at least one of the following: Distribution of similarity data; Historical plagiarism check results; The data source of the image to be processed; Historical plagiarism detection data; Data volume.

9. The method according to claim 8, characterized in that, The historical plagiarism detection results were obtained in the following ways: The confirmation results of the plagiarism detection are recorded through a feedback collection mechanism; wherein the confirmation results indicate whether the plagiarism detection results are true positive or false positive. The plagiarism check results, including the confirmation results, will be used as the historical plagiarism check results.

10. An image plagiarism detection device, characterized in that, include: The retrieval module is used to perform duplicate retrieval in the target database based on the image features of the image to be processed, and obtain candidate images; The acquisition module is used to acquire the multimodal features of the image to be processed, wherein the multimodal features include at least two of the following: text features, time-frequency features, color features, or object features; The plagiarism detection module is used to perform plagiarism analysis on the image to be processed and the candidate images based on the multimodal features, and obtain the plagiarism detection results.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by a processor to implement the method as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the method as described in any one of claims 1-9.

Images