Open world image provenance forensics method based on suspicious set constructing self-training SVM

Abstract

This invention relates to an open-world image source forensics method based on a self-trained SVM constructed from a suspicious set, belonging to the field of computer image processing technology. The technical solution involves dividing images into training and testing sets; extracting CFA features from the training and testing sets respectively; inputting the CFA features into an SVDD classifier for coarse classification, resulting in known and unknown classes; further dividing the unknown classes into a suspicious set and several unknown classes; and then performing a self-trained SVM on the known, suspicious, and unknown classes to output the results. The beneficial effects are: this invention achieves coarse classification of known and unknown sources through hypercursive breadth optimization in the feature space; it utilizes multiple distance metrics for clustering fusion based on ensemble discriminant thinking and a self-trained SVM classifier for iterative allocation and updating based on transductive learning; it solves the problem of image source forensics in open worlds; and it can adapt to constantly updated camera models on the market, effectively utilizing and learning unknown source information to enhance classifier performance.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically the field of computer image processing technology, and particularly relates to an open-world image source forensics method based on a self-trained SVM constructed from a suspicious set. Background Technology

[0002] With the booming development of the camera manufacturing industry, the brands and types of cameras on the market are constantly changing, making it difficult for forensic analysts to keep track of all brands and models available in real time. In traditional closed environments, camera provenance identification is often considered a multi-class classification problem with a defined number of categories. If the image to be identified was taken by an unknown camera outside the training set, traditional closed-environment classification methods will incorrectly classify the image as belonging to one of the known source cameras. In contrast to traditional closed environments, digital image provenance identification in open environments allows the test image to come from an unknown class outside the known dataset. The introduction of the "unknown class" greatly reduces the misclassification problem of the classifier. Therefore, digital image provenance identification in open environments is considered a multi-class classification problem with an uncertain number of categories.

[0003] Several key approaches to image source forensics in open environments:

[0004] 1. Decision boundary sculpting [1] Images from known models are treated as positive samples, and images from other known models are treated as negative samples. Then, a binary support vector machine is trained to distinguish between positive and negative samples by adjusting the decision boundary to minimize false positive matches.

[0005] 2. K-Nearest Neighbors Algorithm [2] Unknown source images are extracted from the test set and separated using the K-nearest neighbor criterion. These images are then used to train an N+1 class SVM. This training and separation process is repeated to further separate unknown source images from the test set.

[0006] 3. Confidence threshold algorithm [3] A convolutional neural network is used to extract camera model recognition features, mapping deep features to confidence scores to distinguish whether an image was captured by the same or different camera models. If the confidence score is below a threshold, the query image is identified as unknown.

[0007] To address image source forensics extending from open environments to the open world, specifically the forensics of unknown samples involved in K+1 to K+k classification problems, the following techniques offer insights:

[0008] 1. Integrated learning [4]To enhance the stability and reliability of the model, a new classifier is formed by constructing a system of multiple weak classifiers. Through effective combination strategies, the resulting strongly supervised model exhibits more comprehensive and superior performance.

[0009] 2. Direct learning [5] Treat the unlabeled sample set as the test set, and obtain the best generalization performance by constructing a suspicious set on this unlabeled set.

[0010] Traditional image source forensics operates in closed environments (assuming the training and test sets maintain consistent data categories). In these closed environments, trained image source forensics models can effectively identify the learned categories to trace the source. However, with rapid technological advancements and the continuous updates of camera models, closed-scene source forensics models can no longer encompass all available camera models, leading to misclassification and rendering them unsuitable for current environments. In recent years, researchers have focused on this issue and proposed image source forensics for open environments. However, these algorithms are limited in that they only identify and exclude samples of unknown categories as errors to ensure good forensic performance for images within the original training set. Open environments struggle to learn and utilize constantly updated, unknown image categories.

[0011] [1]Costa FO,Silva E,Eckmann M,et al.Open set source cameraattribution and device linking[J].Pattern Recognit Letters,2014,39:92-101.

[0012] [2]HuangY G, Zhang J, HUANG HY.Camera model identification with unknownmodels[J]. IEEE Transactions on Information Forensics and Security, 2015,10:2692-2704.

[0013] [3]BAYAR B,STAMM M.Towards open set camera model identification using a deep learning framework[C].2018IEEE International Conference on Acoustics,Speech and Signal Processing,CalgaryAB,2018:2007-2011.

[0014] [4]Gretton,A.,Borgwardt,KM,Rasch,MJ, B.,&Smola,A.(2012).Akernel two-sample test.Journal ofMachine Learning Research,13(Mar),723-773.

[0015] [5]Xiao Y, Feng J, Liu BA new transductive learning method with universal data[J].Applied Intelligence, 2021(6245):1-13. Summary of the Invention

[0016] To address the problem of updating and iterating unknown categories, and to learn and utilize them, this invention introduces the concept of an open world. The main steps under this concept are: identifying unintroduced objects as "unknown" without explicit supervision; progressively learning these identified unknown categories as corresponding labels are received, without forgetting previously learned categories; and to maximize this progressive learning capability, this invention constructs a suspicious set based on ensemble learning and iteratively assigns it, enabling the model to gradually acquire the ability to investigate the source of unknown categories.

[0017] To achieve the above objectives, the present invention provides the following technical solution:

[0018] An open-world image source forensics method based on a suspicious set to construct a self-trained SVM, comprising the following steps:

[0019] S1. Divide the images into training and testing sets;

[0020] S2. Extract CFA features from the training and test sets shown respectively;

[0021] S3. Input the CFA features of the above two into the SVDD classifier for coarse classification, classifying them into known classes and unknown classes;

[0022] S4. Divide the unknown class into a suspicious set and several unknown classes;

[0023] S5. Perform self-training SVM on the known classes, the suspicious set, and several unknown classes to output the results.

[0024] Furthermore, in step S2, CFA interpolation weight coefficients based on the covariance matrix are used to extract image sample features. For an interpolation point (x, y) in the G color channel, the neighborhood of (2k+1)·(2k+1) of that point is selected, and interpolation is performed on the three channels to calculate the pixel value of that point:

[0025]

[0026] Suppose we need to interpolate N pixels in the G channel, we get:

[0027]

[0028] Equation 2 can be represented by eigenvectors:

[0029]

[0030] To estimate the weights of the interpolation coefficients, a covariance matrix is ​​introduced into Equation 3:

[0031]

[0032] This allows us to obtain the interpolation coefficient vector a. g for:

[0033]

[0034] For the interpolation coefficients a of the other two color channels r and a b Solve using the same method as described above.

[0035] Furthermore, choosing k=3, the neighborhood size of the interpolation point is calculated to be 2k+1=7; based on the Bayer CFA structure, the interpolation coefficients of the G channel of the R and B sampling points in the neighborhood and the R and B interpolation coefficients of the two G sampling points are estimated, resulting in a total interpolation coefficient of ((2k+1)). 2 -1)×5=240; take multiple non-overlapping 64×64 block regions from the image to estimate the interpolation coefficients, calculate their variance and mean, and thus obtain a total of 240×2=480 dimensions of features for training and testing.

[0036] Furthermore, in step S3, the k-means clustering algorithm is used to cluster the original category camera models into k subclasses; the elbow method is applied to determine the optimal number of clusters for the k-means algorithm; the core metric of the elbow method is the sum of squared errors, and its expression is:

[0037]

[0038] SSE and k form a relationship diagram similar to an elbow, where the k value corresponding to the elbow joint is the true number of clusters.

[0039] Furthermore, in step S4, after obtaining the set of unknown classes, a suspicious set is constructed in the set of unknown classes obtained in the coarse classification by using several distances as clustering conditions and combining the idea of ​​ensemble decision, and the initial clustering of camera sample data from unknown sources is obtained.

[0040] Furthermore, the distance includes Euclidean distance, used to measure the absolute distance between two points in n-dimensional Euclidean space; in two-dimensional space, Euclidean distance is used to represent the straight-line distance between point a(x1,x2) and point b(y1,y2).

[0041]

[0042] In the CFA feature space, there are two 480-dimensional vectors a(x1, x2, ..., x...). n ) and b(y1,y2,…,y n The Euclidean distance between them can be expressed as:

[0043]

[0044] By standardizing the coordinates within the intervals where the variance and mean are equal through weighted calculation, the changes in different coordinate components correspond to their weights, thus obtaining the standardized Euclidean distance in the CFA feature space.

[0045]

[0046] Furthermore, the distance includes the bulldozer distance, assuming there are two discrete mounds P with the same mass but different distribution patterns. r and P θ If the mound P r Convert to P θ Work needs to be done on it, and the work done is equal to the mass of a unit of soil multiplied by the distance of work. If we want to complete the transformation of the soil pile shape with the minimum amount of work, we need to optimize the total transportation cost.

[0047] x and y represent the mound P, respectively. r and P θ The positional independent variable is set to P for each bar. r(x) and P θ (y), P r (x) represents the soil inventory at position x, P θ (x) represents the amount of soil to be stored after the location of x changes; assuming the joint distribution γ(x,y) represents the amount of soil to be transported from x to y, its marginal distribution is P r and P θ If the work required to transport each unit of soil from location x to location y is d(x,y), then the bulldozer distance can be defined as:

[0048]

[0049] The bulldozer distance is transformed into a minimum optimization problem; the marginal distribution of γ(x,y) and γ(x,y)≥0 together constitute the constraints of this optimization problem. Expanding Equation 10 yields:

[0050]

[0051] Rewrite Γ and D in matrix form:

[0052]

[0053]

[0054] Let the marginal distribution of the joint distribution γ(x,y) be P r and P θ Expand to summation form:

[0055]

[0056]

[0057] Combined with formulas 14 and 15, P r (x) and P θ (y) concatenates to obtain the long vector b, then the constraint condition can be simplified to AΓ = b, where:

[0058] b = [P] r (x1),P r (x2),…,P r (x n )|P θ (y1),P θ (y2),…,P θ (y n )] T (16)

[0059]

[0060] The bulldozer distance problem can be abstracted as a minimum optimization problem under linear constraints, and expressed as:

[0061]

[0062] Furthermore, in step S5, after obtaining the preliminary set distribution results by constructing a suspicious set, the next step is to construct a self-trained SVM classifier based on the idea of ​​push learning, and complete the division of the camera model from unknown sources through multiple rounds of iteration; the data set with "known" labels and the data set with "unknown" labels obtained from the coarse classification are combined into a new training set and corresponding labels and fed into the SVM classifier for self-training.

[0063] Furthermore, in the s-th iteration, the following operation is performed:

[0064] (1) Assume the dataset labeled "Known" is Data sets labeled "unknown" are set as follows: in k is the number of unknown sample clusters calculated using the elbow method; let k be the set of data marked as unknown camera models in the coarse classification results. Then the suspicious set

[0065] (2) Note and The union of is And used as a new training set to train an SVM classifier C s ;

[0066] (3) Using C s right The classification is performed, and the sum of the images labeled as "known" and the "unknown" class samples with an assigned label "unknown t" is denoted as .

[0067] (4) and Update;

[0068] Furthermore, an iterative assignment method is used during the training of the self-trained SVM classifier: where color represents the label assigned to the sample by the classifier, and shape represents the true label of the sample; the specific iterative assignment criteria are as follows:

[0069] (1) In the first iteration, the set of suspicious samples is fed into a self-trained SVM classifier, which is trained by a dataset of known samples obtained through coarse classification and k unknown samples.

[0070] (2) Sort the data and labels obtained by the SVM classifier output according to the number of samples; add the category with the most identical sample labels to the corresponding category set in the training set of the self-trained SVM classifier;

[0071] (3) If, after sorting the prediction results, it is found that the number of labels of the first-ranked category is ≥1, then in this round, the samples with the same number of labels in all classification results will be assigned and added to the corresponding training set category of the self-trained SVM classifier for training, and a new classifier will be obtained.

[0072] (4) Judge the suspicious set after the sample allocation. If the suspicious set is empty, it means that all suspicious samples have been allocated and the iteration stops. If the suspicious set is not empty, the remaining samples are set as suspicious labels to form a new suspicious set and put into the new round of self-trained SVM classifier obtained in step (3) for classification.

[0073] (5) Repeat steps (2) to (4) until the iteration stops, and output the data and labels in the self-trained SVM classifier as the final result.

[0074] The beneficial effects of this invention are:

[0075] Compared with existing technologies, the open-world image source forensics method based on a suspected set to construct a self-trained SVM provides by using hypercursive breading optimization in the feature space to achieve coarse classification of known and unknown sources. It employs multi-distance metric clustering fusion based on ensemble discriminant thinking and an iteratively updated self-trained SVM classifier based on transductive learning. This method solves the problem of image source forensics in open worlds. The adaptively trained classification framework established by this method outperforms existing open-environment image source forensics methods and can adapt to constantly updated camera models on the market, effectively utilizing and learning unknown source information to enhance classifier performance.

[0076] This invention is of great significance in the following aspects:

[0077] 1. Verifying Image Authenticity: In open-world environments, updated images that have been edited, altered, or forged are difficult to detect using existing methods. By conducting open-world image source forensics using this method, the authenticity of an image can be verified, and it can be determined whether it has been modified or edited. This is particularly important in areas such as legal proceedings, news reporting, and social media.

[0078] 2. Protecting Intellectual Property: For creators and intellectual property holders, the image capture equipment they use is market-leading. Open-world image source forensics can provide evidence to protect their rights. If they discover that others are using their images without authorization, they can use image source forensics to prove that they are the original creators or have legal authorization.

[0079] 3. Identifying Misinformation and Online Fraud: In open worlds, misinformation and online fraud based on constantly updated images are rampant. Open world image source forensics can help investigators identify and reveal the sources of misinformation and track online fraud activities. This is crucial for combating online fraud and protecting the public from misinformation.

[0080] 4. Supporting Criminal Investigations: Open-world image source forensics plays a crucial role in criminal investigations. By tracing the source of images, it is possible to keep pace with technological advancements and identify key information such as suspects, locations, and times. This is of great significance for solving criminal cases, tracking criminal activity, and ensuring judicial fairness. Attached Figure Description

[0081] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0082] Figure 1 This is a diagram illustrating image source forensics in an open environment.

[0083] Figure 2 A diagram illustrating image source forensics in an open world;

[0084] Figure 3 A diagram illustrating an open-world image source identification framework for constructing a self-trained SVM based on a suspicious set;

[0085] Figure 4 For adaptive clustering envelope graph;

[0086] Figure 5 This is a schematic diagram of discrete soil pile conversion;

[0087] Figure 6 Schematic diagram of work done on a discrete soil pile;

[0088] Figure 7 Constructing a diagram for a suspicious set;

[0089] Figure 8 This is a schematic diagram of a self-trained SVM classifier;

[0090] Figure 9 A schematic diagram of the iterative allocation rule;

[0091] Figure 10 This is a graph showing the performance comparison between the method of the present invention and existing methods;

[0092] Figure 11 The figure shows the experimental results of the method of the present invention in an open world (T is the number of unknown categories). Detailed Implementation

[0093] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The following description, in conjunction with the accompanying drawings... Figure 1-11 This paper further explains the open-world image source forensics method based on a self-trained SVM constructed from a suspicious set.

[0094] This invention belongs to the field of artificial intelligence. Its primary application in AI is in image source forensics within open environments and their extensions, the open world. Considering the rapid pace of updates and iterations in image capture devices such as cameras and mobile phones, traditional image source forensics models cannot effectively utilize, learn, and classify newly introduced, unknown categories. Existing methods, when encountering unknown categories during testing, either misclassify or simply exclude unknown category information, resulting in low sample utilization and rendering their training methods infeasible. Therefore, this invention uses a self-trained unsupervised learning framework and constructs a suspicious set to implement a self-trained SVM classification model. This model can effectively fit the data sample distribution of samples confused with unknown categories.

[0095] This invention presents an algorithm for training an SVM regression model by constructing suspicious sets using multiple clustering metrics, and establishes a self-learning training framework through iterative allocation criteria. This self-learning training framework is well-suited for image source forensics in open-world environments. The specific differences between open-environment and open-world image source forensics are as follows: Figure 1 , 2 As shown.

[0096] Current research on open-world image source forensics primarily focuses on open environments. However, faced with numerous and novel unknown camera models, simply abstracting them into an unknown class and discarding them to ensure good performance of traditional closed-scene source forensics models is insufficient to advance research in open-world image source forensics. To learn from and classify these constantly updated unknown camera models, this paper introduces ensemble learning and transductive learning to construct a suspicious set for training an SVM classifier in open-world image source forensics. This creates an adaptive learning training framework, ultimately resulting in an open-world image source forensics algorithm based on a self-trained SVM built from the suspicious set. The basic framework of the algorithm is as follows: Figure 3 As shown.

[0097] The following details the algorithm:

[0098] I. Explanation of Symbols

[0099] G (x,y) : Pixel value of point (x,y)

[0100] G i The i-th pixel in the G channel that needs interpolation.

[0101] Represents the interpolation coefficients in the R, G, and B channels.

[0102] m x+i,y+j (m=r,g,b): Represents the neighborhood interpolation coefficients of the point (x,y) in the three channels.

[0103] This represents the s-th interpolation coefficient in the neighborhood of the k-th pixel in the R, G, and B channels.

[0104] cov(·): indicates the calculation of covariance.

[0105] k: represents the total number of internal clusters determined by the feature space distribution of the data for this type of camera model.

[0106] C i : Represents the i-th subset of datasets belonging to this class of trained camera models.

[0107] SSE: Represents the clustering error of all samples, used to characterize the quality of clustering.

[0108] s i : Represents the standard deviation of each component

[0109] inf: indicates taking the infunc of a bound.

[0110] II. Algorithm Description

[0111] This method extracts image sample features by estimating CFA interpolation weight coefficients based on the covariance matrix. For an interpolation point (x, y) in the G color channel, by selecting a neighborhood of (2k+1)·(2k+1) of that point and interpolating across the three channels, the pixel value of that point can be calculated as follows:

[0112]

[0113] Suppose we need to interpolate N pixels in the G channel, we get:

[0114]

[0115] Equation 2 can be represented by eigenvectors:

[0116]

[0117] To estimate the weights of the interpolation coefficients, a covariance matrix is ​​introduced into Equation 3:

[0118]

[0119] This allows us to obtain the interpolation coefficient vector a. g for:

[0120]

[0121] For the interpolation coefficients a of the other two color channels r and a b The same method is used to solve this problem. Since the color digital image has three channels (R, G, B), k = 3 is chosen, and the neighborhood size of the interpolation point is calculated to be 2k + 1 = 7. Based on the Bayer CFA structure, the interpolation coefficients of the G channel at the R and B sampling points within the neighborhood, as well as the R and B interpolation coefficients of two G sampling points, are estimated, resulting in a total interpolation coefficient of ((2k + 1)). 2 -1)×5=240. To increase the reliability of interpolation estimation, multiple non-overlapping 64×64 block regions were taken from the image to estimate the interpolation coefficients, and their variance and mean were calculated. Thus, a total of 240×2=480 dimensions of features were obtained for training and testing.

[0122] After extracting CFA features from the images, high-dimensional data clustering preprocessing is required to characterize the envelope of the camera model. Clustering is a data mining task aimed at effectively processing datasets. This method uses the k-means clustering algorithm to cluster the original camera models into k subclasses. Existing k-means clustering methods do not specify the number of clusters, and manually setting the number of clusters often fails to yield optimal results. Therefore, this method uses the elbow method to determine the optimal number of clusters for the k-means algorithm. The core metric of the elbow method is the sum of squared errors, expressed as:

[0123]

[0124] Since k is positively correlated with the degree of cluster aggregation, the SSE decreases significantly when k is less than the actual number of clusters; however, when k approaches the actual number of clusters, the aggregation reward brought by increasing k decreases sharply, so the SSE tends to stabilize as k increases. At this point, a relationship diagram similar to an elbow is formed between SSE and k, where the k value corresponding to the elbow joint is the actual number of clusters.

[0125] After clustering preprocessing of the data mapped to the 480-dimensional feature space, this method performs envelope characterization for each class of camera model sub-dataset. As a type of single-class classifier, SVDD (hypersurface envelope) is widely used by characterizing the minimum volume hypersurface of the boundary of positive class samples. However, the sample distribution in the feature space is often complex and unpredictable, and SVDD often fails to adequately describe boundary samples in these cases.

[0126] Therefore, this method proposes to pre-divide the positive class samples according to the feature distribution of different categories, determine the number of clusters using the elbow method, and divide the positive class samples into multiple sub-class datasets using the k-means clustering method. Then, a hypersurface is constructed for the boundary data of each sub-class, forming k sub-envelopes. Since the data within the sub-envelopes belonging to the same camera model have the same true labels, multiple sub-envelopes can be merged to form a new hypersurface. At this point, this envelope covers more detailed features of the category compared to simple SVDD, thus better rejecting unknown samples while ensuring the recall rate of positive class samples. Figure 4 As shown, when k equals 2, the two hypersurfaces constructed provide more detailed descriptions of the boundary features compared to the previous hypersurface, greatly reducing misclassification.

[0127] After hyperbola mapping, the data and their true labels are fed into training to obtain a preliminary self-trained SVM classifier. This classifier is then used to test the dataset and perform coarse classification, dividing it into a known source set and an unknown source set. After obtaining the unknown source set, multiple distances are used as clustering conditions, and an ensemble decision approach is combined to construct a suspicious set from the unknown source set obtained in the coarse classification, thus obtaining the initial clustering of the camera sample data from unknown sources.

[0128] The essence of clustering algorithms is to group the most similar data objects into the same group, separating different data into different groups, so that data objects in the same group have the highest similarity, while data objects in different groups have the lowest similarity. The concept of similarity mainly relies on the attribute values ​​describing the data objects, which usually means distance metrics. This method introduces the idea of ​​ensemble learning in the construction of suspicious sets, using different distance metrics for clustering to obtain different clustering results.

[0129] First, there's Euclidean distance, also known as Euclidean distance, one of the most common distance metrics. Physically, it measures the absolute distance between two points in n-dimensional Euclidean space. In two-dimensional space, Euclidean distance represents the straight-line distance between point a(x1,x2) and point b(y1,y2):

[0130]

[0131] Because image samples need to be mapped to a 480-dimensional feature space, the CFA feature space contains two 480-dimensional vectors a(x1,x2,…,x…). n ) and b(y1,y2,…,y n The Euclidean distance between them can be expressed as:

[0132]

[0133] To avoid the influence of inherent random fluctuations in the coordinates on the Euclidean distance, the coordinates are weighted and standardized within intervals where the variance and mean are equal. This ensures that the changes in different coordinate components correspond to their weights, thus obtaining the standardized Euclidean distance in the CFA feature space.

[0134]

[0135] Finally, there's the bulldozer distance. For example... Figure 5 As shown, assume there exist two discrete soil piles P with the same mass but different distribution patterns. r and P θ If the mound P is... r Convert to P θWork needs to be done on it, and the work done is equal to the mass of a unit of soil multiplied by the distance of work. If we want to complete the transformation of the soil mound shape with the minimum amount of work, we need to optimize the total transportation cost.

[0136] like Figure 6 As shown, x and y represent the soil mound P. r and P θ The positional independent variable is set to P for each bar. r (x) and P θ (y), P r (x) represents the soil inventory at position x, P θ (x) represents the amount of soil to be stored after the location of x changes. Assume the joint distribution γ(x,y) represents the amount of soil required to be transported from x to y, and its marginal distribution is P. r and P θ If the work required to transport each unit of soil from location x to location y is d(x,y), then the bulldozer distance can be defined as:

[0137]

[0138] The bulldozer distance is transformed into a minimum optimization problem. The marginal distribution of γ(x,y) and γ(x,y)≥0 (buldozing amount greater than or equal to 0) together constitute the constraints of this optimization problem. Expanding Equation 10 yields:

[0139]

[0140] Rewrite Γ and D in matrix form:

[0141]

[0142]

[0143] Let the marginal distribution of the joint distribution γ(x,y) be P r and P θ Expand to summation form:

[0144]

[0145]

[0146] Combined with formulas 14 and 15, P r (x) and P θ (y) concatenates to obtain the long vector b, then the constraint condition can be simplified to AΓ = b, where:

[0147] b = [P] r (x1),P r (x2),…,P r (x n)|P θ (y1),P θ (y2),…,P θ (y n )] T (16)

[0148]

[0149] In summary, the bulldozer distance problem can be abstracted as an optimization problem of finding the minimum value under linear constraints, and expressed as:

[0150]

[0151] After constructing a hyperbola on the single-class camera model data in the feature space, a coarse classification of known-source and unknown-source camera models can be obtained. Then, the clustering results using the aforementioned multiple distance metrics are integrated and discriminated to construct a strong-confidence class dataset and a weak-confidence dataset, with the weak-confidence dataset being the suspicious set. Specifically, as follows... Figure 7 As shown, taking Euclidean distance and bulldozer distance as examples, assuming the number of categories in the unknown category dataset is k=2, the Euclidean distance clustering result shows higher intra-cluster purity for purple triangles, which are labeled "Unknown 1", and the remaining samples are labeled "Unknown 2". The bulldozer distance clustering result also labels "Unknown 1" and "Unknown 2", and in this case, the intra-cluster purity of the red square category is higher. The labels of the two clustering results are combined, and the combined results are sorted by quantity. Since k=2, the first two datasets are taken as the high-confidence set, labeled "Unknown 1" and "Unknown 2", and the remaining samples are labeled as the "suspicious set". At this point, the clustering result changes from the original k categories to k+1 categories, where "1" is the suspicious set.

[0152] After obtaining preliminary set distribution results by constructing a suspicious set, the next step is to construct a self-trained SVM classifier based on the idea of ​​transductive learning, and complete the classification of camera models from unknown sources through multiple iterations. For example... Figure 8 The diagram illustrates a self-trained SVM classifier. A new training set, consisting of the dataset with "known" labels and the dataset with "unknown" labels obtained from the initial coarse classification, is fed into the SVM classifier for self-training. Specifically, the following operations are performed in the s-th iteration:

[0153] Suppose the dataset labeled "known" is K, and the dataset labeled "unknown" is U, where U = U1∪U2…U k Let k be the number of unknown sample clusters calculated using the elbow method. Let the dataset labeled as the unknown camera model in the coarse classification results be denoted as [missing information]. Then the suspicious set

[0154] Let T = U∪K be the union of U and K, and use it as a new training set to train an SVM classifier C. s ;

[0155] Using C s Classify S, and denot the sum of the images of samples labeled as "known" and samples of the "unknown" class with an assigned label "unknown t" as ΔU;

[0156] Update T and S. T = U∪K∪ΔU, S = S - ΔU.

[0157] Among them, the iterative allocation rules involved in training a self-trained SVM classifier are as follows: Figure 9 As shown in the diagram. Color represents the label assigned to a sample by the classifier, while shape represents the sample's true label. Assuming the sample situation in the suspicious set is as shown in the top left box, yellow represents suspicious samples obtained through the construction of the suspicious set. The specific iterative allocation criteria are as follows:

[0158] In the first iteration, the set of suspicious samples is put into... Figure 9 In the first row of the self-trained SVM classifier, this classifier is trained using known samples obtained by coarse classification and k unknown sample datasets.

[0159] Sort the data and labels obtained from the SVM classifier output according to the quantity. Assuming the prediction result is as shown in the right box of the first row in the figure, add the category with the most identical sample labels to the corresponding category set in the training set of the self-trained SVM classifier (as shown by the dashed boundary triangle in the middle box, which is the result of the first round of iteration).

[0160] If, after sorting the prediction results, it is found that the number of labels of the category ranked first is ≥1, then in this round, the samples with the same number of labels among all classification results will be assigned and added to the corresponding training set category of the self-trained SVM classifier for training, thus obtaining a new classifier.

[0161] The process involves judging the suspicious set after sample allocation. If the suspicious set is empty, it means that all suspicious samples have been allocated, and the iteration stops. If the suspicious set is not empty, the remaining samples are assigned suspicious labels to form a new suspicious set, which is then fed into the new round of self-trained SVM classifier obtained in step 3 for classification.

[0162] Repeat steps 2 through 4 until the iteration stops, and output the data and labels from the self-trained SVM classifier as the final result.

[0163] This invention relates to digital image forensics within the field of machine learning. It primarily addresses the issue of insufficient detailed classification of camera models from unknown sources in open worlds, proposing a method for digital image source identification based on a self-trained SVM constructed from a suspicious set. This method performs SVDD in the image's CFA feature space to achieve coarse classification of samples from known and unknown sources. Subsequently, it uses Euclidean distance, standard Euclidean distance, and bulldozer distance as clustering metrics combined with ensemble discriminant analysis to construct a suspicious set. The SVM classifier is then self-trained through semi-supervised learning, and finally, the final classification result is obtained based on an iterative allocation criterion. The resulting algorithm can further learn and classify camera models from unknown sources, effectively addressing the problem of ineffective utilization of unknown source camera model samples in open environments.

[0164] This algorithm is used for image source forensics in open-world environments. In open-world scenarios, with the rapid development of technology, the number of image capture device models is exponentially increasing. It is impossible for any single dataset to be updated with data on all models and categories at all times. In such cases, the algorithm itself needs to possess good adaptability and scalability to utilize and learn from updated samples of unknown categories. Therefore, this method introduces the ideas of ensemble learning and transductive learning. After optimizing the hypercursive breadth in the feature space, a coarse classification is achieved. The suspicious set constructed by clustering and fusing multiple distance metrics is used for subsequent iterative updates and training of the self-trained SVM, ultimately realizing the utilization and learning from samples of unknown categories.

[0165] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. An open-world image source forensics method based on a suspicious set to construct a self-trained SVM, characterized in that, The steps are as follows: S1. Divide the images into training and testing sets; S2. Extract CFA features from the training and test sets shown respectively; S3. Input the CFA features of the above two into the SVDD classifier for coarse classification, classifying them into known classes and unknown classes; S4. Divide the unknown class into a suspicious set and several unknown classes; S5. Perform a self-trained SVM on the known classes, the suspicious set, and several unknown classes, and output the results. In step S5, after obtaining the preliminary set distribution results by constructing a suspicious set, the next step is to construct a self-trained SVM classifier based on the idea of ​​push learning, and complete the division of the camera model from unknown sources through multiple rounds of iteration; the data set with "known" labels and the data set with "unknown" labels obtained from the coarse classification are combined into a new training set and corresponding labels and fed into the SVM classifier for self-training. In the The following operations are performed in the next iteration: (1) Assume the dataset labeled "Known" is Data sets labeled "unknown" are set as follows: ,in , Let be the number of unknown sample clusters calculated using the elbow method; let be the set of camera model data marked as unknown in the coarse classification results. Then the suspicious set ; (2) Record and The union of is And used as a new training set to train an SVM classifier. ; (3) Use right The classification is performed, and the sum of the images labeled as "known" and the "unknown" class samples with an assigned label "unknown t" is denoted as . ; (4) and Update; , .

2. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 1, characterized in that, In step S2, CFA interpolation weight coefficients are estimated based on the covariance matrix to extract image sample features. An interpolation point in the color channel Select this point The pixel value of a point can be calculated by interpolating its neighborhood across the three channels: (1) Assuming it is necessary to Interpolating N pixels in the channel yields: Equation 2 can be represented as an eigenvector: To estimate the weights of the interpolation coefficients, a covariance matrix is ​​introduced into Equation 3: (4) This allows us to obtain the interpolation coefficient vector. for: For the interpolation coefficients of the other two color channels and Solve using the same method as described above.

3. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 2, characterized in that, choose The calculated neighborhood size of the interpolation point is Selecting neighborhoods based on Bayer CFA structure , sampling points Channel interpolation coefficients and two sampling points , The interpolation coefficients are estimated to obtain the total interpolation coefficients. One; take multiple non-overlapping values ​​from the image. A block-shaped region of a certain size is used to estimate the interpolation coefficients, and their variance and mean are calculated, thus yielding a total... The features of the dimension are used for training and testing.

4. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 1, characterized in that, In step S3, the following is adopted: -The mean clustering algorithm clusters the original category camera models into... Subclasses; using the elbow method model to provide - The means algorithm determines the optimal number of clusters; the core metric of the elbow method is the sum of squared errors, which is expressed as: (6) and This forms a relationship diagram similar to that of an elbow, with the elbow joint corresponding to... The value represents the actual number of clusters.

5. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 1, characterized in that, In step S4, after obtaining the set of unknown classes, a suspicious set is constructed in the set of unknown classes obtained in the coarse classification by using several distances as clustering conditions and combining the idea of ​​ensemble decision, and the initial clustering of camera sample data from unknown sources is obtained.

6. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 5, characterized in that, The distances mentioned include Euclidean distances, used to measure... The absolute distance between two points in Euclidean space; in two-dimensional space, Euclidean distance is used to represent the distance between two points. With point The straight-line distance between them: Two 480-dimensional vectors in the CFA feature space and The Euclidean distance between them can be expressed as: By standardizing the coordinates within the intervals where the variance and mean are equal through weighted calculation, the changes in different coordinate components correspond to their weights, thus obtaining the standardized Euclidean distance in the CFA feature space. 。 7. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 5, characterized in that, The distance includes the bulldozer distance, assuming there are two discrete piles of earth with the same mass but different distribution patterns. and If the mound of earth Convert to Work needs to be done on it, and the work done is equal to the mass of a unit of soil. To minimize the workload in transforming the soil mound, the total transportation cost needs to be optimized. and They are mounds of earth and The positional independent variable is set for each bar. and , express The amount of soil in the location, It means The amount of soil to be stored after the location change; assuming a joint distribution. Indicates from Transport to The required amount of soil to be transported is distributed along its edges. and From position Transporting each unit of soil to its destination (gv) The work required is Therefore, the bulldozer distance can be defined as: The bulldozer distance is transformed into a minimum optimization problem; Edge distribution and Together, they constitute the constraints of this optimization problem; at this point, expanding equation 10 yields: rewrite and Written in matrix form: Joint distribution The edge distribution is and Expand to summation form: Combined formulas 14 and 15, will and The concatenated long vector Then the constraints can be simplified to ,in: The bulldozer distance can be abstracted as an optimization problem of finding the minimum value under linear constraints, and expressed as: 。 8. The open-world image source forensics method based on a self-trained SVM constructed from a suspicious set as described in claim 7, characterized in that, The training of the self-trained SVM classifier uses an iterative assignment method: where color represents the label assigned to the sample by the classifier, and shape represents the true label of the sample; the specific iterative assignment criteria are as follows: (1) In the first iteration, the set of suspicious samples is fed into a self-trained SVM classifier, which is a set of known samples obtained through coarse classification. The dataset was obtained by training on samples from an unknown sample class. (2) Sort the data and labels obtained by the SVM classifier output according to the number of samples; add the category with the most identical sample labels to the corresponding category set in the training set of the self-trained SVM classifier; (3) If, after sorting the prediction results, the number of category labels ranked first is found to be...

1. In this round, the sample with the same number of labels among all classification results is assigned and added to the corresponding training set category of the self-trained SVM classifier for training, thus obtaining a new classifier. (4) Judge the suspicious set after the sample allocation. If the suspicious set is empty, it means that all suspicious samples have been allocated and the iteration stops. If the suspicious set is not empty, the remaining samples are all set as suspicious labels to form a new suspicious set and put into the new round of self-trained SVM classifier obtained in step (3) for classification. (5) Repeat steps (2) to (4) until the iteration stops, and output the data and labels in the self-trained SVM classifier as the final result.

Images