A video source recognition method combining PRNU matching and inter-frame geometric transformation estimation

Abstract

The present invention discloses a video source identification method combining PRNU matching and inter-frame geometric transformation estimation, which relates to the technical field of video source identification and includes the following steps: S1: extraction of photosensitive response inhomogeneity; S2: three-dimensional geometric transformation; S3: video source identification combining photosensitive response inhomogeneity matching and inter-frame geometric transformation estimation; S4: geometric transformation parameter fitting; S5: shooting behavior trajectory modeling and visualization. The technical problem to be solved by the present invention is to provide a video source identification method combining PRNU matching and inter-frame geometric transformation estimation, which uses a multi-scale transformation method to perform inter-frame alignment of a reference fingerprint and a test fingerprint, uses deep learning to construct a PRNU matching model to calculate the correlation score of the test video frame and the reference video frame, and uses an inter-frame geometric transformation algorithm to fit the optimal geometric parameters, restore the shooting behavior, and improve the matching accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of video source recognition, and in particular to a video source recognition method combining PRNU matching and inter-frame geometric transformation estimation. Background Art

[0002] With the widespread adoption of video recording devices and online platforms, digital video has become a crucial medium for information dissemination and evidence preservation. However, video data is susceptible to duplication, tampering, and dissemination, presenting significant challenges in verifying its authenticity and traceability in judicial practice and cybersecurity governance. Effectively identifying the source of video recording equipment and reconstructing the recording process has become a research hotspot in the field of digital forensics.

[0003] PRNU (Photosensitive Response Nonuniformity), a unique noise characteristic of imaging sensors, has excellent device fingerprint properties and has been widely used in video and image attribution tasks. However, in real-world video scenarios, traditional static image-level PRNU matching methods suffer from low accuracy and robustness in video source identification due to inter-frame motion, compression distortion, and occlusion. Therefore, some research has attempted to combine deep learning with PRNU technology to improve attribution efficiency and accuracy. Furthermore, if digital forensics technology can exploit the "data-behavior" correlation, it can help investigators reconstruct the cybercrime process and establish a more coherent chain of evidence encompassing the "video-capturing device-capturing behavior" link.

[0004] Deficiencies of existing technology:

[0005] 1. Limitations of Video Source Identification Methods: Compared to image-based forensics, research in video forensics lags behind, and video source identification still primarily relies on traditional methods to calculate PCE. In real-world applications, video data accounts for a significant portion of network traffic and is often used for illegal recording and privacy violations, necessitating efficient and reliable methods for tracing the source. However, existing image-based PRNU research results are difficult to directly apply to video scenarios due to the unique characteristics of video, such as inter-frame motion and compression distortion. Therefore, video-based forensics technology is particularly important.

[0006] 2. Insufficient evidence collection capabilities for filming behavior: While existing technology can identify the device that captured a video to a certain extent, the ability to reconstruct and analyze the specific actions taken during the video capture process remains relatively limited. Most current evidence collection methods struggle to provide detailed information such as the location, time, angle, or operator behavior of the footage, factors that are often crucial to case determination and evidence integrity. Without behavioral support, single device fingerprint information often fails to meet the high legal requirements for a closed-loop chain of evidence and provides investigators with limited leads for further investigation. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a video source identification method that combines PRNU matching and inter-frame geometric transformation estimation. The method first performs preprocessing, collects homologous videos corresponding to multiple camera sources to construct a sample set, and uses a PRNU extraction network to extract the PRNU camera fingerprint of the reference video frame as a camera fingerprint reference, and extracts the camera fingerprint of the test video frame (the video to be traced). Since the anti-shake function causes misalignment between video frames when shooting videos, we use a multi-scale transformation method to align the reference fingerprint and the test fingerprint between frames, and use deep learning to build a PRNU matching model to calculate the correlation score of the test video frame and the reference video frame, thereby improving the matching accuracy and efficiency. In this process, we can achieve two results: one is to realize video source identification and determine whether the video is shot by the same device, and the other is to find the initial transformation parameters of the frame geometric transformation. Finally, the RAM method is used to achieve the final inter-frame geometric transformation estimation, and the optimal parameters of the video frame are found, providing a basis for restoring the shooting behavior.

[0008] The present invention adopts the following technical solutions to achieve the invention objectives:

[0009] A video source recognition method combining PRNU matching and inter-frame geometric transformation estimation is characterized by comprising the following steps:

[0010] S1: Extraction of photosensory response heterogeneity;

[0011] S2: 3D geometric transformation;

[0012] S3: Video source recognition combining photosensitive response inhomogeneity matching and inter-frame geometric transformation estimation;

[0013] S4: geometric transformation parameter fitting;

[0014] S5: Shooting behavior trajectory modeling and visualization.

[0015] As a further limitation of this technical solution, the step S1 is:

[0016] S11: Construct a training sample set by collecting multiple videos shot by the same device;

[0017] S12: Automatically extract photosensitivity response heterogeneity using convolutional neural networks and save it as a reference fingerprint.

[0018] S13: Extract multiple frames from the video to be identified, input the same feature extraction model, and obtain the corresponding test fingerprint. The loss function of the extraction network is:

[0019] (1);

[0020] Where: X is the set of camera fingerprints of the reference video frame; Y is the set of estimated fingerprints of the test video frame; It is reference video frames; It is Test video frames; Represents the correlation coefficient between set X and set Y; 、 are the means of sets X and Y respectively; represents element-wise multiplication; represents the Euclidean norm;

[0021] (2);

[0022] in: To balance the losses;

[0023] (3);

[0024] in: is the mean square error loss; N is the number of video frames;

[0025] (4);

[0026] Where: L represents the final loss function; Represents the weight parameter.

[0027] As a further limitation of this technical solution, the specific steps of S2 are:

[0028] Quaternions are used to represent three-dimensional rotation operations. Usually, a quaternion consists of a real part and three imaginary parts. The four parameters in the quaternion correspond to the rotation information in three-dimensional space. Different parameter combinations can represent rotations around different rotation axes. Its composition is expressed as:

[0029] (5);

[0030] in: , is the real part, is the imaginary coefficient;

[0031] In geometric motion, It can be understood as the rotation components corresponding to the three coordinate axis directions; using unit quaternion To represent the rotation around the unit vector Rotation Angle, space point The rotation result is:

[0032] (6);

[0033] After expansion:

[0034] (7);

[0035] At this point, the entire quaternion represents any rotational motion in three-dimensional space.

[0036] As a further limitation of this technical solution, the specific steps of S3 are:

[0037] S31: First, the matching model for video source recognition is quantized into a binary classification problem. Feature extraction and training are performed by inputting homologous video frames from multiple devices. Finally, the 3D geometric transformation of the video frames is integrated into the model, enabling it to accurately calculate similarity scores under multi-scale transformation conditions.

[0038] S32: Divide the test photosensitive response non-uniformity noise and the reference photosensitive response non-uniformity noise into non-overlapping blocks and perform geometric transformation model matching. In the matching result, if the number of block matches exceeds a set threshold, it indicates that the video was shot by the device, thereby achieving video source identification. The inter-frame geometric transformation estimation algorithm is further used to update the optimization transformation matrix to achieve the goal of finding the optimal parameters.

[0039] S33: inter-frame geometric transformation estimation algorithm;

[0040] S331: Smoothing the transformed video frame to reduce noise. Then, the reference video frame and the transformed query video frame are divided into multiple overlapping small blocks. Matching scores are performed on each pair of small blocks using a geometric transformation model to select matching blocks with high confidence.

[0041] S332: For these small blocks with high matching scores, the center point of each block is used as a geometric matching point. If the matching score is high, it is considered that the center point is an aligned point pair in the reference video frame and the query video frame;

[0042] By fine-tuning the position in the local area, the matching accuracy is further improved, and then the spatial position difference of each pair of matching blocks after pixel-level fine-tuning is calculated to calculate the matching error. :

[0043] (8);

[0044] in: It is Matching blocks, is the similarity score of the match;

[0045] S333: Calculate matching error Then, combined with the probability model, using the Bayesian formula, calculate the The posterior probability that a matching block belongs to the correct match:

[0046] (9);

[0047] (10);

[0048] in: is the standard deviation; is the prior probability of the match being correct; is the probability model constant of the noise; Represents the initial transformation matrix;

[0049] S334: Estimate the credibility weight of each pair of matching blocks through posterior probability and construct a diagonal matrix :

[0050] (11);

[0051] Among them: diag represents the operation of constructing a diagonal matrix;

[0052] Using these weights as weighting factors, the updated affine transformation parameters are fitted by the weighted least squares method to obtain a more accurate transformation matrix;

[0053] (12);

[0054] (13);

[0055] in: Indicates the parameter value when the matrix H is minimized; It is used to measure the error or difference of the matrix elements as a whole; the matrix A is the set of matching point coordinates in the reference video frame. ; Matrix B is the set of matching point coordinates in the test video frame ; The diagonal elements of the weight matrix W are the posterior probability weights of each matching point;

[0056] S335: The process of fitting the updated affine transformation parameters will continue to iterate until the parameter change of the transformation matrix and the matching error meet the preset convergence conditions;

[0057] (14);

[0058] in: is the current affine transformation matrix, is the last affine transformation matrix; Represents the change in transformation matrix;

[0059] If the matching error meets the preset threshold, the iteration stops, otherwise it continues to update;

[0060] S336: Finally, when the conditions are met for convergence, the optimized affine matrix is ​​output to achieve high-precision registration.

[0061] As a further limitation of this technical solution, the step S4 is:

[0062] The inter-frame transformation is represented as the product of three spatial rotation matrices, which are combined with scaling and translation operations to form a complete geometric transformation matrix. The three rotation angles represent rotations around different axes, and their corresponding rotation matrices are:

[0063] Rotation around the X axis: ;

[0064] Rotation around the Y axis: ;

[0065] Rotation around the Z axis: ;

[0066] in: is the rotation angle around the X axis; is the rotation angle around the Y axis; is the rotation angle around the Z axis;

[0067] Therefore, the corresponding geometric transformation matrix is:

[0068] (15);

[0069] in: Represents scaling and translation transformation matrices:

[0070] (16);

[0071] in: represents a uniform scaling factor; , Indicates the translation along the x-axis and y-axis.

[0072] As a further limitation of this technical solution, the step S5 is:

[0073] S51: Geometric transformation analysis of different videos on the same device;

[0074] To further verify the effectiveness of inter-frame geometric transformation parameters in modeling shooting behavior, we selected multiple videos shot with the same camera and extracted the transformation parameters of their second frames for analysis. The distribution of Euler angles and the corresponding scaling factor changes in the second frames of these videos were displayed.

[0075] S52: Geometric transformation analysis of the same video on the same device.

[0076] As a further limitation of this technical solution, the geometric transformation model mainly includes:

[0077] Feature Extractor: Utilizes convolutional neural networks to extract uniform features from the noise of photosensitive response inhomogeneity, facilitating the identification of matching points and providing a basis for estimating geometric relationships.

[0078] Mask predictor: Generates a mask to exclude areas where feature pairs cannot be formed, improving matching accuracy;

[0079] Transformation Matrix Estimator: It predicts the spatial transformation relationship between the non-uniformity of the two photosensitive responses based on the effective feature pairs and outputs the transformation matrix for alignment.

[0080] Compared with related technologies, the video source recognition method provided by the present invention, which combines PRNU matching with inter-frame geometric transformation estimation, has the following beneficial effects:

[0081] Improving the efficiency and accuracy of video source identification: Existing methods typically rely on traditional methods to calculate the PCE value of the PRNU for video source identification. This involves a one-by-one comparison within a large database, resulting in high computational complexity and low recognition efficiency. To address this, the present invention introduces a deep learning model that uses end-to-end training to learn the matching relationship between the PRNU features of reference and query video frames. This enables efficient feature extraction and rapid similarity determination for video source identification, significantly improving recognition accuracy and processing efficiency, making it more suitable for large-scale video forensics scenarios.

[0082] Enhance the ability to analyze shooting behaviors and build chains of evidence: Traditional technologies have difficulty restoring specific behavioral characteristics during video shooting, resulting in a lack of evidence chains and difficulty meeting judicial needs. On the basis of completing device tracing, the present invention further introduces an inter-frame geometric transformation estimation algorithm - the RAM method, to model the motion relationship between video frames, and infer possible camera displacement, rotation and other behavioral parameters during the shooting process, thereby providing a quantitative basis for restoring shooting behaviors. This process not only enriches the correlation between video content and equipment, but also provides important technical support for behavioral-level evidence collection, and helps to build a more complete and verifiable electronic chain of evidence. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 This is the overall architecture diagram of the present invention.

[0084] Figure 2 Schematic diagram of the depth matching model architecture integrating geometric transformation of the present invention.

[0085] Figure 3This is a specific flow chart of the RAM method of the present invention.

[0086] Figure 4 is the scaling factor of the second frame in different videos for the same device of the present invention.

[0087] Figure 5 is the three-dimensional rotation angle of the second frame of each video of the present invention.

[0088] Figure 6 is the scaling factor of the same video on the same device in the present invention.

[0089] Figure 7 It is the rotation angle of the same video under the same device of the present invention. DETAILED DESCRIPTION

[0090] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0091] A video source recognition method combining PRNU matching and inter-frame geometric transformation estimation includes the following steps:

[0092] S1: Photo Response Non-Uniformity (PRNU) extraction;

[0093] In order to realize the full-process end-to-end video source recognition method, the traditional PRNU extraction and matching operations are broken. The first step is the PRNU extraction operation. The steps of S1 are:

[0094] S11: Construct a training sample set by collecting multiple videos shot by the same device;

[0095] S12: Automatically extract photosensitivity response heterogeneity using convolutional neural networks and save it as a reference fingerprint.

[0096] S13: Extract multiple frames from the video to be identified, input the same feature extraction model, and obtain the corresponding test fingerprint. The loss function of the extraction network is:

[0097] (1);

[0098] Where: X is the set of camera fingerprints of the reference video frame; Y is the set of estimated fingerprints of the test video frame; It is reference video frames; It is Test video frames; Represents the correlation coefficient between set X and set Y, which measures the degree of linear correlation between two sets of data. The closer the value is to 1, the stronger the linear correlation is, and the closer it is to 0, the weaker the linear correlation is. It is used to quantify the matching degree between fingerprint sets; 、 are the means of sets X and Y respectively; represents element-wise multiplication; represents the Euclidean norm;

[0099] (2);

[0100] in: To balance the losses;

[0101] (3);

[0102] in: is the mean square error loss; N is the number of video frames;

[0103] (4);

[0104] Where: L represents the final loss function; Represents the weight parameter.

[0105] S2: Three-dimensional geometric transformation.

[0106] The specific steps of S2 are:

[0107] Due to the upgrade of modern mobile devices, more and more mobile phones have anti-shake functions, which enables the captured video to self-correct the geometric dislocation between frames. Therefore, after extracting the reference PRNU noise and the test PRNU noise, the inter-frame alignment operation is performed through the multi-scale transformation method. Video shooting actually belongs to the three-dimensional space. Because three-dimensional geometric transformation can be introduced, in order to reduce the complexity of the algorithm, quaternions are used to represent three-dimensional rotation operations. Usually, a quaternion consists of a real part and three imaginary parts. The four parameters in the quaternion correspond to the rotation information in the three-dimensional space. Different parameter combinations can represent rotations around different rotation axes. Its composition can be expressed as:

[0108] (5);

[0109] in: , is the real part, is the imaginary coefficient;

[0110] In geometric motion, It can be understood as the rotation components corresponding to the three coordinate axis directions; if quaternions are used to represent three-dimensional rotation, unit quaternions can be used ( Represents the half-angle parameter of the rotation operation in three-dimensional space) to represent the rotation around the unit vector Rotation Angle, space point (treated as pure quaternion ) is:

[0111] (6);

[0112] After expansion:

[0113] (7);

[0114] At this point, the entire quaternion represents any rotational motion in three-dimensional space.

[0115] S3: Video source recognition combining photosensitive response inhomogeneity matching and inter-frame geometric transformation estimation.

[0116] The traditional algorithm uses normalized cross-correlation to calculate the PCE value to evaluate the degree of match between the test video and the reference device, thereby determining the source of the video. However, there may be limitations in the final evaluation of the degree of match. The model based on deep learning has a powerful feature learning ability and can learn more complex patterns and features. Therefore, the present invention proposes a video source recognition method that combines PRNU matching with inter-frame geometric transformation estimation. Since the matching model of the video frame is not as accurate as the matching model of the image, the accuracy of video frame source recognition is improved by a block algorithm, and the complexity of a large number of algorithms generated by the introduction of three-dimensional geometric transformation is further reduced. The extraction of reference PRNU and test PRNU noise has been introduced above, so video source recognition and inter-frame geometric transformation sequence estimation are performed on this basis. The specific steps of S3 are:

[0117] S31: First, the matching model for video source recognition is quantized into a binary classification problem. Feature extraction and training are performed by inputting homologous video frames from multiple devices. Finally, the 3D geometric transformation of the video frames is integrated into the model, enabling it to accurately calculate similarity scores under multi-scale transformation conditions. Compared with traditional methods, the geometric transformation model improves the accuracy of video source recognition.

[0118] S32: Divide the test photosensitive response non-uniformity noise and the reference photosensitive response non-uniformity noise into non-overlapping blocks and perform geometric transformation model matching. In the matching result, if the number of block matches exceeds a set threshold, it indicates that the video was shot by the device, thereby achieving video source identification. The inter-frame geometric transformation estimation algorithm is further used to update the optimization transformation matrix to achieve the goal of finding the optimal parameters.

[0119] S33: Inter-frame geometric transformation estimation (Update Affine Matrix, RAM) algorithm;

[0120] The transformed test video frame is Gaussian filtered to suppress high-frequency noise. Subsequently, the reference video frame and the transformed test video frame are divided into multiple overlapping blocks using a sliding window method. The pre-trained matching model is used to perform matching scores on each pair of overlapping blocks to screen out block pairs with high matching scores. For these candidate matching blocks, the matching accuracy is further improved by fine-tuning the block position at the pixel level in the local area, and a weighted block center coordinate pair is constructed based on the score. Afterwards, the posterior probability of each pair of matching blocks is calculated using the Bayesian estimation method based on the inter-block matching error, and is used as the weight of the weighted least squares method to fit the updated affine transformation matrix. This process is iterated until the change in transformation parameters and the matching error meet the convergence conditions, thereby obtaining a more accurate registration result. The specific operations are as follows:

[0121] Different from the existing block matching affine estimation, this method introduces a score-driven probability modeling mechanism to improve robustness and accuracy.

[0122] S331: Smoothing the transformed video frame to reduce noise. Then, the reference video frame and the transformed query video frame are divided into multiple overlapping small blocks. Matching scores are performed on each pair of small blocks using a geometric transformation model to select matching blocks with high confidence.

[0123] S332: For these small blocks with high matching scores, the center point of each block is used as a geometric matching point. If the matching score is high, the center point is considered to be an aligned point pair in the reference video frame and the query video frame. In other words, a weighted block center coordinate pair is constructed to estimate the affine or projective transformation matrix.

[0124] By fine-tuning the position in the local area, the matching accuracy is further improved, and then the spatial position difference of each pair of matching blocks after pixel-level fine-tuning is calculated to calculate the matching error. :

[0125] (8);

[0126] in: It is Matching blocks, is the similarity score of the match, which is used here as the matching probability;

[0127] S333: Calculate matching error Then, combined with the probability model, using the Bayesian formula, calculate the The posterior probability that a matching block belongs to the correct match:

[0128] (9);

[0129] (10);

[0130] in: is the standard deviation; is the prior probability of the match being correct; is the probability model constant of the noise; Represents the initial transformation matrix;

[0131] From many experimental data , ;

[0132] S334: Estimate the credibility weight of each pair of matching blocks through posterior probability and construct a diagonal matrix :

[0133] (11);

[0134] Among them: diag represents the operation of constructing a diagonal matrix;

[0135] Using these weights as weighting factors, the updated affine transformation parameters are fitted by the weighted least squares method to obtain a more accurate transformation matrix;

[0136] (12);

[0137] (13);

[0138] in: Indicates the parameter value when the matrix H is minimized; It is used to measure the error or difference of the matrix elements as a whole. It is calculated by summing the squares of all elements of the matrix. Matrix A is the coordinate set of the matching points (source points) in the reference video frame. ; Matrix B is the coordinate set of matching points (source points) in the test video frame ; The diagonal elements of the weight matrix W are the posterior probability weights of each matching point;

[0139] S335: The process of fitting the updated affine transformation parameters will continue to iterate until the parameter change of the transformation matrix and the matching error meet the preset convergence conditions;

[0140] (14);

[0141] in: is the current affine transformation matrix, is the last affine transformation matrix; Represents the change in transformation matrix;

[0142] If the matching error meets the preset threshold, the iteration stops, otherwise it continues to update;

[0143] S336: Finally, when the conditions are met for convergence, the optimized affine matrix is ​​output to achieve high-precision registration.

[0144] S4: Geometric transformation parameter fitting.

[0145] The steps of S4 are:

[0146] To more accurately model the geometric relationship between video frames, this paper decomposes the inter-frame transformation matrix into multiple components with physical meaning. Specifically, the inter-frame transformation is represented as the product of three spatial rotation matrices (pitch, yaw, roll), combined with scaling and translation operations to form a complete geometric transformation matrix. The three rotation angles represent rotations around different axes, and their corresponding rotation matrices are:

[0147] Rotation around the X axis: ;

[0148] Rotation around the Y axis: ;

[0149] Rotation around the Z axis: ;

[0150] in: is the rotation angle around the X axis; is the rotation angle around the Y axis; is the rotation angle around the Z axis;

[0151] Therefore, the corresponding geometric transformation matrix is:

[0152] (15);

[0153] in: Represents scaling and translation transformation matrices:

[0154] (16);

[0155] in: represents a uniform scaling factor; , Indicates the translation along the x-axis and y-axis.

[0156] S5: Shooting behavior trajectory modeling and visualization.

[0157] The steps of S5 are:

[0158] S51: Geometric transformation analysis of different videos on the same device;

[0159] To further verify the effectiveness of inter-frame geometric transformation parameters in modeling shooting behavior, we selected multiple videos shot with the same camera (SP0016) and extracted the transformation parameters of their second frames for analysis. The distribution of Euler angles (Roll, Pitch, Yaw) and the corresponding scaling factor changes in the second frames of these videos were displayed.

[0160] like Figure 4 and Figure 5 As shown in Figure 3, there are significant differences in the geometric transformation parameters (such as pitch and scaling factor) of the same device in different videos, reflecting the changes in shooting behavior, indicating that these parameters can be used to quantify the shooting characteristics of the video.

[0161] S52: Geometric transformation analysis of the same video on the same device;

[0162] like Figure 6 and Figure 7 As shown in the figure, the changing trends of the scale factor and Euler angles indicate that in the first 10 frames of the video, the device undergoes continuous pitch adjustments (reflected by changes in pitch), slight tilt (indicated by fluctuations in roll), and periodic forward and backward displacement or zoom (indicated by fluctuations in the scale factor). The overall motion is smooth, with minimal changes in yaw, indicating no drastic device rotation, which is more consistent with the behavior of slow-motion indoor filming. Therefore, the significant differences in geometric transformation parameters (such as pitch and scale factor) across ten consecutive frames of the same device in the same video reflect the dynamic changes during the filming process.

[0163] The geometric transformation model improves the accuracy of video source recognition. The geometric transformation model mainly includes:

[0164] Feature Extractor: Uses a Convolutional Neural Network (CNN) to extract unified features from PRNU noise, facilitating the identification of matching points and providing a basis for estimating geometric relationships.

[0165] Mask predictor: Generates a mask to exclude areas where feature pairs cannot be formed, improving matching accuracy;

[0166] Transformation Matrix Estimator: Predicts the spatial transformation relationship between two PRNUs based on valid feature pairs and outputs the transformation matrix for alignment.

[0167] In summary, this paper proposes a video source identification method that combines PRNU matching with inter-frame geometric transformation estimation, effectively overcoming the limitations of traditional video forensics techniques in terms of device identification accuracy and recording behavior reconstruction. By introducing a deep learning network for end-to-end extraction and matching of device fingerprint information (PRNU) within video frames, the accuracy and processing efficiency of video source identification are significantly improved. Furthermore, multi-scale geometric transformations are used to model inter-frame motion relationships. Quaternion rotation and affine transformation parameter estimation techniques are employed to quantitatively restore behavioral characteristics such as displacement and rotation during video recording. In particular, the introduction of the RAM method and Bayesian weighted fitting mechanism enables stable high-precision registration and 3D motion inference even in complex scenes. This method not only improves the robustness of video forensics against inter-frame misalignment and compression distortion, but also, by correlating these transformation parameters with actual recording operations, provides technical support for establishing a video evidence chain encompassing the "device-content-behavior" triple link. This method has promising forensic application value and engineering application prospects.

[0168] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A video source recognition method combining PRNU matching and inter-frame geometric transformation estimation, characterized in that: The following steps are involved: S1: Construct a training sample set and perform photosensitive response inhomogeneity extraction on the training sample set; S2: Perform three-dimensional geometric transformation on the extracted training sample set; S3: Combining photosensitive response non-uniformity matching with inter-frame geometric transformation estimation to perform video source recognition on the training sample set; The specific steps of S3 are: S31: First, the matching model for video source recognition is quantized into a binary classification problem. Feature extraction and training are performed by inputting homologous video frames from multiple devices. Finally, the 3D geometric transformation of the video frames is integrated into the model, enabling it to accurately calculate similarity scores under multi-scale transformation conditions. S32: Divide the test photosensitive response non-uniformity noise and the reference photosensitive response non-uniformity noise into non-overlapping blocks and perform geometric transformation model matching. In the matching result, if the number of block matches exceeds a set threshold, it indicates that the video was shot by the device, thereby achieving video source identification. The inter-frame geometric transformation estimation algorithm is further used to update the optimization transformation matrix to achieve the goal of finding the optimal parameters. S33: inter-frame geometric transformation estimation algorithm; S331: Smoothing the transformed video frame to reduce noise. Then, the reference video frame and the transformed query video frame are divided into multiple overlapping small blocks. Matching scores are performed on each pair of small blocks using a geometric transformation model to select matching blocks with high confidence. S332: For these small blocks with high matching scores, the center point of each block is used as a geometric matching point. If the matching score is high, it is considered that the center point is an aligned point pair in the reference video frame and the query video frame; By fine-tuning the position in the local area, the matching accuracy is further improved, and then the spatial position difference of each pair of matching blocks after pixel-level fine-tuning is calculated to calculate the matching error. : （8）； in: It is Matching blocks, is the similarity score of the match; S333: Calculate matching error Then, combined with the probability model, using the Bayesian formula, calculate the The posterior probability that a matching block belongs to the correct match: （9）； （10）； in: is the standard deviation; is the prior probability of the match being correct; is the probability model constant of the noise; Represents the initial transformation matrix; S334: Estimate the credibility weight of each pair of matching blocks through the posterior probability and construct a diagonal matrix : （11）； Among them: diag represents the operation of constructing a diagonal matrix; Using these weights as weighting factors, the updated affine transformation parameters are fitted by the weighted least squares method to obtain a more accurate transformation matrix; （12）； （13）； in: Indicates the parameter value when the matrix H is minimized; It is used to measure the error or difference of the matrix elements as a whole; the matrix A is the set of matching point coordinates in the reference video frame. ; Matrix B is the set of matching point coordinates in the test video frame ; The diagonal elements of the weight matrix W are the posterior probability weights of each matching point; S335: The process of fitting the updated affine transformation parameters will continue to iterate until the parameter change of the transformation matrix and the matching error meet the preset convergence conditions; （14）； in: is the current affine transformation matrix, is the last affine transformation matrix; Represents the change in the transformation matrix; If the matching error meets the preset threshold, the iteration stops, otherwise it continues to update; S336: Finally, when the conditions are met for convergence, the optimized affine matrix is ​​output to achieve high-precision registration; S4: Geometric transformation parameter fitting of the training sample set for video source recognition; S5: Modeling and visualization of the shooting behavior trajectory of the training sample set.

2. The video source identification method combining PRNU matching and inter-frame geometric transformation estimation according to claim 1, characterized in that: The steps of S1 are: S11: Construct a training sample set by collecting multiple videos shot by the same device; S12: Automatically extract photosensitivity response heterogeneity using convolutional neural networks and save it as a reference fingerprint. S13: Extract multiple frames from the video to be identified, input the same feature extraction model, and obtain the corresponding test fingerprint. The loss function of the extraction network is: （1）； Where: X is the set of camera fingerprints of the reference video frame; Y is the set of estimated fingerprints of the test video frame; It is reference video frames; It is Test video frames; Represents the correlation coefficient between set X and set Y; 、 are the means of sets X and Y respectively; represents element-wise multiplication; represents the Euclidean norm; （2）； in: To balance the losses; （3）； in: is the mean square error loss; N is the number of video frames; （4）； Where: L represents the final loss function; Represents the weight parameter.

3. The video source identification method combining PRNU matching and inter-frame geometric transformation estimation according to claim 1, characterized in that: The specific steps of S2 are: Quaternions are used to represent three-dimensional rotation operations. Usually, a quaternion consists of a real part and three imaginary parts. The four parameters in the quaternion correspond to the rotation information in three-dimensional space. Different parameter combinations can represent rotations around different rotation axes. Its composition is expressed as: （5）； in: , is the real part, is the imaginary coefficient; In geometric motion, It can be understood as the rotation components corresponding to the three coordinate axis directions; using unit quaternion To represent the rotation around the unit vector Rotation Angle, space point The rotation result is: （6）； After expansion: （7）； At this point, the entire quaternion represents any rotational motion in three-dimensional space.

4. The video source identification method combining PRNU matching and inter-frame geometric transformation estimation according to claim 3, characterized in that: The steps of S4 are: The inter-frame transformation is represented as the product of three spatial rotation matrices, which are combined with scaling and translation operations to form a complete geometric transformation matrix. The three rotation angles represent rotations around different axes, and their corresponding rotation matrices are: Rotation around the X axis: ; Rotation around the Y axis: ; Rotation around the Z axis: ; in: is the rotation angle around the X axis; is the rotation angle around the Y axis; is the rotation angle around the Z axis; Therefore, the corresponding geometric transformation matrix is: （15）； in: Represents scaling and translation transformation matrices: （16）； in: represents a uniform scaling factor; , Indicates the translation along the x-axis and y-axis.

5. The video source identification method combining PRNU matching and inter-frame geometric transformation estimation according to claim 4, characterized in that: The steps of S5 are: S51: Geometric transformation analysis of different videos on the same device; To further verify the effectiveness of inter-frame geometric transformation parameters in modeling shooting behavior, we selected multiple videos shot with the same camera and extracted the transformation parameters of their second frames for analysis. The distribution of Euler angles and the corresponding scaling factor changes in the second frames of these videos were displayed. S52: Geometric transformation analysis of the same video on the same device.

6. The video source identification method combining PRNU matching and inter-frame geometric transformation estimation according to claim 4, characterized in that: The geometric transformation model includes: a feature extractor: using a convolutional neural network to extract unified features from the photosensitive response non-uniformity noise, so as to facilitate the identification of matching points and provide a basis for estimating geometric relationships; Mask predictor: Generates a mask to exclude areas where feature pairs cannot be formed, improving matching accuracy; Transformation Matrix Estimator: It predicts the spatial transformation relationship between the non-uniformity of the two photosensitive responses based on the effective feature pairs and outputs the transformation matrix for alignment.

Images