A multi-feature fusion content tamper detection method, system and device

Abstract

The application provides a content tamper detection method, system and device with multi-feature fusion. The scheme provided by the application obtains the structural features of each video group GOP in a video to be played, fuses the structural features to generate first verification information, embeds the first verification information representing the structural features of each GOP in the GOP, and sends the first verification information to an edge device, so that the edge device determines whether the video content is tampered based on the first verification information. Since the information representing the structural features of the GOP is recorded in the first verification information, any tampering of the GOP will affect the features, causing the verification information determined in the same way to change. Therefore, the edge device can determine whether the video content is tampered based on the first verification information, thereby improving the accuracy of the video content security detection.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method, system and apparatus for content anti-tampering detection using multi-feature fusion. Background Technology

[0002] During video transmission, to prevent tampering, a checksum is typically created on the video file in the cloud, inserted into the video, and sent to the edge device (the video playback end) for storage. Upon receiving the video from the cloud device, the edge device verifies the locally stored checksum against the one embedded in the video to determine if the content has been altered. However, if an attacker intercepts the video containing the checksum during transmission, they can replace the video content and send it to the edge device with the same checksum. In this case, the edge device cannot detect the tampering, posing a security risk to the video content. Summary of the Invention

[0003] In view of this, this application provides a multi-feature fusion content anti-tampering detection method, system and apparatus to improve the accuracy of video content security detection.

[0004] The technical solution provided in this application is as follows: According to an embodiment of the first aspect of this application, a multi-feature fusion content anti-tampering detection method is provided, which is applied to a cloud device, and the method includes: For each video group (GOP) in the video to be played, the structural features of the GOP are obtained. The structural features include global features and local features. The global features are used to characterize the overall structure of the video to be played, and the local features include the structural attributes and statistical features of the GOP. The structural features of the GOP are fused to obtain the feature fused watermark corresponding to the GOP. Based on the feature fusion watermark corresponding to the GOP, the first verification information corresponding to the GOP is generated, and the first verification information is embedded in the GOP; The watermarked video generated based on each GOP with embedded first verification information is sent to the edge device, so that the edge device can verify whether the content of the watermarked video has been tampered with based on the first verification information corresponding to each GOP in the watermarked video.

[0005] According to an embodiment of the second aspect of this application, a multi-feature fusion content anti-tampering detection method is provided, which is applied to an edge device, and the method includes: The system receives a watermarked video sent by a cloud device and extracts the first verification information embedded in each group of videos (GOPs) in the watermarked video. The first verification information is generated by the cloud device based on the feature fusion watermark corresponding to the GOP. The feature fusion watermark is obtained by the cloud device by fusing the structural features of the GOP. The structural features include global features and local features. The global features are used to characterize the overall structure of the watermarked video, and the local features include the structural attributes and statistical features of the GOP. Based on the structural characteristics of each GOP, the second verification information corresponding to each GOP is determined; The content of the watermarked video is verified to have been tampered with based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP.

[0006] According to an embodiment of a third aspect of this application, a multi-feature fusion content anti-tampering detection device is provided, which is applied to a cloud device, and the device includes: The obtaining unit is used to obtain the structural features of each video group GOP in the video to be played; the structural features include global features and local features, the global features are used to characterize the overall structure of the video to be played, and the local features include the structural attributes and statistical features of the GOP. The fusion unit is used to fuse the structural features of the GOP to obtain the feature fusion watermark corresponding to the GOP. The generation unit is used to generate first verification information corresponding to the GOP based on the feature fusion watermark corresponding to the GOP, and embed the first verification information into the GOP; The sending unit is used to send the watermarked video generated based on each GOP with embedded first verification information to the edge device, so that the edge device can verify whether the content of the watermarked video has been tampered with based on the first verification information corresponding to each GOP in the watermarked video.

[0007] According to an embodiment of the fourth aspect of this application, a multi-feature fusion content anti-tampering detection device is provided. This device is applied to an edge device and includes: A receiving unit is configured to receive a watermarked video sent by a cloud device and extract first verification information embedded in each group of videos (GOPs) in the watermarked video. The first verification information is generated by the cloud device based on the feature fusion watermark corresponding to the GOP. The feature fusion watermark is obtained by the cloud device by fusing the structural features of the GOP. The structural features include global features and local features. The global features are used to characterize the overall structure of the watermarked video, and the local features include the structural attributes and statistical features of the GOP. The determining unit is used to determine the second verification information corresponding to each GOP based on the structural characteristics of each GOP; The verification unit is used to verify whether the content of the watermarked video has been tampered with, based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP.

[0008] According to an embodiment of the fifth aspect of this application, a content anti-tampering detection system with multi-feature fusion is provided, comprising: a cloud device and at least one edge device; the cloud device and the at least one edge device are communicatively connected. The cloud device is used to perform the method as described in the first aspect; Either end device is used to perform the method described in the second aspect.

[0009] As can be seen from the above technical solutions, the proposed solution obtains the structural features of each video group (GOP) in the video to be played, fuses the structural features to generate first verification information, embeds the first verification information representing the structural features of each GOP into each GOP, and sends it to the edge device. This allows the edge device to determine whether the video content has been tampered with based on the first verification information. Since the first verification information records information representing the structural features of the GOP, any tampering with the GOP will affect this feature, causing the verification information determined in the same way to change. Therefore, the edge device can determine whether the video content has been tampered with based on the first verification information, improving the accuracy of video content security detection. Attached Figure Description

[0010] Figure 1 A flowchart illustrating the content anti-tampering detection method using multi-feature fusion provided in this application embodiment; Figure 2 A flowchart illustrating another multi-feature fusion content anti-tampering detection method provided in this application embodiment; Figure 3 This is a schematic diagram of the overall process provided for an embodiment of this application; Figure 4 This is a schematic diagram of the overall modules provided in the embodiments of this application; Figure 5 A structural diagram of the content anti-tampering detection device with multi-feature fusion provided in the embodiments of this application; Figure 6 A structural diagram of another multi-feature fusion content anti-tampering detection device provided in this application embodiment; Figure 7 This application provides an architecture diagram of a content anti-tampering detection system based on multi-feature fusion. Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0011] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, and to make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions in the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0012] Currently, in order to prevent video content from being tampered with during video transmission, a verification code is usually created on the video file on the cloud device, inserted into the video, and then sent to the edge device (i.e., the video playback end) for storage.

[0013] Specifically, an MD5 or SHA256 checksum can be created for each video file on a cloud device. After obtaining the checksum, the cloud device inserts it into the video (e.g., embedding it at a specific location such as the beginning or end of the video file) and distributes it to edge devices for storage. Upon receiving the video file from the cloud device, the edge device extracts the embedded checksum from the video file and compares it with the locally stored checksum. If they match, the video file is considered intact; otherwise, the video file has been tampered with during transmission.

[0014] However, if the video carrying the verification code is intercepted by an attacker during transmission, the attacker can replace the video content and send it to the edge device with the same verification code. In this case, the edge device cannot verify whether the video content has been tampered with.

[0015] In another feasible approach, the cloud device segments the video file to be played. For each segment except the first one, the cloud device generates a checksum based on the content of the current segment and the content of the previous segment (e.g., by performing a hash operation after concatenating the data from the two segments), and inserts the checksum into the corresponding video segment. Each time the edge device needs to play the video file, it needs to download the segments sequentially. For each downloaded segment, the edge device recalculates the checksum based on the content of the current segment and the previous segment, and compares it with the embedded checksum. Only when the current segment passes verification will the download and verification of the next segment continue, thus ensuring the continuity and integrity of the video stream.

[0016] However, if the video file is intercepted by an attacker, the attacker can analyze the data structure in the slice to determine the checksum generation method, tamper with the video file, and forge a checksum according to the checksum generation method and add it to the corresponding slice. At this time, the edge device will recalculate the checksum based on the content of the current slice and the content of the previous slice, and will still be unable to verify whether the video content has been tampered with.

[0017] Based on this, this application proposes a multi-feature fusion content anti-tampering detection method to improve the accuracy of edge devices in verifying video content.

[0018] Please refer to Figure 1 , Figure 1 A flowchart illustrating the content anti-tampering detection method using multi-feature fusion provided in this application embodiment.

[0019] In this embodiment, the method can be applied to cloud devices. Cloud devices can be servers or service clusters with strong computing and storage capabilities deployed at the network center or remotely. For example, in a security monitoring system, cloud devices can be video intelligent analysis platforms or cloud storage management servers deployed in data centers, etc. This application does not impose any limitations on this.

[0020] like Figure 1 As shown, the method may include the following steps: Step 101: For each video group GOP in the video to be played, obtain the structural features of that GOP.

[0021] In this embodiment, the video to be played can be divided into multiple consecutive GOPs (Groups of Pictures) during encoding. Each GOP is a basic independent unit for video encoding, decoding, and storage. By parsing the encoded bitstream of the video, the cloud device can identify the start and end positions of each GOP, thereby dividing the video to be played into multiple consecutive GOPs for subsequent processing.

[0022] Each Group of Pictures (GOP) can consist of an intra picture (I-frame) and one or more predicted pictures (P-frames) that follow it. The I-frame is usually the first frame of the GOP and contains complete picture information; the P-frames are encoded by referring to the preceding I-frames or P-frames and mainly record the change information between the pictures.

[0023] In this embodiment, the structural features of a GOP may include global features and local features. Global features are used to characterize the overall structure of the video to be played, while local features include the structural attributes and statistical features of the GOP.

[0024] As an example, a GOP includes an internal picture frame (I-frame) and multiple predicted frames (P-frames); global features include the total number of I-frames included in the video to be played; local features include at least one of the following: the frame spacing between the I-frame and the first P-frame in the GOP, the size of the I-frame in the GOP, the total size of all P-frames in the GOP, and the number of P-frames in the GOP.

[0025] Specifically, for each identified GOP, the cloud device can extract its structural features. These structural features include both global and local features.

[0026] Among them, global features are used to characterize the overall structural properties of the entire video to be played. For example, the total number of I-frames contained in the video to be played is a global scalar for the entire video file.

[0027] Local features are used to characterize the coding structure and data volume attributes of the current GOP itself, which can specifically include the structural attributes and statistical features of the GOP.

[0028] Structural attributes refer to features that describe the relationships between frames within a group of frames (GOPs). For example, the inter-frame spacing between an I-frame and its adjacent P-frame within the GOP.

[0029] Statistical characteristics refer to the features that describe the data volume of each frame within a GOP. For example, the size of I-frames in the GOP, the total size of all P-frames in the GOP, and the number of P-frames contained in the GOP.

[0030] In this embodiment, extracting the structural features of a GOP only requires acquiring data such as frame type and size, consuming very few resources to achieve real-time processing of massive video streams. Furthermore, since the extracted GOP structural features are inherent attributes of the video file, any addition, deletion, or replacement of video content will cause changes in the feature values, making it easier to detect subtle or covert tampering by attackers.

[0031] This concludes the description of step 101. We will now proceed to step 102.

[0032] Step 102: Fuse the structural features of the GOP to obtain the feature fusion watermark corresponding to the GOP.

[0033] In this embodiment, after obtaining the structural features of each GOP through step 101, for each GOP, the multi-dimensional structural features of the GOP can be fused to obtain a feature fusion watermark used to characterize the comprehensive features of the GOP.

[0034] Specifically, methods for generating feature fusion watermarks may include: Based on the mean and standard deviation of each feature value included in the structural features, the feature values ​​included in the structural features of the GOP are standardized to obtain the standardized feature values. Based on the standardized feature values ​​included in the structural features of the GOP, and the first principal component weights corresponding to each feature value, the feature fusion watermark of the GOP is obtained; the first principal component weight corresponding to any feature value is used to indicate the importance of the structural feature corresponding to that feature value in generating the feature fusion watermark. The structural features include the mean, standard deviation, and first principal component weights of each feature value, which are determined based on the feature values ​​included in the structural features of all GOPs in the video to be played.

[0035] As an example, the PCA principal component analysis algorithm can be used to perform feature fusion on the keyframe features obtained in step 1 to generate fused features that can characterize the GOP group to which the keyframe belongs.

[0036] Specifically, it can be from the video to be played. Extract the structural features of each GOP video group.

[0037] For the The original feature vectors extracted from each GOP group It can be represented as follows:

[0038] in This video indicates The total number of all I-frames included; This represents the frame spacing between an internal frame I and an adjacent predicted frame P in the k-th GOP. This indicates the size of the I-frame in bytes within the k-th GOP group; This represents the total size in bytes of all P-frames in the k-th GOP group; This represents the total number of frames between the I-frame of the k-th GOP and the I-frame of the next GOP (i.e., the number of P-frames contained in the k-th GOP).

[0039] Based on the original feature vector Calculate each feature dimension in the entire dataset. mean and standard deviation :

[0040]

[0041] in, This indicates the total number of GOP groups in the video. Represents the original feature vector The feature value of the j-th dimension in the matrix.

[0042] After obtaining the mean and standard deviation of the feature values ​​for each feature dimension, each feature can be standardized:

[0043] in, This represents the standardized eigenvalue of the j-th dimension. The above standardization operation is performed on the eigenvalues ​​of each dimension to obtain the standardized eigenvalue of the j-th dimension. The feature vectors of a group of GOPs, taking an example with 5-dimensional feature values:

[0044] in, Indicates the first The standardized feature vectors of each GOP group These represent the standardized feature values ​​corresponding to the 1st, 2nd, 3rd, 4th, and 5th dimensions in the kth GOP group, respectively.

[0045] For each group of Groups of Pictures (GOPs), the above feature extraction and normalization process is performed. After obtaining the normalized feature vectors of each GOP, the normalized feature vectors of each GOP can be arranged and combined according to the GOP order to form a normalized feature matrix. Standardized feature matrix In the diagram, each row corresponds to a normalized feature vector of a GOP, and each column corresponds to a feature of a specific dimension (such as I-frame size, number of P-frames, etc.).

[0046] Determine the covariance matrix corresponding to the standardized feature matrix based on the standardized feature matrix:

[0047] in, It is a 5×5 covariance matrix (taking the original feature vector as an example with 5 eigenvalues). yes The standardized characteristic matrix is ​​5×5. Covariance matrix. Used to characterize the relationship and intensity of change between various feature dimensions.

[0048] The covariance matrix is ​​calculated below. Eigenvalues ​​and eigenvectors:

[0049] Where λ refers to the eigenvalues ​​of the covariance matrix. It refers to the eigenvectors of the covariance matrix.

[0050] Solving for each eigenvalue λ and its corresponding multiple eigenvectors yields a set of eigenvalues ​​λ and their corresponding multiple eigenvectors. Then, the eigenvalues Sort by size from smallest to largest, corresponding feature vectors This refers to the direction of the unit principal component. Select the largest eigenvalue. corresponding feature vector As the first principal component.

[0051]

[0052] in, Indicates the first principal component in the th order. Weights on each feature dimension.

[0053] The normalized feature vector of each GOP Projected onto the direction of the first principal component The resulting scalar is the fusion feature of the GOP. :

[0054] Combining and organizing the above two steps, we obtain the first... Final fusion features of each GOP for:

[0055] Should That is, the first Each GOP's feature fusion watermark .

[0056] In this embodiment, as can be seen from the above formula, any change to the GOP group will affect the original eigenvalues. This leads to changes in the feature fusion watermark.

[0057] This concludes the description of step 102. We will now proceed to step 103.

[0058] Step 103: Generate the first verification information corresponding to the GOP based on the feature fusion watermark corresponding to the GOP, and embed the first verification information into the GOP.

[0059] In this embodiment, the feature fusion watermark can be directly determined as the first verification information corresponding to the GOP, or the feature fusion watermark can be encrypted to obtain the first verification information.

[0060] As an example, a specified key can be used to encrypt the feature fusion watermark to generate first verification information.

[0061] Specifically, the specified key here can be the public key in an asymmetric encryption algorithm; for example, it can be used to fuse generated features with a watermark. Perform RSA asymmetric encryption.

[0062] For the general form of RSA encryption public keys It can be represented as follows:

[0063]

[0064]

[0065] in and For randomly generated large prime numbers, The value of Euler's totient function. The encryption exponent is typically 65537, and n is the publicly known modulus used to characterize the key length.

[0066] For the fusion feature watermark of the k-th GOP First, normalization is performed:

[0067] in, This refers to the minimum value of the fused feature watermark of all GOPs in the entire video file. This refers to the maximum value of the fused feature watermark of all GOPs in the entire video file. This refers to the normalized watermark value of the k-th GOP (this value is between 0 and 1).

[0068] Furthermore, the normalized watermark value can be mapped to discrete integer values ​​in the integer field:

[0069] in, This refers to the fused feature watermark after integerization.

[0070] After obtaining integer-based fusion feature watermark Then, the fused feature watermark can be encrypted:

[0071] in, It is an encrypted feature fusion watermark. It is an RSA encryption public key.

[0072] Generate encrypted feature fusion watermark Then, the watermark can be... Embedded into the I-frame of the corresponding GOP group.

[0073] This concludes the description of step 103. We will now proceed to step 104.

[0074] Step 104: The watermarked video generated based on each GOP with embedded first verification information is sent to the edge device, so that the edge device can verify whether the content of the watermarked video has been tampered with based on the first verification information corresponding to each GOP in the watermarked video.

[0075] In this embodiment, after generating first verification information for each GOP in step 103 and embedding the first verification information into the GOP, all processed GOPs can be sequentially combined and packaged into a complete watermarked video file. .

[0076] In step 104, the watermarked video can be... Send to the edge device for subsequent verification.

[0077] As an example, if the first verification information is an unencrypted feature fusion watermark, the cloud device will generate a video with the feature fusion watermark. Send it to the edge device for verification.

[0078] If the first verification information is an encrypted feature-fused watermark, the cloud device will generate a video with the feature-fused watermark. and encryption key All data is distributed to edge playback devices for verification. The verification process for edge devices will be detailed below and will not be repeated here.

[0079] This concludes the discussion on... Figure 1 The description.

[0080] The proposed solution obtains the structural features of each Group of Pictures (GOPs) in the video to be played, fuses these features to generate first verification information, embeds the first verification information representing the structural features of each GOP into each GOP, and sends it to the edge device. This allows the edge device to determine whether the video content has been tampered with based on the first verification information. Since the first verification information records information representing the structural features of the GOP, any tampering with the GOP will affect this feature, causing the verification information determined in the same way to change. Therefore, the edge device can determine whether the video content has been tampered with based on the first verification information, improving the accuracy of video content security detection.

[0081] Please refer to Figure 2 , Figure 2 A flowchart illustrating another multi-feature fusion content anti-tampering detection method provided in this application embodiment.

[0082] In this embodiment, the method is applied to an edge device. The edge device can be a user-side terminal device used to receive, decode, and play video content, such as a smartphone or a monitoring client.

[0083] like Figure 2 As shown, the method includes the following steps: Step 201: Receive the watermarked video sent by the cloud device and extract the first verification information embedded in each video group GOP in the watermarked video.

[0084] The first verification information is generated by the cloud device based on the feature fusion watermark corresponding to the GOP; the feature fusion watermark is obtained by the cloud device by fusing the structural features of the GOP; the structural features include global features and local features, the global features are used to characterize the overall structure of the watermarked video, and the local features include the structural attributes and statistical features of the GOP.

[0085] In this embodiment, after receiving the watermarked video sent by the cloud device, the edge device can extract the first verification information embedded in each video group (GOP) in the watermarked video to determine whether the video content has been tampered with.

[0086] As one embodiment, receiving a watermarked video sent by a cloud device and extracting the first verification information embedded in each video group (GOP) of the watermarked video specifically includes: If any GOP is found to lack the first verification information, it is determined that the watermarked video has been tampered with. If it is detected that all GOPs contain the first verification information, then the step of determining the second verification information corresponding to each GOP based on the structural characteristics of each GOP is executed.

[0087] In this embodiment, if at least one GOP is found to lack first verification information when extracting the first verification information embedded in each GOP group in the watermarked video, it indicates that the GOP has been tampered with and does not carry the first verification information used for verification. In this case, it can be directly determined that the watermarked video has been tampered with.

[0088] If the first verification information is detected to be embedded in all GOPs, then step 202 can be further executed to verify the authenticity of the first verification information.

[0089] This concludes the description of step 201. We will now proceed to step 202.

[0090] Step 202: Based on the structural characteristics of each GOP, determine the second verification information corresponding to each GOP.

[0091] In this embodiment, after determining that the first verification information exists in all GOPs, the second verification information corresponding to each GOP can be determined based on the structural characteristics of each received GOP.

[0092] Specifically, for each GOP, the edge device can obtain the structural features used by the cloud device when generating the feature fusion watermark corresponding to that GOP; The structural features of the GOP are fused according to the feature fusion method used when the cloud device generates the feature fusion watermark corresponding to the GOP to obtain the reference watermark corresponding to the GOP.

[0093] In this embodiment, the edge device can generate a reference watermark corresponding to each GOP based on the received GOP, using the same method as the feature fusion watermark generated by the cloud device, according to the pre-agreed structural features used when generating the watermark.

[0094] As an example, if the edge device does not receive the specified key sent by the cloud device, it indicates that the cloud device directly uses the fused feature watermark as the first verification information. In this case, the edge device can also directly use the determined reference watermark as the second verification information.

[0095] If the edge device receives the specified key sent by the cloud device, it indicates that the cloud device has encrypted the fused feature watermark as the first verification information. At this time, the edge device can encrypt the reference watermark based on the specified key to obtain the second verification information corresponding to the GOP.

[0096] This concludes the description of step 202. We will now proceed to step 203.

[0097] Step 203: Verify whether the content of the watermarked video has been tampered with based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP.

[0098] In this embodiment, for each GOP, the second verification information obtained in step 202 can be compared with the first verification information carried in the GOP to verify whether the video content has been tampered with.

[0099] Specifically, if the second verification information corresponding to any GOP does not match the first verification information extracted from that GOP, it is determined that the content of the watermarked video has been tampered with. If the second verification information corresponding to each GOP is detected to match the first verification information extracted from that GOP, then it is determined that the content of the watermarked video has not been tampered with.

[0100] In this embodiment, if a GOP is found where the second verification information does not match the corresponding first verification information, it indicates that the content of the watermarked video has been tampered with, and the tampered position is located at that GOP.

[0101] If the second verification information corresponding to all GOPs is found to match the first verification information extracted from that GOP, it indicates that the verification was successful and the content of the watermarked video has not been tampered with.

[0102] This concludes the description of step 203.

[0103] In this embodiment, videos that successfully extract and verify feature fusion watermarks can be played normally on edge devices; videos that fail to extract or verify feature fusion watermarks can be alerted and their playback on edge devices prohibited.

[0104] This concludes the discussion on... Figure 2 The description.

[0105] This application uses PCA feature fusion watermarking to detect and control video content, and adopts a cloud-edge collaborative detection mechanism that combines cloud and edge devices, thereby improving the efficiency and accuracy of video content security detection.

[0106] Specifically, this application employs the PCA principal component analysis algorithm to extract and fuse key features of video content to the greatest extent possible. This avoids the insensitivity to minor and subtle tampering issues caused by feature loss due to video slicing in existing methods, while significantly improving detection efficiency and reducing resource consumption. Furthermore, this method utilizes a cloud-edge collaborative detection mechanism, saving edge computing resources and improving detection accuracy.

[0107] In this embodiment, even if an attacker can intercept the transmitted video file, since it is impossible to know which structural features were extracted to generate the first verification information, even if the attacker tampers with the video file, it is impossible to forge the first verification information. The edge device can verify the first verification information carried in the video file based on the second verification information generated based on the structural features. As long as the second verification information generated by any GOP does not match the first verification information carried by that GOP, it can be determined that the video file has been tampered with, and specifically the tampered GOP can be located.

[0108] The following is through Figure 3 as well as Figure 4 The proposed solution in this application is described in its entirety.

[0109] Please refer to Figure 3 , Figure 3 This is a schematic diagram of the overall process provided for an embodiment of this application.

[0110] like Figure 3 As shown, the specific steps of the content anti-tampering detection method proposed in this application through multi-feature fusion are as follows: Cloud devices obtain videos to be played and the video to be played Structural features are extracted from each GOP video group.

[0111] The PCA principal component analysis algorithm is used to fuse all extracted structural features to generate a fused feature watermark for each GOP group. The fused feature watermark is generated by encrypting it using key K. (First verification information) the watermark Embedded into the I-frames of the corresponding GOP group to generate a video with fused feature watermarks. .

[0112] The cloud device will have the key and videos with watermarks Distribute to peripheral playback devices.

[0113] When the edge device receives a watermarked video Afterwards, the received video content Watermark extraction is performed on each GOP (first verification information is extracted). If in the video Watermark could not be extracted from the GOP group If this is the case, then it is considered that the video frame content in that GOP video group has been tampered with or replaced, and that video... Content deemed to have been tampered with will not be played (not shown in the image).

[0114] Meanwhile, edge devices obtain the structural features of each Group of Pictures (GOP) in the same way as cloud devices, and generate fused feature watermarks. (Referencing the watermark), the fused feature watermark is further encrypted using key K to obtain the encrypted watermark. (Second verification information).

[0115] For each GOP, if the corresponding GOP and If the results are the same, the video content to be played is considered normal; otherwise, the video content is considered to have been tampered with.

[0116] In this embodiment, the edge device can play normal video content, block tampered content (i.e., not play it), and issue an alarm.

[0117] This concludes the discussion on... Figure 3 The description.

[0118] Please refer to Figure 4 , Figure 4 This is a schematic diagram of the overall modules provided for an embodiment of this application.

[0119] like Figure 4 As shown, the solution proposed in this application can be implemented through the following six modules: (1) Feature extraction module The feature extraction module is used to extract features from normal videos containing consecutive frames. Extracting structural features from each GOP: This video Total number of I-frames The inter-frame spacing between I-frames and P-frames in this GOP The size of the I-frames in this GOP group (in bytes). The total size in bytes of all P-frames in this GOP group. The total number of I-frames between the current GOP group and the next GOP group. .

[0120] (2) Feature fusion watermark generation module The feature fusion watermark generation module is used to fuse the extracted features using the PCA principal component analysis algorithm to generate a fused feature watermark. .

[0121] (3) Integration of watermark encryption module The integrated watermark encryption module is used to transmit via key The above-generated merged watermark Encryption is performed to generate an encrypted merged watermark. and the watermark Embedded into the corresponding keyframes to generate a watermarked video file. .

[0122] (4) Video and Key Distribution Module The video and key distribution module is used to distribute the above keys. and watermarked video files Distribute to peripheral playback devices.

[0123] (5) Edge watermark extraction and verification module The edge watermark extraction and verification module is used to obtain the aforementioned video file. Watermark extraction and verification were then performed.

[0124] If watermark information cannot be extracted from a GOP, the video is considered to have been tampered with and is therefore not allowed to be played. If watermark information is extracted from all I-frames in a GOP... Then, feature watermarking is performed on the GOP using the same feature fusion method as the cloud device. Generation, if using a key encryption The feature watermark obtained later and If the watermarks are the same, the video content to be played is considered normal; otherwise, the video content is considered to have been tampered with.

[0125] (6) Blocking and Playback Module The blocking and playback module is used to play videos that have successfully extracted and verified the watermark normally; and to block videos that have failed to extract or verify the watermark and issue an alarm.

[0126] This concludes the discussion on... Figure 4 The description.

[0127] Please refer to Figure 5 , Figure 5 This is a structural diagram of a content anti-tampering detection device with multi-feature fusion proposed in an embodiment of this application. This device is applied to cloud devices. Figure 5 As shown, the device may include an acquisition unit 501, a fusion unit 502, a generation unit 503, and a transmission unit 504. Specifically, the device includes: The obtaining unit 501 is used to obtain the structural features of each video group GOP in the video to be played. The structural features include global features and local features. The global features are used to characterize the overall structure of the video to be played, and the local features include the structural attributes and statistical features of the GOP. The fusion unit 502 is used to fuse the structural features of the GOP to obtain the feature fusion watermark corresponding to the GOP; The generation unit 503 is used to generate first verification information corresponding to the GOP based on the feature fusion watermark corresponding to the GOP, and embed the first verification information into the GOP; The sending unit 504 is used to send the watermarked video generated based on each GOP with embedded first verification information to the edge device, so that the edge device can verify whether the content of the watermarked video has been tampered with based on the first verification information corresponding to each GOP in the watermarked video.

[0128] As an example, a GOP includes an internal picture frame (I-frame) and multiple predicted frames (P-frames); global features include the total number of I-frames included in the video to be played; local features include at least one of the following: the frame spacing between the I-frame and the first P-frame in the GOP, the size of the I-frame in the GOP, the total size of all P-frames in the GOP, and the number of P-frames in the GOP.

[0129] As one embodiment, the fusion unit 502 is specifically used for: Based on the average value and standard deviation of each feature value included in the structural features, the feature values ​​included in the structural features of the GOP are standardized to obtain standardized feature values. Based on the standardized feature values ​​included in the structural features of the GOP, and the first principal component weights corresponding to each feature value, the feature fusion watermark of the GOP is obtained; the first principal component weight corresponding to any feature value is used to indicate the importance of the structural feature corresponding to that feature value in generating the feature fusion watermark. The structural features include the mean, standard deviation, and first principal component weights of each feature value, which are determined based on the feature values ​​included in the structural features of all GOPs in the video to be played.

[0130] As one embodiment, the generation unit 503 is specifically used for: The feature fusion watermark is encrypted using a specified key to generate the first verification information; The transmitting unit 504 is specifically used for: The watermarked video generated based on each GOP with embedded first verification information, along with the specified key, is sent to the edge device.

[0131] This concludes the process. Figure 5 Description of the device.

[0132] Please refer to Figure 6 , Figure 6 This is a structural diagram of another multi-feature fusion content anti-tampering detection device proposed in an embodiment of this application. This device is applied to edge devices. Figure 6 As shown, the device may include a receiving unit 601, a determining unit 602, and a verification unit 603. Specifically, the device includes: The receiving unit 601 is used to receive the watermarked video sent by the cloud device and extract the first verification information embedded in each video group (GOP) in the watermarked video. The first verification information is generated by the cloud device based on the feature fusion watermark corresponding to the GOP. The feature fusion watermark is obtained by the cloud device by fusing the structural features of the GOP. The structural features include global features and local features. The global features are used to characterize the overall structure of the watermarked video, and the local features include the structural attributes and statistical features of the GOP. The determining unit 602 is used to determine the second verification information corresponding to each GOP based on the structural characteristics of each GOP; Verification unit 603 is used to verify whether the content of the watermarked video has been tampered with, based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP.

[0133] As one embodiment, the receiving unit 601 is specifically used for: If any GOP is found to lack the first verification information, it is determined that the watermarked video has been tampered with. If it is detected that all GOPs contain embedded first verification information, then the step of determining the second verification information corresponding to each GOP based on the structural characteristics of each GOP is executed.

[0134] As an example, the edge device also receives a specified key sent by the cloud device; the first verification information is obtained by the cloud device encrypting the feature watermark corresponding to the GOP using the specified key; the determining unit 602 is specifically used for: For each GOP, obtain the structural features used by the cloud device when generating the feature fusion watermark corresponding to that GOP; The structural features of the GOP are fused according to the feature fusion method used when the cloud device generates the feature fusion watermark corresponding to the GOP to obtain the reference watermark corresponding to the GOP. The reference watermark is encrypted using the specified key to obtain the second verification information corresponding to the GOP.

[0135] As one embodiment, the verification unit 603 is specifically used for: If the second verification information corresponding to any GOP does not match the first verification information extracted from that GOP, it is determined that the content of the watermarked video has been tampered with. If the second verification information corresponding to each GOP is detected to match the first verification information extracted from that GOP, then it is determined that the content of the watermarked video has not been tampered with.

[0136] This concludes the discussion on... Figure 6 Description of the device.

[0137] This embodiment also proposes a content anti-tampering detection system based on multi-feature fusion. Please refer to [link / reference]. Figure 7 , Figure 7 This is a diagram illustrating the architecture of a content anti-tampering detection system based on multi-feature fusion, as provided in an embodiment of this application.

[0138] like Figure 7 As shown, the system may include a cloud device and at least one edge device (e.g., edge device 1, edge device 2, ... edge device m, where m is greater than or equal to 1), wherein the cloud device communicates with each edge device.

[0139] Specifically, cloud devices are used to perform tasks such as Figure 1 The method shown; either end device is used to perform, as Figure 2 The method shown.

[0140] In this embodiment, the multi-feature fusion content anti-tampering detection method executed by the cloud device and the edge device in the multi-feature fusion content anti-tampering detection system has been implemented. Figure 1 as well as Figure 2 The details are described in detail elsewhere, so they will not be repeated here.

[0141] This application also provides embodiments that... Figure 5 , Figure 6 Hardware structure description of the illustrated device. This hardware structure is... Figure 5 or Figure 6 The structure in the illustrated electronic device. Please refer to [link / reference]. Figure 8 , Figure 8 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 8 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0142] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0143] For example, the aforementioned computer-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, computer-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

Claims

1. A content anti-tampering detection method based on multi-feature fusion, characterized in that, This method is applied to cloud devices, and includes: For each group of videos (GOPs) in the video to be played, the structural features of the GOP are obtained. The structural features include global features and local features. The global features are used to characterize the overall structure of the video to be played, and the local features include the structural attributes and statistical features of the GOP. The GOP includes one internal frame (I-frame) and multiple predicted frames (P-frames). The global features include the total number of I-frames in the video to be played. The local features include at least one of the following: the frame interval between the I-frame and the first P-frame in the GOP, the size of the I-frame in the GOP, the total size of all P-frames in the GOP, and the number of P-frames in the GOP. Based on the average value and standard deviation of each feature value included in the structural features, the feature values ​​included in the structural features of the GOP are standardized to obtain standardized feature values. Based on the standardized feature values ​​included in the structural features of the GOP, and the first principal component weights corresponding to each feature value, a feature fusion watermark for the GOP is obtained; the first principal component weight corresponding to any feature value is used to indicate the importance of the structural feature corresponding to that feature value in generating the feature fusion watermark; wherein, the mean, standard deviation, and first principal component weights of each feature value included in the structural features are determined based on the feature values ​​included in the structural features of all GOPs in the video to be played; Based on the feature fusion watermark corresponding to the GOP, the first verification information corresponding to the GOP is generated, and the first verification information is embedded in the GOP; The watermarked video generated based on each GOP with embedded first verification information is sent to the edge device, so that the edge device can verify whether the content of the watermarked video has been tampered with based on the first verification information corresponding to each GOP in the watermarked video.

2. The method according to claim 1, characterized in that, The step of generating the first verification information corresponding to the GOP based on the feature fusion watermark includes: The feature fusion watermark is encrypted using a specified key to generate the first verification information; The step of sending the watermarked video generated based on each GOP with embedded first verification information to the edge device includes: The watermarked video generated based on each GOP with embedded first verification information and the specified key are sent to the edge device.

3. A content anti-tampering detection method based on multi-feature fusion, characterized in that, This method is applied to edge devices, and the method includes: The system receives a watermarked video sent by a cloud device and extracts the first verification information embedded in each group of videos (GOPs) in the watermarked video. The first verification information is generated by the cloud device based on the feature-fused watermark corresponding to the GOP. The feature-fused watermark is obtained by the cloud device by fusing the structural features of the GOP. The structural features include global features and local features. The global features characterize the overall structure of the watermarked video, and the local features include the structural attributes and statistical features of the GOP. The GOP includes one internal frame (I-frame) and multiple predicted frames (P-frames). The global features include the total number of I-frames included in the video to be played. The local features include at least one of the following: the frame interval between the I-frame and the first P-frame in the GOP, and the I-frame in the GOP... The size of the GOP, the total size of all P-frames in the GOP, and the number of P-frames in the GOP; wherein, the feature fusion watermark is obtained by the cloud device based on the average value and standard deviation of each feature value included in the structural features of the GOP, standardizing each feature value included in the structural features of the GOP to obtain standardized feature values, and obtaining them according to each standardized feature value included in the structural features of the GOP, and the first principal component weight corresponding to each feature value; the first principal component weight corresponding to any feature value is used to indicate the importance of the structural feature corresponding to the feature value in generating the feature fusion watermark; wherein, the average value, standard deviation, and first principal component weight of each feature value included in the structural features are determined according to the feature values ​​included in the structural features of all GOPs in the video to be played; Based on the structural characteristics of each GOP, the second verification information corresponding to each GOP is determined; wherein, the method by which the edge device determines the second verification information is the same as the method by which the cloud device generates the first verification information. Based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP, verify whether the content of the watermarked video has been tampered with.

4. The method according to claim 3, characterized in that, The method of receiving watermarked videos sent by cloud devices and extracting the first verification information embedded in each group of videos (GOPs) within the watermarked videos includes: If it is detected that the first verification information is not embedded in any GOP, then it is determined that the watermarked video has been tampered with; If it is detected that the first verification information is embedded in all GOPs, then the step of determining the second verification information corresponding to each GOP based on the structural features of each GOP is executed.

5. The method according to claim 3, characterized in that, The edge device also receives a specified key sent by the cloud device; the first verification information is obtained by the cloud device encrypting the feature watermark corresponding to the GOP using the specified key; the step of determining the second verification information corresponding to each GOP based on the structural features of each GOP includes: For each GOP, obtain the structural features used by the cloud device when generating the feature fusion watermark corresponding to that GOP; The structural features of the GOP are fused according to the feature fusion method used when the cloud device generates the feature fusion watermark corresponding to the GOP to obtain the reference watermark corresponding to the GOP. The reference watermark is encrypted using the specified key to obtain the second verification information corresponding to the GOP.

6. The method according to claim 3, characterized in that, The step of verifying whether the content of the watermarked video has been tampered with based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP includes: If the second verification information corresponding to any GOP does not match the first verification information extracted from that GOP, it is determined that the content of the watermarked video has been tampered with. If the second verification information corresponding to each GOP is detected to match the first verification information extracted from that GOP, then it is determined that the content of the watermarked video has not been tampered with.

7. A content anti-tampering detection device based on multi-feature fusion, characterized in that, This device is used in cloud-based devices and includes: The obtaining unit is configured to obtain the structural features of each Group of Pictures (GOPs) in the video to be played. The structural features include global features and local features. The global features characterize the overall structure of the video to be played, and the local features include the structural attributes and statistical features of the GOP. Each GOP includes one internal frame (I-frame) and multiple predicted frames (P-frames). The global features include the total number of I-frames in the video to be played. The local features include at least one of the following: the frame interval between the I-frame and the first P-frame in the GOP, the size of the I-frame in the GOP, the total size of all P-frames in the GOP, and the number of P-frames in the GOP. A fusion unit is used to standardize the feature values ​​of a GOP based on the average value and standard deviation of each feature value included in the structural features, to obtain standardized feature values; and to obtain a feature fusion watermark for the GOP based on the standardized feature values ​​included in the structural features of the GOP and the first principal component weights corresponding to each feature value; the first principal component weights corresponding to any feature value are used to indicate the importance of the structural feature corresponding to that feature value in generating the feature fusion watermark; wherein, the average value, standard deviation, and first principal component weights of each feature value included in the structural features are determined based on the feature values ​​included in the structural features of all GOPs in the video to be played. The generation unit is used to generate first verification information corresponding to the GOP based on the feature fusion watermark corresponding to the GOP, and embed the first verification information into the GOP; The sending unit is used to send the watermarked video generated based on each GOP with embedded first verification information to the edge device, so that the edge device can verify whether the content of the watermarked video has been tampered with based on the first verification information corresponding to each GOP in the watermarked video.

8. A content anti-tampering detection device with multi-feature fusion, characterized in that, This device is used in edge devices, and the device includes: A receiving unit is configured to receive a watermarked video sent by a cloud device and extract first verification information embedded in each Group of Pictures (GOPs) of the watermarked video. The first verification information is generated by the cloud device based on the feature-fused watermark corresponding to the GOP. The feature-fused watermark is obtained by the cloud device by fusing the structural features of the GOP. The structural features include global features and local features. The global features characterize the overall structure of the watermarked video, and the local features include the structural attributes and statistical features of the GOP. The GOP includes one internal frame (I-frame) and multiple predicted frames (P-frames). The global features include the total number of I-frames included in the video to be played. The local features include at least one of the following: the frame interval between the I-frame and the first P-frame in the GOP, the GOP's... The size of the I-frame in the GOP, the total size of all P-frames in the GOP, and the number of P-frames in the GOP; wherein, the feature fusion watermark is obtained by the cloud device based on the average value and standard deviation of each feature value included in the structural features of the GOP, standardizing each feature value included in the structural features of the GOP to obtain standardized feature values, and obtaining them according to each standardized feature value included in the structural features of the GOP, and the first principal component weight corresponding to each feature value; the first principal component weight corresponding to any feature value is used to indicate the importance of the structural feature corresponding to the feature value in generating the feature fusion watermark; wherein, the average value, standard deviation, and first principal component weight of each feature value included in the structural features are determined according to the feature values ​​included in the structural features of all GOPs in the video to be played; The determining unit is used to determine the second verification information corresponding to each GOP based on the structural characteristics of each GOP; wherein, the edge device determines the second verification information in the same way as the cloud device generates the first verification information; The verification unit is used to verify whether the content of the watermarked video has been tampered with, based on the second verification information corresponding to each GOP and the first verification information extracted from each GOP.

9. A content anti-tampering detection system based on multi-feature fusion, characterized in that, include: A cloud device and at least one edge device; the cloud device is communicatively connected to the at least one edge device; The cloud device is used to perform the method as described in claim 1 or 2; Any of the end devices is used to perform the method as described in any one of claims 3 to 6.

Images