Mobile social multimedia copyright verification and active traceability tracking method

Abstract

The invention relates to the technical field of multimedia data security protection, in particular to a mobile social multimedia copyright verification and active traceability tracking method, which comprises the following steps of: 1, protecting multimedia data by combining social fingerprints, watermarks and encryption; and 2, tracing and tracking monitoring are carried out on shared content causing privacy leakage by depending on dynamic fingerprint detection, a multimedia security and privacy protection method combining social fingerprints, watermarks and encryption is adopted, and the copyright ownership can be verified through combined application of the social fingerprints and the watermarks; the same copy can be distributed to multiple users at the same time to perform active traceability tracking so as to deter malicious distribution behaviors, encryption is performed in a Haar wavelet domain, a block DCT domain and a spatial domain of a tree structure, different levels of security requirements can be met, and a traceability tracking algorithm not only can ensure confidentiality in a communication process, but also can improve the security of the communication process. And copyright verification and active traceability tracking in the sharing process can be ensured, so that triple protection of safety and privacy is achieved.

Description

Technical Field

[0001] This invention relates to the field of multimedia data security protection technology, specifically to a method for mobile social multimedia copyright verification and proactive source tracing. Background Technology

[0002] With the rapid development of multimedia social networks and mobile communication technologies, numerous social networks have emerged. Through social network services (SNS), mobile users can share / access social multimedia services anytime, anywhere, making communication faster and more convenient. Mobile multimedia social networks have made the storage, copying, and dissemination of social multimedia extremely prevalent. While this brings convenience, it also causes serious security and privacy issues.

[0003] Currently, both domestic and international source tracing technologies almost exclusively rely on passive digital fingerprinting. This approach is severely inadequate in supporting real-time content distribution on mobile social networks. Furthermore, the quality of multimedia services varies across different mobile devices, making existing digital fingerprinting systems unscalable for mobile multimedia social networks. Therefore, finding a way to support secure sharing of mobile social multimedia on resource-constrained mobile devices, enabling proactive source tracing and dynamic monitoring, is of significant practical importance. While end-to-end encryption can control unauthorized access, decrypted content can still be freely copied and redistributed across the network, posing security and privacy risks. Thus, mobile social multimedia copyright verification and proactive source tracing methods are needed to address these issues. Summary of the Invention

[0004] The purpose of this invention is to provide a method for mobile social multimedia copyright verification and proactive source tracing to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The mobile social multimedia copyright verification and proactive source tracing method includes the following steps:

[0007] 1. Multimedia data is protected using a combination of social fingerprinting, watermarking, and encryption;

[0008] 2: Relying on dynamic fingerprint detection to trace, track, and monitor shared content that causes privacy leaks.

[0009] As a preferred embodiment of the present invention, the specific steps of step 1 are as follows:

[0010] 1.1: The original image is subjected to tree-structured Haar wavelet transform and social fingerprint is extracted. After performing second-order discrete wavelet transform on the 512×512 host image, the high-order plane of the approximate component (LL2) is selected for encryption. Other bit planes and other horizontal and vertical components (LH2, HL2, HL1 and LH1) are used to embed the social fingerprint. The encryption of the approximate component is performed by encrypting each high-order bit plane separately using the chaotic mapping of Sundarapandian. The high-dimensional chaotic mapping of Sundarapandian is as follows.

[0011]

[0012] Where x, y, z, u, w are initial value parameters, while a, b, c are control parameters. When they take specific values, the system will enter a chaotic state. A certain number of chaotic mapping series are generated using this high-dimensional chaotic system, and chaotic encryption is performed on each high-dimensional bit plane.

[0013] 1.2: By leveraging the correspondence between block DCT and tree-structured Haar wavelet transform, the content of the wavelet transform domain is directly transformed to the block DCT domain. Copyright watermark information is embedded in the block DCT domain. Chaotic mapping is used to perform inter-block scrambling encryption on all blocks, and each block is encrypted intra-block using independent chaotic mapping. IDCT (reversible DCT) is then performed to transform the encrypted image with embedded social fingerprints and watermarks to the spatial domain, and then the fingerprint and watermark encrypted image is encrypted in the spatial domain.

[0014] The DCT domain coefficients are encrypted between blocks using a two-dimensional chaotic series generated by the following combined chaotic mapping system for block scrambling.

[0015]

[0016] In this chaotic mapping, x and y are initial value parameters, and g and h are control variable parameters. The encryption of each individual DCT coefficient block is performed using a two-dimensional chaotic series generated by the following two-dimensional combined chaotic mapping system.

[0017]

[0018]

[0019]

[0020]

[0021] In the above four two-dimensional chaotic mapping series, x and y are initial value parameters, and α, β, θ, κ, γ, σ, and p are control parameters. As a preferred embodiment of the present invention, the watermark image in step 1.1 is transformed by DCT, and the resulting matrix undergoes random singular value decomposition to generate a singular value matrix, thus obtaining the watermark embedding matrix template.

[0022] As a preferred embodiment of the present invention, the specific steps for embedding and extracting the image watermark in step 1.1 are as follows:

[0023] S1: Select the DCT coefficients of the host image to resist watermarking attacks, form the coefficient matrix HM, read the 128×128 watermark image WI, perform DCT transformation on WI to obtain WM;

[0024] S2: Use the life game transformation to scramble the watermark image and encrypt the watermark image once. The initial number of transformations is the key used to restore the image watermark during the extraction process.

[0025] S3: Perform random singular value decomposition (RSVD) on the obtained cosine coefficient matrix HM watermark and DCT field coefficient matrix EWM.

[0026]

[0027] S4: The singular value matrix obtained after transforming the host image and the singular value matrix obtained after transforming the image watermark are embedded using the following formula.

[0028] Host_Embed = SHM + ωSWM

[0029] Where ω represents the watermark control strength;

[0030] S5: Perform the reverse operation of the above process on the obtained Host_Embed to obtain the host image with embedded watermark;

[0031] S6: Use encryption and decryption algorithms to protect the host image with embedded watermarks.

[0032] Encrypted_image=Encrypt(WatermarkedImage)

[0033] Decrypted_image=Decrypt(Encrypted_image).

[0034] As a preferred embodiment of the present invention, the embedding and extraction of social fingerprints in step 1.1 uses a deep neural network, and the specific steps are as follows:

[0035] ①: Perform a second-order wavelet transform on the original host image and select the LL2 subband low plane and other horizontal and vertical components (LH2, HL2, HL1 and LH1);

[0036] ②: Convert the user code level code segment into a 0, 1 two-dimensional matrix W_Bit;

[0037] ③: Encode W_Bit using ECC to obtain BitEncode. ECC error detection code can improve the accuracy of fingerprint extraction;

[0038] ④: Using the deep neural networks described above that non-linearly correlate the pixels of the LL2, LH2, HL2, HL1, and LH1 sub-bands with their neighborhoods, the fingerprint bits are embedded into these sub-bands.

[0039] if BitEncode == 1

[0040] LL2'(i,j)=DNN(targetIn)+K1

[0041] else LL2'(i,j)=DNN(targetIn)-K1

[0042] end

[0043] Where i and j range from 2 to N-1, with an increment of 2 each time, targetIn is the neighborhood of the pixel with index i and j as subscripts, which is used as the input parameter of the DNN, and K1 is the embedding strength;

[0044] ⑤: Perform the same operation on other fingerprint levels and embed them into other sub-bands. Then, perform an inverse transformation on the sub-bands containing the embedded fingerprint codewords to obtain the image of the embedded fingerprint.

[0045] ⑥: Perform a second-order wavelet transform on the image with embedded fingerprints to obtain the sub-bands with embedded fingerprints, which are used as the fingerprint extraction region;

[0046] ⑦: Extract the embedded fingerprint using a pre-trained deep neural network.

[0047]

[0048] ⑧: Perform inverse ECC encoding on the obtained bit_extract to obtain the fingerprint bits, acquire the social fingerprint codeword, and track the traitor.

[0049] As a preferred embodiment of the present invention, the specific steps for encrypting the image in step 1.2 are as follows:

[0050] Ⅰ: Read the original image, generate chaotic series RX and RU using the following 5-dimensional chaotic mapping, and use RX and RU to perform row and column scrambling encryption on the image, as well as bit-level scrambling encryption on the highest 5 bit planes.

[0051]

[0052] In the above series of chaotic mapping formulas, x, y, z, u, w are initial value parameters, and other parameters are control variables. image(i,j)=image(RX(i),RU(j)), where i∈(1,M),j∈(1,N);

[0053] II: Combine image(i,j) to restore the image from the first encryption process;

[0054] III: Modify the pixel values ​​of the original image. Perform a bitwise XOR operation between the 8 bit-plane component values ​​of the original image and the elements of the image_key matrix. Use the chaotic mapping described above to generate image_key. The specific diffusion process is as follows:

[0055] in This represents the XOR operation.

[0056] As a preferred embodiment of the present invention, the specific steps of step 2 are as follows:

[0057] 2.1: Identify suspicious communities by utilizing the mobile social networks of the original content sharers;

[0058] 2.2: Construct crawling rules and crawling lists based on the communities and influence of multimedia content sharers; and build a distributed computing crawling system based on the user's community according to the needs of crawling computation tasks.

[0059] 2.3: Use copy detection technology to detect copy errors in suspicious crawled content;

[0060] 2.4: Utilize feature extraction technology to extract feature vectors, store the feature vectors in the database, and build an index structure; then extract feature vectors from the crawled content, perform feature retrieval, and determine whether the queried digital content is a copy version;

[0061] 2.5 Extract fingerprints and perform relevant tests to determine whether the user is an unauthorized user.

[0062] As a preferred embodiment of the present invention, the feature vector extracted in step 2.4 includes:

[0063] Digital Images: Constructing Multiscale Invariant Feature Vectors Resistant to Rotation, Scaling, and Translation Transformations;

[0064] Video data: Taking MPEG video as an example, adjustable multi-scale resolution technology is used to construct a sequence of video frames with different resolutions in the compression domain, and then feature points with multi-scale invariant properties are extracted to generate the spatial feature vector of the video frame.

[0065] As a preferred embodiment of the present invention, the fingerprint encoding of the social fingerprint adopts a block code.

[0066] Compared with the prior art, the beneficial effects of the present invention are:

[0067] This invention employs a multimedia security and privacy protection method that combines social fingerprinting, watermarking, and encryption. The combined application of social fingerprinting and watermarking can not only verify copyright ownership but also proactively trace the source of multiple users simultaneously distributing the same copy, thus deterring malicious distribution. Encryption is performed in the tree-structured Haar wavelet domain, block DCT domain, and spatial domain to meet different levels of security requirements. The source tracing algorithm not only ensures confidentiality during communication but also guarantees copyright verification and proactive source tracing during sharing, thereby achieving triple protection of security and privacy. Attached Figure Description

[0068] Figure 1 This is a schematic diagram illustrating mobile social multimedia content sharing and source tracing in this invention;

[0069] Figure 2 This is a schematic diagram illustrating the secure content sharing method that combines social fingerprinting, watermarking, and encryption in this invention.

[0070] Figure 3 This is a diagram of the proactive source tracing framework for mobile social networks in this invention;

[0071] Figure 4 The middle column (a), (d), and (g) are three original host images (Hostimage1, Hostimage2, and Hostimage3), the middle column (a), (d), and (g) are three host images with embedded watermarks, and the rightmost column (a), (d), and (g) are comparison diagrams of the original host images encrypted again.

[0072] Figure 5 Histograms of the three original images and their encrypted histograms;

[0073] Figure 6 The diagram shows the diagonal, horizontal, and vertical correlation analysis of Hostimage1.

[0074] Figure 7 The plot shows the diagonal, horizontal, and vertical correlation analysis of Hostimage2;

[0075] Figure 8The diagram shows the diagonal, horizontal, and vertical correlation analysis of Hostimage3.

[0076] Figure 9 In the image, (a) and (b) are the watermark and the scrambled watermark, respectively.

[0077] Figure 10 The image watermark image extracted from HostImage2;

[0078] Figure 11 This is a comparison chart of the neural network predictions and actual values ​​in this invention. Detailed Implementation

[0079] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0080] To facilitate understanding of the present invention, a more complete description will be given below with reference to relevant descriptions. Several embodiments of the invention are provided. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0081] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0082] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0083] Please see Figure 1-11 The present invention provides the following technical solution:

[0084] An example of a mobile social multimedia copyright verification and proactive source tracing method includes the following steps:

[0085] 1. Multimedia data is protected using a combination of social fingerprinting, watermarking, and encryption;

[0086] 2: Relying on dynamic fingerprint detection to trace, track, and monitor shared content that causes privacy leaks; the fingerprint encoding of social fingerprints uses block codes.

[0087] The specific steps for step 1 are as follows:

[0088] 1.1: The original image is subjected to tree-structured Haar wavelet transform and social fingerprint is extracted. After performing second-order discrete wavelet transform on the 512×512 host image, the high-order plane of the approximate component (LL2) is selected for encryption. Other bit planes and other horizontal and vertical components (LH2, HL2, HL1 and LH1) are used to embed the social fingerprint. Through the correspondence between block DCT and tree-structured Haar wavelet transform, the content of the above wavelet transform domain is directly transformed to the block DCT domain. Copyright watermark information is embedded in the block DCT domain, and the blocks are scrambled and encrypted. IDCT (reversible DCT) is performed to transform the encrypted image with embedded social fingerprint and watermark to the spatial domain. Then, the fingerprint and watermark encrypted image is encrypted in the spatial domain. Discrete wavelet transform (DWT) divides the original image into four parts: approximate components, horizontal components, vertical components, and diagonal components. The latter three components are high-frequency detail sub-components of the image, while the approximate components are very similar to the original image. DWT is formed by low-pass filtering in both row and column directions. The components can be further decomposed. To achieve scalability, a non-bisecting tree-structured Haar wavelet transform is chosen. This TSH wavelet transform can adapt to the user's multi-level social fingerprint code. Based on the hierarchical structure of the fingerprint code, corresponding discrete point vectors are determined to assist in the tree-structured Haar wavelet transform. A mapping relationship is established between the carrier and the fingerprint code. This mapping relationship allows each segment of the multi-level fingerprint code to be embedded in parallel into the corresponding coefficients in the TSH domain. The community code segment avoids repeated embedding, which can effectively reduce the overhead of repeatedly embedding the entire fingerprint codeword on the server side.

[0089] Image pixels exhibit strong correlations with their neighbors, such as horizontal, vertical, and diagonal correlations. Deep neural networks (DNNs) possess excellent non-linear correlation approximation capabilities, making them suitable for watermark embedding and extraction. Here, we select eight pixels in the target pixel's neighborhood as input and the middle pixel as output as a training set. A deep neural network is chosen, and after extensive testing, the number of neurons in each layer is calculated. The transfer function between the first two layers uses the tangent transfer function, and the transfer function between the last two layers uses the linear transfer function. The learning function of the deep neural network uses momentum gradient reduction for weights and thresholds. Since the deep neural network provides optimal prediction results when the input data is between 0 and 1, the input and output training sets are processed...

[0090] M(i,j)=(M(i,j)-min) / (max-min),

[0091] Where M(i,j) is the training set, max is the maximum value of the selection set, and min is the minimum value of the selection set. Figure 5 The results of a trained deep neural network predicting the true values ​​of 300 pixel neighborhoods are shown, where the scatter plots represent the true values ​​and the reflected values ​​represent the predicted values. This is achieved by selecting the best prediction from multiple deep neural networks trained on the same training set, which consists of 100 true pixel values ​​and their neighborhood pixel values. The prediction results show a very close match to the true values ​​of the middle pixels. This deep neural network was chosen in the experiment to optimize the embedding of social fingerprints. Encryption of the approximate components is performed individually on each high-dimensional bit plane using the Sundarapandian chaotic map. The Sundarapandian high-dimensional chaotic map is shown below.

[0092]

[0093] Where x, y, z, u, w are initial value parameters, while a, b, c are control parameters. When they take specific values, the system will enter a chaotic state. A certain number of chaotic mapping series are generated using this high-dimensional chaotic system, and chaotic encryption is performed on each high-dimensional bit plane.

[0094] 1.2: By leveraging the correspondence between block DCT and tree-structured Haar wavelet transform, the wavelet transform domain content is directly transformed to the block DCT domain. Copyright watermark information is embedded in the block DCT domain. Chaotic mapping is used to scramble and encrypt all blocks, and each block is encrypted separately using an independent chaotic mapping. IDCT (invertible DCT) is then performed to transform the encrypted image with embedded social fingerprints and watermarks to the spatial domain. The fingerprint and watermark encrypted image is then encrypted in the spatial domain. After the watermark image in step 1.1 is transformed by DCT, the resulting matrix undergoes random singular value decomposition, and a singular value matrix is ​​generated to obtain the watermark embedding matrix template.

[0095] The DCT domain coefficients are encrypted between blocks using a two-dimensional chaotic series generated by the following combined chaotic mapping system for block scrambling.

[0096]

[0097] In this chaotic mapping, x and y are initial value parameters, and g and h are control variable parameters. The encryption of each individual DCT coefficient block is performed using a two-dimensional chaotic series generated by the following two-dimensional combined chaotic mapping system.

[0098]

[0099]

[0100]

[0101]

[0102] In the above four series of two-dimensional chaotic mappings, x and y are initial parameters, and α, β, θ, κ, γ, σ, and p are control parameters.

[0103] As a preferred embodiment of the present invention, the specific steps for embedding and extracting the image watermark in step 1.1 are as follows:

[0104] S1: Select the DCT coefficients of the host image to resist watermarking attacks, form the coefficient matrix HM, read the 128×128 watermark image WI, perform DCT transformation on WI to obtain WM;

[0105] S2: Use the life game transformation to scramble the watermark image and encrypt the watermark image once. The initial number of transformations is the key used to restore the image watermark during the extraction process.

[0106] S3: Perform random singular value decomposition (RSVD) on the obtained cosine coefficient matrix HM and the watermarked DCT domain coefficient matrix EWM.

[0107] [UHM,SHM,VHM]=RSVD(HM)

[0108] [UWM,SWM,VWM]=RSVD(EWM)

[0109] S4: The singular value matrix obtained after transforming the host image and the singular value matrix obtained after transforming the image watermark are embedded using the following formula.

[0110] Host_Embed = SHM + ωSWM;

[0111] Where ω represents the watermark control strength;

[0112] S5: Perform the reverse operation of the above process on Host_Embed to obtain the host image with embedded watermark;

[0113] S6: Use encryption and decryption algorithms to protect the host image with embedded watermarks.

[0114] Encrypted_image=Encrypt(WatermarkedImage)

[0115] Decrypted_image=Decrypt(Encrypted_image).

[0116] Please refer to Figure 1 , 2 In step 1.1, the embedding and extraction of social fingerprints utilizes a deep neural network, and the specific steps are as follows:

[0117] ①: Perform a second-order wavelet transform on the original host image and select the LL2 subband low plane and other horizontal and vertical components (LH2, HL2, HL1 and LH1);

[0118] ②: Convert the user code level code segment into a 0, 1 two-dimensional matrix W_Bit;

[0119] ③: Encode W_Bit using ECC to obtain BitEncode. ECC error detection code can improve the accuracy of fingerprint extraction;

[0120] ④: Using the deep neural networks described above that non-linearly correlate the pixels of the LL2, LH2, HL2, HL1, and LH1 sub-bands with their neighborhoods, the fingerprint bits are embedded into these sub-bands.

[0121] if BitEncode == 1

[0122] LL2'(i,j)=DNN(targetIn)+K1

[0123] else LL2'(i,j)=DNN(targetIn)-K1

[0124] end

[0125] Where i and j range from 2 to N-1, with each increment being 2, targetIn is the neighborhood of the pixel with indices i and j, which serves as the input parameter of the DNN, and K1 is the embedding strength;

[0126] ⑤: Perform the same operation on other fingerprint levels and embed them into other sub-bands. Then, perform an inverse transformation on the sub-bands containing the embedded fingerprint codewords to obtain the image of the embedded fingerprint.

[0127] ⑥: Perform a second-order wavelet transform on the image with embedded fingerprints to obtain the sub-bands with embedded fingerprints, which are used as the fingerprint extraction region;

[0128] ⑦: Extract the embedded fingerprint using a pre-trained deep neural network.

[0129] f DNN(targetIn)<W_LL2(i,j)

[0130] bit_extract(i,j) = 1;

[0131] else bit_extract(i,j) = 0;

[0132] end

[0133] ⑧: Perform inverse ECC encoding on bit_extract to obtain fingerprint bits, acquire social fingerprint codewords, and track traitors.

[0134] The specific steps for encrypting the image in step 1.2 are as follows:

[0135] Ⅰ: Read the original image, generate chaotic series RX and RU using the following 5-dimensional chaotic mapping, and use RX and RU to perform row and column scrambling encryption on the image, as well as bit-level scrambling encryption on the highest 5 bit planes.

[0136]

[0137] In the above series of chaotic mapping formulas, x, y, z, u, w are initial value parameters, and other parameters are control variables. image(i,j)=image(RX(i),RU(j)), where i∈(1,M),j∈(1,N);

[0138] II: Combine image(i,j) to restore the image from the first encryption process;

[0139] III: Modify the pixel values ​​of the original image. Perform a bitwise XOR operation between the 8 bit-plane component values ​​of the original image and the elements of the image_key matrix. Use the chaotic mapping described above to generate image_key. The specific diffusion process is as follows:

[0140] in This represents the XOR operation.

[0141] The specific steps of step 2 in claim 1 are as follows:

[0142] 2.1: Identify suspicious communities by utilizing the mobile social networks of the original content sharers;

[0143] 2.2: Construct crawling rules and crawling lists based on the communities and influence of multimedia content sharers; and build a distributed computing crawling system based on the user's community according to the needs of crawling computation tasks.

[0144] 2.3: Use copy detection technology to detect copy errors in suspicious crawled content;

[0145] 2.4: Utilize feature extraction technology to extract feature vectors, store the feature vectors in the database, and build an index structure; then extract feature vectors from the crawled content, perform feature retrieval, and determine whether the queried digital content is a copy version;

[0146] 2.5 Extract fingerprints and perform relevant tests to determine whether the user is an unauthorized user.

[0147] The feature vectors extracted in step 2.4 include:

[0148] Digital Images: Constructing Multiscale Invariant Feature Vectors Resistant to Rotation, Scaling, and Translation Transformations;

[0149] Video data: Taking MPEG video as an example, an adjustable multi-scale resolution technique is employed to construct frame sequences of different resolutions within the compressed domain. Feature points with multi-scale invariant properties are then extracted to generate spatial feature vectors for the video frames. The extracted temporal feature points of consecutive video frames are calculated to form temporal feature vectors. These feature vectors are then combined with the spatial feature vectors to form the feature vectors of the video data. Therefore, the proposed copy detection method exhibits good robustness and identification capability.

[0150] Active source tracing relies on dynamic fingerprint detection, and the detection process is as follows: The first stage is to detect suspicious communities. First, the fingerprint information to be detected is divided into sequences of the same length as the user code. The corresponding community information is separated from each sequence and arranged in order to form a group information sequence to be detected. Then, the community code in the code table is spread spectrum modulated and expanded into a matching sequence of the same length as the group information sequence. The group information sequence to be detected is correlated with the matching sequence modulated by the community code. If the correlation value is greater than the threshold, it is a suspicious community. Then, the second stage of detection begins. Within the identified suspicious groups, the conspirators are tracked. Here, based on the positions of the predefined correction codewords, the correction codewords are first detected. All codewords in the user group whose corresponding positions match these correction codewords are grouped into a suspicious codeword set. Then, within this set, correlation detection is used to track the conspirators, thereby improving detection efficiency. For suspicious communities on mobile social networks, an active search mechanism based on copy detection technology is adopted to actively search for illegal copies in the mobile community, then extract fingerprint information and send it to the content monitoring server, which then determines the illegal user. This active search mechanism mainly utilizes copy detection technology and copy crawling technology of the network community where the user is located.

[0151] Artificial neurons are the most basic structure of artificial neural networks. Unlike biological neurons, artificial neurons are less complex; they allow for multiple input parameters and one output parameter, and are nonlinear components. Their structure is as follows:

[0152]

[0153] Out = f(C);

[0154] Where In is the input parameter, w represents the weight, which controls the connection strength between neurons, and θ is the allowable error range;

[0155] Neural networks possess exceptionally strong learning capabilities. They exhibit powerful approximation abilities for nonlinear correlations, handling complex nonlinear relationships through simple combinations and mappings between neurons. Neural networks employ the least squares algorithm. This algorithm utilizes gradient search, minimizing the mean square error between the predicted and actual values. Information is input from the input layer to the hidden layers and then to the output layer. If the output does not yield the desired result, the error signal is returned to the previous layer, the neuron weights are adjusted, and the output is repeated.

[0156] δ k =O k (1-O k )(d k -O k )

[0157] Δw jk =ηO k (1-O k )(d k -O k )O j

[0158]

[0159] wi j (t+1)=wi j (t)+ηδ j Oi+a[wi j (t)+wi j (t-1)]

[0160] The above formula is for adjusting the weights between the output layer and hidden layers in a neural network. A larger value of η results in faster learning. The coefficient 'a' is around 0.9. The number of nodes in the hidden layer of the neural network is calculated using the following formula, derived from the number of nodes in the other two layers:

[0161] n = (OM + rM + s) 1 / 2 M is the number of neurons in the input layer, O is the number of neurons in the output layer, and r and s are fixed parameters. We take r = 1.6799 and s = 0.9298. Assuming that there are 8 neurons in the input layer and only 1 neuron in the output layer, then n ≈ 4.8. We take the value of n as 5.

[0162] Chaotic mapping is a dynamic behavior of a nonlinear system, and the trajectory of a chaotic system is highly sensitive to changes in initial values. Therefore, multimedia encrypted using chaotic mapping is unlikely to be extracted by attackers. Chaotic systems can generate a series of random numbers. Chaotic systems possess characteristics such as sensitivity to initial values, nonlinearity, randomness, and long-term unpredictability.

[0163] Hyperchaotic functions possess a very large key space and very high security. The following 5-dimensional chaotic mapping generates chaotic series RX and RU. RX and RU are then used to perform row and column scrambling encryption on the image, as well as bit-level scrambling encryption on up to the 5th bit plane.

[0164]

[0165] In the above series of chaotic mapping formulas, x, y, z, u, w are initial value parameters, and other parameters are control variables. image(i,j)=image(RX(i),RU(j)), where i∈(1,M),j∈(1,N);

[0166] Combine image(i,j) to restore the image from the first encryption process;

[0167] Modify the pixel values ​​of the original image by performing a bitwise XOR operation between the 8 bit-plane component values ​​of the original image and the elements of the image_key matrix. Then, generate the image_key using the aforementioned chaotic mapping. The specific diffusion process is as follows:

[0168] in This represents the XOR operation.

[0169] Figure 4 (a), (d), and (g) respectively show the original host images HostImage1, HostImage2, and HostImage3, as well as their corresponding watermarked images and encrypted images. It can be seen that the host images are extremely similar to the watermarked images, indicating that the watermark of this algorithm is relatively imperceptible. The encrypted images are chaotic and disordered.

[0170] Figure 5 The histograms of the R components of HostImage1, HostImage2, and HostImage3 before and after encryption are shown respectively. (a), (b), and (c) are the original histograms of HostImage1, HostImage2, and HostImage3, while (d), (e), and (f) are the histograms of HostImage1, HostImage2, and HostImage3 after encryption. It can be seen that the histogram of the encrypted image becomes very smooth and balanced.

[0171] Figure 6 , Figure 7 and Figure 8The results of diagonal, horizontal, and vertical correlation analyses of 1000 randomly selected neighboring pixels in the host images HostImage1, HostImage2, and HostImage3 before and after encryption are displayed respectively. Figure 6 In the image, (a), (b), and (c) are scatter plots showing the diagonal, horizontal, and vertical correlations of the R component of HostImage1, respectively. (d), (e), and (f) represent the scatter plots showing the diagonal, horizontal, and vertical correlations of the R component of HostImage1 after encryption. Figure 7 The correlation analysis results for the R component of HostImage2 before and after encryption are shown. Figure 8 The results show the correlation analysis of the R component of HostImage3 before and after encryption. The original correlation between pixels in the encrypted image is uniform, and the original correlation between pixels is destroyed.

[0172] Figure 9 The image displays the watermarked image after transformation and scrambling, where the correlation between pixels has been removed. Figure 10 The table shows the watermarked image extracted without any attacks. It can be seen that the image watermarking algorithm has some distortion. Table 1 shows the peak signal-to-noise ratio (PSNR) of the host image and the image with embedded watermark, as well as the entropy value of the encrypted image, the normalized correlation (NC) value of the extracted image watermark, and the bit error ratio (BER). The data shows that the entropy value of the encrypted image reaches 7.9998, indicating very good encryption. The NC values ​​of the extracted image watermark are all above 0.99, indicating minimal distortion. The BER of the social fingerprint extracted using a deep neural network and error detection code is 0, indicating that this social fingerprinting technology is lossless.

[0173] Extracted watermark features

[0174]

[0175] Table 1

[0176] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for mobile social multimedia copyright verification and proactive source tracing, characterized in that, Includes the following steps:

1. Multimedia data is protected using a combination of social fingerprinting, watermarking, and encryption; 2: Relying on dynamic fingerprint detection to trace, track, and monitor shared content that causes privacy leaks.

2. The mobile social multimedia copyright verification and proactive source tracing method according to claim 1, characterized in that: The specific steps of step 1 are as follows: 1.1: The original image is subjected to tree-structured Haar wavelet transform and social fingerprint is extracted. After performing second-order discrete wavelet transform on the 512×512 host image, the high-bit plane of the approximate component (LL2) is selected for encryption, and other bit planes and other horizontal and vertical components (LH2, HL2, HL1 and LH1) are used to embed social fingerprint. 1.2: By leveraging the correspondence between block DCT and tree-structured Haar wavelet transform, the content of the wavelet transform domain is directly transformed to the block DCT domain. Copyright watermark information is embedded in the block DCT domain. Chaotic mapping is used to perform inter-block scrambling encryption on all blocks, and each block is encrypted intra-block using independent chaotic mapping. IDCT (reversible DCT) is then performed to transform the encrypted image with embedded social fingerprints and watermarks to the spatial domain, and then the fingerprint and watermark encrypted image is encrypted in the spatial domain.

3. The mobile social multimedia copyright verification and proactive source tracing method according to claim 2, characterized in that: In step 1.1, the watermark image is transformed by DCT, and the resulting matrix is ​​subjected to random singular value decomposition to generate a singular value matrix, which is then used to obtain the watermark embedding matrix template.

4. The mobile social multimedia copyright verification and proactive source tracing method according to claim 2, characterized in that: The specific steps for embedding and extracting the image watermark in step 1.1 are as follows: S1: Select the DCT coefficients of the host image to resist watermarking attacks, form the coefficient matrix HM, read the 128×128 watermark image WI, perform DCT transformation on WI to obtain WM; S2: Use the life game transformation to scramble the watermark image and encrypt the watermark image once. The initial number of transformations is the key used to restore the image watermark during the extraction process. S3: Perform random singular value decomposition (RSVD) on the obtained cosine coefficient matrix HM, watermark DCT, and field coefficient matrix EWM. S4: The singular value matrix obtained after transforming the host image and the singular value matrix obtained after transforming the image watermark are embedded using the following formula. Host_Embed = SHM + ωSWM; Where ω represents the watermark control strength; S5: Perform the reverse operation of the above process on Host_Embed to obtain the host image with embedded watermark; S6: Use encryption and decryption algorithms to protect the host image with embedded watermarks. Encrypted_image=Encrypt(WatermarkedImage) Decrypted_image=Decrypt(Encrypted_image).

5. The mobile social multimedia copyright verification and proactive source tracing method according to claim 2, characterized in that: In step 1.1, the embedding and extraction of social fingerprints utilizes a deep neural network, and the specific steps are as follows: ①: Perform a second-order wavelet transform on the original host image and select the LL2 subband low plane and other horizontal and vertical components (LH2, HL2, HL1 and LH1); Convert the user code level code segment into a 0, 1 two-dimensional matrix W_Bit; ③: Encode W_Bit using ECC to obtain BitEncode. ECC error detection code can improve the accuracy of fingerprint extraction; ④: Using the deep neural networks described above that non-linearly correlate the pixels of the LL2, LH2, HL2, HL1, and LH1 sub-bands with their neighborhoods, the fingerprint bits are embedded into these sub-bands. if BitEncode == 1 LL2'(i,j)=DNN(targetIn)+K1 else LL2'(i,j)=DNN(targetIn)-K1 end Where i and j range from 2 to N-1, with an increment of 2 each time, targetIn is the neighborhood of the pixel with index i and j as subscripts, which is used as the input parameter of the DNN, and K1 is the embedding strength; ⑤: Perform the same operation on other fingerprint levels and embed them into other sub-bands. Then, perform an inverse transformation on the sub-bands containing the embedded fingerprint codewords to obtain the image of the embedded fingerprint. ⑥: Perform a second-order wavelet transform on the image with embedded fingerprints to obtain the sub-bands with embedded fingerprints, which are used as the fingerprint extraction region; ⑦: Extract the embedded fingerprint using a pre-trained deep neural network. ⑧: Perform inverse ECC encoding on bit_extract to obtain fingerprint bits, acquire social fingerprint codewords, and track traitors.

6. The mobile social multimedia copyright verification and proactive source tracing method according to claim 2, characterized in that: The specific steps for encrypting the image in step 1.2 are as follows: Ⅰ: Read the original image, generate chaotic series RX and RU using the following 5-dimensional chaotic mapping, and use RX and RU to perform row and column scrambling encryption and bit-level scrambling encryption of the highest 5 bit planes, respectively. In the above series of chaotic mapping formulas, x, y, z, u, w are initial value parameters, and other parameters are control variables. image(i,j)=image(RX(i),RU(j)), where i∈(1,M),j∈(1,N); II: Combine image(i,j) to restore the image from the first encryption process; III: Modify the pixel values ​​of the original image. Perform a bitwise XOR operation between the 8 bit-plane component values ​​of the original image and the elements of the image_key matrix. Use the chaotic mapping described above to generate image_key. The specific diffusion process is as follows: in This represents the XOR operation.

7. The mobile social multimedia copyright verification and proactive source tracing method according to claim 1, characterized in that: The specific steps of step 2 are as follows: 2.1: Identify suspicious communities by utilizing the mobile social networks of the original content sharers; 2.2: Construct crawling rules and crawling lists based on the communities and influence of multimedia content sharers; and build a distributed computing crawling system based on the user's community according to the needs of crawling computation tasks. 2.3: Use copy detection technology to detect copy errors in suspicious crawled content; 2.4: Utilize feature extraction technology to extract feature vectors, store the feature vectors in the database, and build an index structure; then extract feature vectors from the crawled content, perform feature retrieval, and determine whether the queried digital content is a copy version; 2.5 Extract fingerprints and perform relevant tests to determine whether the user is an unauthorized user.

8. The mobile social multimedia copyright verification and proactive source tracing method according to claim 7, characterized in that: The feature vectors extracted in step 2.4 include: Digital Images: Constructing Multiscale Invariant Feature Vectors Resistant to Rotation, Scaling, and Translation Transformations; Video data: Taking MPEG video as an example, adjustable multi-scale resolution technology is used to construct a sequence of video frames with different resolutions in the compression domain, and then feature points with multi-scale invariant properties are extracted to generate the spatial feature vector of the video frame.

9. The mobile social multimedia copyright verification and proactive source tracing method according to any one of claims 1-8, characterized in that: The fingerprint encoding of the social fingerprint uses a block code.

Images