A copyright tracking method, system, and program product for unlicensed video distribution

Abstract

The embodiment of the present application provides a copyright tracking method and system for unlicensed video distribution, which relates to the field of digital copyright protection technology. The method includes: performing the following operations based on the core node of the CDN system: generating corresponding encrypted digital watermarks for each legitimate video in the legitimate database, and embedding the encrypted digital watermarks into the pixel values ​​of the target redundant pixel area; training the video matching model and deploying it to each edge node respectively; performing the following operations based on each edge node: calling the video matching model to determine the target legitimate video that matches the video to be distributed; decrypting and verifying the encrypted digital watermark; generating a real-time distribution record based on the device information of the edge node and the current timestamp of the system, and using the real-time distribution record to update the encrypted digital watermark, and performing distribution processing based on the video to be distributed with the updated encrypted digital watermark. In this way, the reliability, accuracy and real-time performance of video copyright protection can be significantly improved.

Description

Technical Field

[0001] The present application relates to the field of digital copyright protection, and in particular to a copyright tracking method, system, and program product for unlicensed video distribution. Background Art

[0002] In today's digital and networked world, the production and distribution of video content has become extremely convenient, but this has also led to a large number of unauthorized video distribution phenomena. These unauthorized video distributions not only infringe on the rights of copyright owners, but also seriously affect the healthy development of the digital content industry.

[0003] At present, the commonly used digital copyright protection technology for video content is digital watermarking technology, which embeds invisible information in the video content to achieve video identification and tracking. Watermark information is usually embedded in various parts of the video in an imperceptible way, such as frames, frequencies, or time domain signals. However, although the watermark information is invisible, it is still vulnerable to various attacks, such as noise attacks, geometric attacks (rotation, scaling, cropping) and format conversion. These attacks can significantly weaken the detection effect of the watermark, or even make it completely ineffective, making it unable to fully meet the reliability requirements of video copyright protection.

[0004] In response to the above problems, the industry has not yet proposed a better technical solution. Summary of the invention

[0005] The present application provides a copyright tracking method, system, storage medium, computer program product and electronic device for unlicensed video distribution, which are used to at least solve the problem of insufficient reliability of video copyright protection caused by embedding digital watermarks only in video content in the current related technology.

[0006] In the first aspect, the present application embodiment provides a copyright tracking method for unlicensed video distribution, which is applied to CDN (Content Delivery Network). The invention discloses a content distribution network (CDN) system, wherein the CDN system comprises a core node and a plurality of edge nodes. The method comprises: the core node is used to perform the following operations: for each legitimate video in a legitimate database, an identifier corresponding to the legitimate video is generated, and a low-level video feature group and a video semantic feature group corresponding to the legitimate video are extracted; the low-level video feature group comprises: inter-frame difference, color histogram and texture pattern; the video semantic feature group comprises: video object information, video scene information and video event information; for each first video frame in the legitimate video, a color histogram difference between the first video frame and an adjacent second video frame is calculated; if the color histogram difference exceeds a preset threshold, the first video frame and / or the second video frame is determined as a key frame, at least one redundant pixel region in each of the key frames is identified, and a target redundant pixel region is randomly selected from each of the redundant pixel regions; the redundant pixel region comprises a flat region, a region with complex texture and an edge region; an encrypted digital watermark is generated according to the identifier and the copyright information of the legitimate video, and the encrypted digital watermark is embedded in the pixel value of the target redundant pixel region; based on the low-level video features corresponding to each legitimate video, the encrypted digital watermark is embedded in the pixel value of the target redundant pixel region. The method comprises the steps of: determining a low-level video feature group and a video semantic feature group corresponding to the video to be distributed when the edge node detects the video to be distributed, and calling the video matching model to determine a target legitimate video matching the video to be distributed; screening at least one key frame to be detected according to the color histogram of each video frame in the video to be distributed, and extracting at least one redundant pixel area to be detected in each key frame to be detected; identifying whether there is an encrypted digital watermark in each redundant pixel area to be detected; if so, obtaining the decrypted data corresponding to the encrypted digital watermark; matching and verifying the decrypted data according to the identifier and copyright information corresponding to the target legitimate video; if there is no match, determining that the video to be distributed is an unlicensed video, and refusing to distribute the video to be distributed; and if there is a match, generating a real-time distribution record according to the device information of the edge node and the current timestamp of the system, and updating the encrypted digital watermark by using the real-time distribution record, and performing distribution processing based on the video to be distributed with the updated encrypted digital watermark.

[0007] In a second aspect, an embodiment of the present application provides a copyright tracking system for unlicensed video distribution, which is applied to a CDN system, wherein the CDN system includes a core node and multiple edge nodes, and the system includes a first device deployed at the core node and a second device deployed at each edge node respectively: the first device includes: a watermark embedding unit, which is used to generate an identifier corresponding to each genuine video in a genuine database, and extract a low-level video feature group and a video semantic feature group corresponding to the genuine video; the low-level video feature group includes: frame difference, color histogram and texture pattern; the video semantic feature group includes: video object information, video scene information and video event information information; for each first video frame in the legitimate video, calculating the color histogram difference between the first video frame and the adjacent second video frame; if the color histogram difference exceeds a preset threshold, determining the first video frame and / or the second video frame as a key frame, identifying at least one redundant pixel region in each of the key frames, and randomly selecting a target redundant pixel region from each of the redundant pixel regions; the redundant pixel region includes a flat region, a region with complex textures, and an edge region; generating an encrypted digital watermark according to the identifier and the copyright information of the legitimate video, and embedding the encrypted digital watermark into the pixel value of the target redundant pixel region; a model training unit for The video matching model is trained by using the corresponding low-level video feature group and video semantic feature group; a model deployment unit is used to deploy each of the edge nodes based on the video matching model; the second device comprises: a video matching unit is used to determine the low-level video feature group and video semantic feature group corresponding to the video to be distributed when the edge node detects the video to be distributed, and call the video matching model to determine the target genuine video matching the video to be distributed; a frame area extraction unit is used to screen at least one key frame to be detected according to the color histogram of each video frame in the video to be distributed, and extract at least one redundant pixel area to be detected in each of the key frames to be detected ; a watermark identification unit, used to identify whether there is an encrypted digital watermark in each of the redundant pixel areas to be detected; a decryption verification unit, used to obtain the decrypted data corresponding to the encrypted digital watermark if it exists; the decrypted data is matched and verified according to the identifier and copyright information corresponding to the target genuine video; if there is no match, it is determined that the video to be distributed is an unlicensed video, and the distribution of the video to be distributed is refused; a watermark update unit, used to generate a real-time distribution record according to the device information of the edge node and the current timestamp of the system if there is a match, and use the real-time distribution record to update the encrypted digital watermark, and perform distribution processing based on the video to be distributed with the updated encrypted digital watermark.

[0008] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the copyright tracking method for unlicensed video distribution of any embodiment of the present application.

[0009] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the copyright tracking method for unauthorized video distribution of any embodiment of the present application are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the copyright tracking method for unauthorized video distribution of any embodiment of the present application.

[0011] The copyright tracking method for unlicensed video distribution provided by this application can produce at least the following technical effects:

[0012] (1) By embedding encrypted digital watermarks in the video, the concealment and anti-attack capabilities of the watermarks are significantly enhanced. In particular, by embedding the encrypted digital watermarks in the flat areas, complex texture areas, and edge areas of the redundant pixel areas, the watermarks can still maintain high robustness when subjected to noise attacks, geometric attacks (such as rotation, scaling, cropping), and format conversion operations, effectively improving the reliability and anti-attack capabilities of copyright protection, and ensuring the continuity and reliability of video copyright protection.

[0013] (2) A video matching model is constructed by generating and utilizing low-level video feature groups (such as inter-frame differences, color histograms, and texture patterns) and video semantic feature groups (such as video object information, video scene information, and video event information). In this way, the video matching model can accurately identify the content association between the video to be distributed and the legitimate video, improve the accuracy of video recognition and tracking, and provide a basis for edge nodes to quickly and accurately detect unauthorized video distribution behaviors, avoiding the occurrence of false positives and false negatives.

[0014] (3) When the edge node detects an unlicensed video, it can immediately reject its distribution request, reducing the spread of pirated videos from the source, not only protecting the copyright of genuine videos, but also improving the distribution efficiency of the CDN system. In addition, the edge node dynamically updates the encrypted digital watermark based on the real-time distribution record generated by the system's current timestamp and device information, so that the historical distribution of the video can be analyzed from the video material itself, optimizing the distribution efficiency of the CDN system while also realizing the copyright tracking function of the video.

[0015] (4) By detecting the encrypted digital watermark in the video to be distributed and performing matching verification based on the decrypted data, the copyright information of the video can be accurately traced. If the match fails, the video is determined to be an unlicensed video and its distribution is blocked; if the match succeeds, the encrypted digital watermark is dynamically updated to record detailed information during each real-time distribution process, thereby enhancing the traceability of video copyright information, ensuring the traceability of each distribution behavior, effectively protecting the rights and interests of the copyright owner, and providing strong evidence for subsequent copyright disputes, adapting to the rapidly changing needs in the digital and networked environment, and enhancing the flexibility and response speed of the system.

[0016] Through this technical solution, the enhanced encrypted digital watermark technology and the improved video matching model are comprehensively utilized in the CND system, which significantly improves the reliability, accuracy and real-time performance of video copyright protection, effectively addresses the problem of unlicensed video distribution existing in current related technologies, and promotes the healthy development of the digital content industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of the architecture of an example of a CDN system according to an embodiment of the present application is shown;

[0019] Figure 2 A flowchart showing an example of a copyright tracking method for unlicensed video distribution according to an embodiment of the present application is shown;

[0020] Figure 3 A flowchart showing an example of distributing a video to be distributed according to an embodiment of the present application is shown;

[0021] Figure 4 A schematic diagram showing a structural connection of an example of a video matching model according to an embodiment of the present application is shown;

[0022] Figure 5 A structural block diagram of an example of a copyright tracking system for unlicensed video distribution according to an embodiment of the present application is shown;

[0023] Figure 6 It is a schematic structural diagram of an embodiment of an electronic device of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0026] Figure 1 A schematic diagram of the architecture of an example of a CDN system according to an embodiment of the present application is shown.

[0027] like Figure 1 As shown, the CDN system 100 includes a core node 110 and multiple edge nodes (121, 122...12a). The core node 110 is responsible for storing and managing all original video content, and processing the initial distribution of content. The core node 110 is usually deployed in a data center and has powerful storage and processing capabilities.

[0028] In the embodiment of the present application, the core node 110 mainly contributes in the following two aspects: on the one hand, the enhanced digital watermark technology is used to encrypt and embed watermarks in each original legitimate video, and then the legitimate video is distributed through each edge node for user access and use. On the other hand, the video matching model is trained and constructed through the core node 110, and the video matching model is synchronously deployed to each edge node, so that the edge node can verify the authorization of the video based on the video matching model before distributing the video.

[0029] Each edge node (e.g., 121, 122, ... 12a) is distributed in different geographical locations, close to end users. Edge nodes obtain video content from core nodes and cache it. When a user requests a video, the CDN system routes the request to the closest and least loaded edge node based on the user's geographical location, allowing users to load and play videos faster, reducing latency and buffering time. In addition, edge nodes can perform dynamic content updates and cache refreshes to ensure that users obtain the latest video content, which not only improves the efficiency of video transmission, but also reduces the pressure on core nodes and improves the reliability and stability of the overall system.

[0030] In the embodiment of the present application, each edge node also has the following contributions: on the one hand, it detects whether the video to be distributed is a legitimate video, and more preferably, it can also detect whether the distribution record of the legitimate video is compliant, and refuses to distribute unlicensed videos. On the other hand, before distributing the video to be distributed, a real-time distribution record is generated and updated and recorded in the digital watermark to ensure the traceability of each distribution behavior, effectively protecting the rights and interests of the video copyright owner.

[0031] Figure 2 A flowchart of an example of a copyright tracking method for unlicensed video distribution according to an embodiment of the present application is shown.

[0032] like Figure 2 As shown, the processing flow involves the collaborative processing of the core node and the edge node. According to the operation process processed by the core node, in step S211, for each legitimate video in the legitimate database, an identifier corresponding to the legitimate video is generated, and the low-level video feature group and video semantic feature group corresponding to the legitimate video are extracted.

[0033] Here, the low-level video feature group includes: inter-frame differences, color histograms, and texture patterns. In some embodiments, a pre-trained deep learning model can be used to extract high-level features from key frames, such as inter-frame differences, color histograms, texture patterns, motion vectors, etc. The video semantic feature group includes: video object information, video scene information, and video event information. In some embodiments, pre-trained convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are used to extract semantic features from video content, such as object recognition, action recognition, scene understanding, event detection, etc.

[0034] It should be noted that subtle changes in the video, such as color changes and texture patterns, can be accurately captured and represented at the feature level through low-level video feature extraction. Therefore, by extracting low-level video feature groups such as inter-frame differences, color histograms, and texture patterns, as well as video semantic feature groups such as video object information, video scene information, and video event information, the model is able to comprehensively capture the multi-dimensional features of the video content. In addition, video semantic features (such as objects, scenes, and events) enable the model to understand the context of the video content, further improving the accuracy of recognition. Therefore, the accuracy and robustness of video content recognition are improved through the comprehensive representation of multi-dimensional features, allowing the model to more accurately match and identify genuine videos.

[0035] In step S212, for each first video frame in the legitimate video, a color histogram difference between the first video frame and an adjacent second video frame is calculated.

[0036] In step S213, if the color histogram difference exceeds a preset threshold, the first video frame and / or the second video frame is determined as a key frame, at least one redundant pixel region in each key frame is identified, and a target redundant pixel region is randomly selected from each redundant pixel region.

[0037] More specifically, the color histogram difference can be calculated by the following formula:

[0038]

[0039] In the formula, H t (i) represents the i-th color histogram value of the t-th frame, if ΔH t,t+1 >color histogram threshold, the tth frame and / or the t+1th frame are determined as key frames.

[0040] Therefore, by calculating the color histogram difference, frames with large scene changes in the video are selected as key frames to ensure that the watermark is embedded in the frames with large changes in video content, so that digital watermarks can be added to frames that are more visually representative.

[0041] Here, the redundant pixel areas where the digital watermark is implanted include flat areas, complex texture areas, and edge areas. Specifically, embedding the watermark in the flat areas and complex texture areas of the video does not affect the visual quality of the video, and ensures that the watermark still exists after compression and cropping. Therefore, by identifying the redundant pixel areas (including flat areas, complex texture areas, and edge areas) in the key frame and embedding the encrypted digital watermark, the concealment and robustness of the watermark are improved.

[0042] In step S214, an encrypted digital watermark is generated according to the identifier and the copyright information of the legitimate video, and the encrypted digital watermark is embedded into the pixel value of the target redundant pixel area.

[0043] In some implementations, an advanced encryption algorithm (e.g., AES) is used to encrypt the watermark information to prevent the watermark information from being directly read and tampered with by an attacker. Therefore, by using an encrypted digital watermark, the security of the watermark information can be more effectively guaranteed compared to an ordinary watermark, and the digital watermark can be prevented from being tampered with or deleted.

[0044] More specifically, the watermark information can be embedded in the least significant bit of the pixel using the least significant bit (LSB) technique, ensuring that the embedding process is simple and does not affect the video quality. Thus, through pixel-level implantation, it is possible to ensure that unauthorized videos can still be accurately detected and tracked under various video processing operations (such as compression, editing, rotation, etc.).

[0045] In order to further enhance the concealment and robustness of the embedded watermark, a watermark embedding method with adaptively adjusted embedding strength is also proposed in the embodiments of the present application. In some embodiments, an embedding strength that matches the redundant region type of the target redundant pixel region is determined, and then the encrypted digital watermark is embedded in the pixel value of the target redundant pixel region according to the embedding strength. In this way, the characteristics of the redundant region are analyzed, and the embedding strength is adaptively adjusted according to the regional characteristics. For example, the embedding strength is increased in flat areas, while the embedding strength is appropriately reduced in areas with complex textures. Thus, the embedding strength is adaptively adjusted, so that the embedding of the watermark in different areas is more natural, the risk of being detected is reduced, and the survivability of the watermark under different video processing operations is improved, thereby achieving enhanced robustness.

[0046] In an embodiment of the present application, key frames are strategically selected from the video, and watermarks are adaptively implanted in hidden areas selected in the key frames. Compared with the current related art in which a digital watermark is implanted in each video frame in the video, this embodiment can support the digital watermark to achieve greater stability and robustness.

[0047] In step S215, a video matching model is trained based on the low-level video feature groups and video semantic feature groups corresponding to each legitimate video.

[0048] In step S216, each edge node is deployed respectively based on the video matching model.

[0049] Here, a video matching model is trained based on the low-level video feature group and video semantic feature group of each legitimate video, and the model is deployed on the edge node to achieve efficient video matching and tracking. In some embodiments, the configuration file of the video matching model and the corresponding legitimate video library can be sent to each edge node through an encrypted channel to achieve synchronous identification of the edge network.

[0050] In addition, with the iteration of the genuine video database, the core nodes also need to update the video matching model to learn the features of the newly added videos and then synchronize them on each edge node.

[0051] According to the operation process handled by the edge node, in step S221, when the edge node detects the video to be distributed, the low-level video feature group and the video semantic feature group corresponding to the video to be distributed are determined, and the video matching model is called to determine the target genuine video that matches the video to be distributed.

[0052] Therefore, after the edge node detects the video to be distributed, it quickly determines whether it is a genuine video through the video matching model, thereby improving the real-time nature of copyright protection.

[0053] In step S222, at least one key frame to be detected is screened according to the color histogram of each video frame in the video to be distributed, and at least one redundant pixel region to be detected in each key frame to be detected is extracted.

[0054] Similar to the key frame screening process of the core node, the edge node uses the same reverse engineering in the process of extracting watermarks. The key frames with large scene changes are screened by calculating the color histogram, and the digital watermark is queried in these key frames.

[0055] In step S223, it is identified whether there is an encrypted digital watermark in each redundant pixel region to be detected.

[0056] It should be noted that in order to enhance the concealment and hardness of digital watermarks, when the core node adds digital watermarks to key frames, it randomly selects redundant pixel areas to add encrypted digital watermarks. Accordingly, after detecting each redundant pixel area in the key frame, the edge node can poll or globally detect each redundant pixel area to achieve reliable extraction of the encrypted digital watermark.

[0057] In step S224, if it exists, the decrypted data corresponding to the encrypted digital watermark is obtained.

[0058] Exemplarily, the encrypted digital watermark is decrypted by a decryption algorithm that matches an advanced encryption algorithm (eg, AES) to obtain corresponding decrypted data.

[0059] In step S225, the decrypted data is matched and verified according to the identifier and copyright information corresponding to the target legitimate video.

[0060] In step S226, if there is no match, it is determined that the video to be distributed is an unlicensed video, and the distribution of the video to be distributed is refused.

[0061] Therefore, when an edge node detects an unlicensed video, it can immediately reject its distribution request, thus reducing the spread of pirated videos from the source.

[0062] In step S227, if there is a match, a real-time distribution record is generated according to the device information of the edge node and the current timestamp of the system, and the encrypted digital watermark is updated using the real-time distribution record, and distribution processing is performed based on the video to be distributed with the updated encrypted digital watermark.

[0063] Therefore, the edge node performs real-time detection and matching based on the characteristics of the video to be distributed, generates distribution records and updates encrypted digital watermarks, so as to embed distribution record evidence in the video material, which helps to quickly resolve copyright disputes.

[0064] In some cases, even if the digital watermark carried by the video to be distributed contains the same identifier and copyright information as the genuine video, it is still possible that the video to be distributed has been distributed non-compliantly in the past, such as being transmitted on other unauthorized platform servers.

[0065] In order to effectively curb the video distribution on the above unauthorized platforms, Figure 3 A flowchart of an example of distributing a video to be distributed according to an embodiment of the present application is shown.

[0066] like Figure 3 As shown, in step S310, it is analyzed whether there is a historical distribution record of the encrypted digital watermark.

[0067] In step S320, when there is a historical distribution record, the historical distribution record is sent to the core node, so that the core node verifies the historical distribution record according to the distribution record log corresponding to the target legitimate video.

[0068] In some embodiments, if the core node finds that the historical distribution record does not match the distribution record log, or finds that there is an unrecorded distribution behavior, the core node can determine that the video to be distributed is an unlicensed video, and send a rejection notice to the edge node, so that the edge node refuses to distribute the video to be distributed. Furthermore, a propagation path map for the video to be distributed can be constructed based on the device information and timestamps in the historical distribution record. Thus, through the propagation path map, the propagation path of the video can be intuitively displayed, helping the copyright owner to quickly locate the source of infringement and take measures.

[0069] In step S330, when it is determined that a verification pass message is received from the core node, a real-time distribution record is generated according to the device information of the edge node and the current timestamp of the system.

[0070] In step S340, the encrypted digital watermark is updated using the real-time distribution record, and the real-time distribution record is sent to the core node to update the distribution record log corresponding to the target legitimate video at the core node.

[0071] In step S350, distribution processing is performed based on the video to be distributed with the updated encrypted digital watermark.

[0072] In the embodiment of the present application, by parsing the historical distribution records contained in the encrypted digital watermark, the edge node can detect the past distribution path and behavior of the video to be distributed, so that the system can not only identify whether the current distribution request meets the copyright requirements, but also trace back and detect non-compliance in the historical distribution process, thereby greatly improving the comprehensiveness and sustainability of copyright protection. Especially in the case of dissemination on unauthorized platforms, the system can promptly discover and prevent non-compliant video distribution behavior.

[0073] In addition, after parsing the encrypted digital watermark, the edge node sends the historical distribution record to the core node for verification. The core node verifies the historical distribution record based on the distribution record log of the target genuine video. This verification mechanism ensures the accuracy and credibility of the distribution record and avoids copyright disputes caused by forged or tampered distribution records. Only when the historical distribution record is verified by the core node will a new real-time distribution record be generated, which further guarantees the legality and compliance of the distribution process and ensures the accuracy and credibility of the distribution record.

[0074] Furthermore, each distribution behavior will be updated in the distribution record log of the core node, which enables the system to accurately trace the distribution path and history of each video. Through this detailed tracing and monitoring mechanism, copyright owners can clearly understand the distribution of videos, promptly discover and deal with potential copyright infringements, achieve accurate distribution tracing and monitoring, and effectively safeguard copyright interests.

[0075] Figure 4 A schematic diagram of the structural connection of an example of a video matching model according to an embodiment of the present application is shown.

[0076] like Figure 4 As shown, the video matching model 400 includes a cascaded graph structure construction module 410, a TGN (Temporal Graph Networksm) encoding module 420 and a feature matching module 430.

[0077] The graph structure building module 410 is used to build a graph structure according to the low-level video feature group and the video semantic feature group corresponding to the video to be distributed.

[0078] Here, each graph node in the graph structure represents a corresponding video frame, and each edge connection is used to indicate the relationship between corresponding video frames. The features of each graph node are defined based on the low-level video feature group and video semantic feature group corresponding to the video frame of the graph node. The low-level video feature group includes inter-frame differences, color histograms, and texture patterns, while the video semantic feature group includes video object information, video scene information, and video event information. Through this comprehensive representation of multi-dimensional features, the graph structure can fully reflect the diversity and complexity of video content.

[0079] The TGN encoding module 420 is used to capture the temporal dependencies between the graph nodes in the graph structure to update the node features of the graph nodes in the graph structure. Specifically, the TGN encoding module calculates the relationship weight between each node and its neighbor nodes through the time step attention mechanism, and performs weighted updates on the node features. The module also uses a multi-head attention mechanism to enhance the diversity and expressiveness of feature representation, and aggregates features in the time dimension to capture the complex temporal relationship between video frames.

[0080] The feature matching module 430 is used to determine the overall feature representation of the first video corresponding to the video to be distributed based on the node features of each updated node, and then calculate the similarity between the overall feature representation of the second video corresponding to each genuine video in the genuine database and the overall feature representation of the first video, and detect the target genuine video that matches the video to be distributed based on the similarity calculation results.

[0081] More specifically, first, the feature matching module 430 aggregates the updated node features through the self-attention mechanism to generate an overall feature representation of the video to be distributed. Then, the feature matching module calculates the similarity between the overall feature representation of the second video corresponding to each legitimate video in the legitimate database and the overall feature representation of the first video. Exemplarily, the similarity calculation can use the cosine similarity method to measure the similarity between two feature vectors. Based on the similarity calculation results, the feature matching module can detect the target legitimate video that matches the video to be distributed, thereby achieving efficient and accurate video matching and copyright tracking.

[0082] In some embodiments, the feature matching module also includes a threshold judgment mechanism. When the similarity exceeds a preset threshold, it is determined that the video to be distributed matches the target legitimate video. If the match is successful, the copyright information of the target legitimate video is obtained for further copyright verification and tracking. Through this comprehensive feature matching and similarity calculation method, the model can maintain efficient and accurate matching performance in various complex scenarios.

[0083] More specifically, the video matching model 400 can also achieve matching between the video to be distributed and the target video in the video database in the following manner.

[0084] The structure of the graph structure building module 410 is:

[0085] Determine the node feature representation corresponding to each video frame in the video to be detected M1:

[0086] F t =[D t ,H t ,T t ,O t ,S t,E t ], Formula (2)

[0087] In the formula, F t Represents the graph node v of the tth frame t The corresponding node feature representation; D t represents the Euclidean distance between the t-th frame and the t-1-th frame, which is used to express the difference between frames; H t represents the color histogram of the tth frame; T t represents the texture pattern feature of the tth frame; t , S t and E t They respectively represent the object information, scene information, and event information of the tth frame detected using the pre-trained model.

[0088] Construct the graph node set corresponding to the graph structure:

[0089] V={v t |t=1,2,...,T}, Formula (3)

[0090] Where V represents the graph node set of the graph structure, and T is the total number of video frames of the video M1 to be detected.

[0091] Construct the edge connection set corresponding to the graph structure:

[0092] E={(v t ,v t+1 ,w t,t+1 )|t=1,2,…,T-1}, Formula (4)

[0093] In the formula, each edge connects (v t ,v t+1 ) represents the relationship between the tth frame and the t+1th frame, and the edge weight w t,t+1 It is defined based on the inter-frame difference between the t-th frame and the t+1-th frame.

[0094] In this embodiment, the feature dimension of the video frame contains not only static color and texture information, but also dynamic object, scene and event information. The comprehensive representation of these features can better distinguish different videos. Therefore, the model captures the multi-dimensional features of the video content by extracting multiple features such as inter-frame differences, color histograms, texture patterns, video object information, video scene information and video event information. The comprehensive representation of multiple features can more accurately reflect the content and changes of the video, and improve the richness and accuracy of the video feature representation.

[0095] The structure of the TGN encoding module 420 is:

[0096] For each graph node in the graph structure, the relationship weight between the graph node and its neighboring graph nodes is calculated to capture the local dependencies in the graph structure:

[0097] e ij =LeakyReLU(a T [WF i ‖WF j ]), Formula (5)

[0098] In the formula, F i represents the feature representation of graph node i; e ij represents the attention weight between graph node i and its neighboring graph node j; a T represents the weight vector of the attention mechanism; W represents the feature transformation matrix.

[0099] Calculate the normalized attention coefficient:

[0100]

[0101] In the formula, α ij Represents the normalized attention coefficient between graph node i and graph node j; Represents the set of neighbor graph nodes of graph node i.

[0102] Update node features of graph nodes based on multi-head attention mechanism:

[0103]

[0104] In the formula, H i represents the updated feature representation of graph node i, and G represents the number of attention heads; represents the attention coefficient of the hth attention head; W h Represents the feature transformation matrix of the h-th attention head.

[0105] Perform feature aggregation over multiple time steps to capture dynamic changes in the time dimension:

[0106]

[0107] In the formula, Represents the graph node v of the tth frame t The corresponding aggregate feature is represented; Δt represents the time window length; Aggregate(·) represents the aggregation function.

[0108] It should be noted that the actions and events in the video have obvious temporal dependencies. In this embodiment, TGN can capture these temporal changes and ensure that the model takes into account the dynamic changes of the temporal dimension when matching videos. Therefore, TGN uses a graph structure to model video frames, with nodes representing video frames and edges representing the relationship between frames, which can capture the temporal changes and complex dependencies of video content.

[0109] In addition, the multi-head attention mechanism can simultaneously capture multiple feature patterns in the video, such as color changes, object movement, and scene switching, enhancing the overall expressiveness of the model. Therefore, by calculating features in parallel through multiple attention heads, different attention heads can capture different feature patterns, enhancing the expressiveness and robustness of the model and improving the diversity and accuracy of feature representation.

[0110] The structure of the feature matching module 430 is:

[0111] Determine the overall feature representation of the video based on the aggregated feature representation of each graph node in the graph structure:

[0112]

[0113] In the formula, F M1 It represents the overall feature representation of the video M1 to be detected; Readout represents the global pooling operation of the global features of the reading graph.

[0114] The mean and standard deviation are calculated based on the aggregated features of each graph node in the graph structure to normalize the feature vectors of the overall feature representation of the video to be detected and the legitimate video:

[0115]

[0116]

[0117] In the formula, and They respectively represent the standardized overall video feature representation of the video to be detected M1 and the overall video feature representation of the legitimate video M2; μ and σ respectively represent the mean and standard deviation of the feature vector.

[0118] Calculate the cosine similarity between the overall feature vectors of the videos corresponding to M1 and M2 after standardization:

[0119]

[0120] In the formula, express and The cosine similarity between .

[0121] Will The similarity threshold is compared with the preset similarity threshold to detect whether the legitimate video M2 is the target legitimate video matching the video M1 to be distributed. For example, only when When the similarity threshold is exceeded, the system determines the legitimate video M2 as a potential target legitimate video for the video M1 to be distributed.

[0122] Through the collaborative work of the above modules, the video matching model 400 can achieve efficient matching and copyright tracking of videos to be distributed, significantly improving the reliability and accuracy of video copyright protection.

[0123] In this embodiment, the model can significantly improve the accuracy and robustness of video matching through comprehensive feature representation, temporal dependency capture and dynamic feature weight allocation. The model can adapt to various video processing operations. When facing videos to be distributed, even if the videos have been edited and compressed, the model can still accurately identify and match the corresponding genuine videos through multi-dimensional feature representation and temporal relationship capture.

[0124] In some examples of the embodiments of the present application, the loss function of the video matching model is:

[0125] L total =λ1L contrastive +λ2L triplet +λ3L center , Formula (13)

[0126] Where, L total Represents the comprehensive loss function of the video matching model; L contrastive , L triplet and L center They represent the contrast loss term, triplet loss term and center loss term respectively, and λ1, λ2 and λ3 represent the weight coefficients of the corresponding loss terms respectively.

[0127] The contrastive loss term aims to minimize the distance between matching video pairs and maximize the distance between non-matching video pairs.

[0128] The expression of the contrast loss term is:

[0129]

[0130]

[0131] Where N represents the total number of sample pairs in the data sample set; y i represents the label of sample pair i; d i represents the distance corresponding to sample pair i; m represents the boundary threshold, which is used to control the minimum distance of non-matching sample pairs; and They respectively represent the overall feature representation of the video corresponding to the two sample videos in sample pair i.

[0132] In this embodiment, the contrast loss enhances the discrimination ability of the model by maximizing the distance between non-matching video pairs and minimizing the distance between matching video pairs, thereby improving the accuracy of video matching. In addition, by setting the boundary threshold m, it is ensured that the model can effectively distinguish when facing negative sample pairs, so that positive sample pairs and negative sample pairs have obvious distinction in the feature space.

[0133] In the triplet loss term, the model’s discrimination ability is further improved by minimizing the distance between positive sample pairs and maximizing the distance between negative sample pairs.

[0134] The expression of the triplet loss term is:

[0135]

[0136]

[0137] In the formula, Represents the distance between the anchor sample and the positive sample, Represents the distance between the anchor sample and the negative sample; β represents the boundary threshold, which controls the minimum distance difference between the positive and negative sample pairs; Represents the overall feature representation of the video of anchor sample i, Represents the overall feature representation of the video of positive sample i, Represents the overall feature representation of the video of negative sample i.

[0138] In this embodiment, the triplet loss considers the distance relationship between positive samples and negative samples at the same time, making the features of samples of the same type closer and the features of samples of different types farther apart, thereby enhancing the feature differentiation ability of the model. In addition, by setting the boundary threshold β, it is ensured that the distance difference between positive samples and negative samples is large enough, thereby reducing the probability of mismatching and improving matching accuracy.

[0139] In the center loss term, the intra-class compactness of the feature representation is improved by minimizing the distance between each class sample and its class center.

[0140] The expression of the center loss term is:

[0141]

[0142]

[0143] In the formula, F i represents the overall feature representation of the video of sample i, Indicates the category y to which sample i belongs i center ofj represents the center of category j; δ(y i =j) means that if sample i belongs to category j, it takes 1, otherwise it takes 0.

[0144] It should be noted that in the video matching task, it may not be realistic to directly use class labels to define class centers, because the types and features of video content are diverse. In an embodiment of the present application, video features are clustered by an unsupervised clustering algorithm (such as K-means, DBSCAN, etc.), the clustering results are used as pseudo labels, and the center of each cluster is defined as the class center. Furthermore, during the training process, the class center is dynamically updated to ensure that the feature representation gradually converges. Thus, pseudo labels and class centers are generated by unsupervised clustering to adapt to diverse and complex video content and enhance the generalization ability of the model.

[0145] Through this embodiment, the contrast loss term is introduced to improve the generalization ability of the model on different video contents by minimizing the distance of matching sample pairs and maximizing the distance of non-matching sample pairs. The triplet loss term is introduced to simultaneously consider the distance relationship between anchor samples, positive samples and negative samples, thereby enhancing the model's recognition ability for unseen samples and improving the generalization performance of the model. The center loss term is introduced to minimize the distance between the sample feature and its class center, thereby improving the intra-class compactness of the feature representation, making the features of samples of the same type closer, enhancing the intra-class compactness, and improving the model's ability to distinguish samples of different pseudo-label categories.

[0146] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of actions combined, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0147] Figure 5 A structural block diagram of an example of a copyright tracking system for unlicensed video distribution according to an embodiment of the present application is shown.

[0148] like Figure 5 As shown, the copyright tracking system 500 for unlicensed video distribution includes a first device 510 deployed at a core node of a CDN system and a second device 520 deployed at each edge node.

[0149] The first device 510 includes a watermark embedding unit 511, a model training unit 512, and a model deployment unit 513. The second device 520 includes a video matching unit 521, a frame region extraction unit 522, a watermark identification unit 523, a decryption verification unit 524, and a watermark updating unit 525.

[0150] The watermark embedding unit 511 is used to generate an identifier corresponding to each genuine video in the genuine database, and extract a low-level video feature group and a video semantic feature group corresponding to the genuine video; the low-level video feature group includes: frame difference, color histogram and texture pattern; the video semantic feature group includes: video object information, video scene information and video event information; for each first video frame in the genuine video, calculate the color histogram difference between the first video frame and the adjacent second video frame; if the color histogram difference exceeds a preset threshold, determine the first video frame and / or the second video frame as a key frame, identify at least one redundant pixel area in each of the key frames, and randomly select a target redundant pixel area from each of the redundant pixel areas; the redundant pixel area includes a flat area, a complex texture area and an edge area; generate an encrypted digital watermark based on the identifier and the copyright information of the genuine video, and embed the encrypted digital watermark into the pixel value of the target redundant pixel area.

[0151] The model training unit 512 is used to train the video matching model based on the low-level video feature group and the video semantic feature group corresponding to each legitimate video.

[0152] The model deployment unit 513 is used to deploy each of the edge nodes based on the video matching model.

[0153] The video matching unit 521 is used to determine the low-level video feature group and video semantic feature group corresponding to the video to be distributed when the edge node detects the video to be distributed, and call the video matching model to determine the target genuine video that matches the video to be distributed.

[0154] The frame region extraction unit 522 is used to screen at least one key frame to be detected according to the color histogram of each video frame in the video to be distributed, and extract at least one redundant pixel region to be detected in each key frame to be detected.

[0155] The watermark identification unit 523 is used to identify whether there is an encrypted digital watermark in each of the redundant pixel regions to be detected.

[0156] The decryption verification unit 524 is used to obtain the decrypted data corresponding to the encrypted digital watermark if it exists; to match and verify the decrypted data according to the identifier and copyright information corresponding to the target legitimate video; if there is no match, it is determined that the video to be distributed is an unlicensed video, and the distribution of the video to be distributed is refused.

[0157] The watermark updating unit 525 is used to generate a real-time distribution record according to the device information of the edge node and the current system timestamp if a match is found, and to update the encrypted digital watermark using the real-time distribution record, and to perform distribution processing based on the video to be distributed with the updated encrypted digital watermark.

[0158] In some embodiments, an embodiment of the present application provides a non-volatile computer-readable storage medium, which stores one or more programs including execution instructions, and the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to execute the steps of any of the above-mentioned copyright tracking methods for unlicensed video distribution in the present application.

[0159] In some embodiments, the embodiments of the present application also provide a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer performs any step of the above-mentioned copyright tracking method for unlicensed video distribution.

[0160] In some embodiments, the embodiments of the present application also provide an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of a copyright tracking method for unlicensed video distribution.

[0161] Figure 6 is a schematic diagram of the hardware structure of an electronic device for executing a copyright tracking method for unlicensed video distribution provided by another embodiment of the present application, such as Figure 6 As shown, the device includes:

[0162] One or more processors 610 and memory 620, Figure 6 A processor 610 is taken as an example.

[0163] The apparatus for executing the copyright tracking method for unlicensed video distribution may further include: an input device 630 and an output device 640 .

[0164] The processor 610, the memory 620, the input device 630 and the output device 640 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0165] The memory 620, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the copyright tracking method for unlicensed video distribution in the embodiment of the present application. The processor 610 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 620, that is, the copyright tracking method for unlicensed video distribution in the above method embodiment is implemented.

[0166] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 620 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 620 may optionally include a memory remotely arranged relative to the processor 610, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The input device 630 can receive input digital or character information and generate signals related to user settings and function control of the electronic device. The output device 640 can include a display device such as a display screen.

[0168] The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, the copyright tracking method for unauthorized video distribution in any of the above method embodiments is performed.

[0169] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0170] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to:

[0171] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0172] (2) Ultra-mobile personal computer devices: These devices belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0173] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0174] (4) Other onboard electronic devices with data interaction functions, such as on-board devices installed in vehicles.

[0175] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A copyright tracking method for unlicensed video distribution, applied to a CDN system, wherein the CDN system includes a core node and a plurality of edge nodes, the method comprising: The core node is used to perform the following operations: For each legitimate video in the legitimate database, Generate an identifier corresponding to the legitimate video, and extract a low-level video feature group and a video semantic feature group corresponding to the legitimate video; the low-level video feature group includes: inter-frame difference, color histogram and texture pattern; the video semantic feature group includes: video object information, video scene information and video event information; For each first video frame in the legitimate video, calculating a color histogram difference between the first video frame and an adjacent second video frame; If the color histogram difference exceeds a preset threshold, the first video frame and / or the second video frame is determined as a key frame, at least one redundant pixel region in each of the key frames is identified, and a target redundant pixel region is randomly selected from each of the redundant pixel regions; the redundant pixel region includes a flat region, a complex texture region, and an edge region; Generate an encrypted digital watermark according to the identifier and the copyright information of the legitimate video, and embed the encrypted digital watermark into the pixel value of the target redundant pixel area; Based on the low-level video feature groups and video semantic feature groups corresponding to each legitimate video, a video matching model is trained; Based on the video matching model, deploying each of the edge nodes respectively; Each of the edge nodes is used to perform the following operations: When the edge node detects a video to be distributed, determining a low-level video feature group and a video semantic feature group corresponding to the video to be distributed, and calling the video matching model to determine a target legitimate video that matches the video to be distributed; Screening at least one key frame to be detected according to the color histogram of each video frame in the video to be distributed, and extracting at least one redundant pixel region to be detected in each key frame to be detected; Identify whether there is an encrypted digital watermark in each of the redundant pixel regions to be detected; If it exists, obtaining the decrypted data corresponding to the encrypted digital watermark; Matching and verifying the decrypted data according to the identifier and copyright information corresponding to the target legitimate video; If they do not match, determining that the video to be distributed is an unlicensed video, and refusing to distribute the video to be distributed; as well as If there is a match, a real-time distribution record is generated according to the device information of the edge node and the current system timestamp, and the encrypted digital watermark is updated using the real-time distribution record, and distribution processing is performed based on the video to be distributed with the updated encrypted digital watermark.

2. The method according to claim 1, wherein: If the color histogram difference exceeds a preset threshold, determining the first video frame and / or the second video frame as a key frame includes: In the formula, H t (i) represents the i-th color histogram value of the t-th frame, if ΔH t,t+1 >color histogram threshold, the tth frame and / or the t+1th frame are determined as key frames.

3. The method according to claim 1, wherein: If there is a match, a real-time distribution record is generated according to the device information of the edge node and the current timestamp of the system, and the encrypted digital watermark is updated using the real-time distribution record, and distribution processing is performed based on the video to be distributed with the updated encrypted digital watermark, including: Analyze whether there is a historical distribution record of the encrypted digital watermark; When there is a historical distribution record, the historical distribution record is sent to the core node, so that the core node verifies the historical distribution record according to the distribution record log corresponding to the target legitimate video; In the case of determining that a verification pass message is received from the core node, generating a real-time distribution record according to the device information of the edge node and the current timestamp of the system; Using the real-time distribution record to update the encrypted digital watermark, and sending the real-time distribution record to the core node, so as to update the distribution record log corresponding to the target legitimate video at the core node; Distribution processing is performed based on the video to be distributed with the updated encrypted digital watermark.

4. The method according to claim 3, wherein: After the edge node sends the historical distribution record to the core node so that the core node verifies the historical distribution record according to the distribution record log corresponding to the target legitimate video, the core node is further configured to perform the following operations: Determining that the video to be distributed is an unlicensed video, and sending a rejection notification to the edge node so that the edge node refuses to distribute the video to be distributed; According to the device information and timestamp in the historical distribution record, a propagation path diagram for the video to be distributed is constructed.

5. The method according to claim 1, wherein: The step of embedding the encrypted digital watermark into the pixel value of the target redundant pixel area comprises: determining an embedding strength matching a redundant region type of the target redundant pixel region; The encrypted digital watermark is embedded into the pixel value of the target redundant pixel area according to the embedding strength.

6. The method according to any one of claims 1 to 5, wherein: The video matching model comprises a cascaded graph structure building module, a TGN encoding module and a feature matching module; The graph structure construction module is used to construct a graph structure according to the low-level video feature group and the video semantic feature group corresponding to the video to be distributed; each graph node in the graph structure represents a corresponding video frame, and each edge connection is used to indicate the relationship between the corresponding video frames; the features of each graph node are defined according to the low-level video feature group and the video semantic feature group corresponding to the video frame of the graph node; The TGN encoding module is used to capture the temporal dependency between each graph node in the graph structure to update the node features of each graph node in the graph structure; The feature matching module is used to determine the overall feature representation of the first video corresponding to the video to be distributed based on the node features of each node after update, and then calculate the similarity between the overall feature representation of the second video corresponding to each genuine video in the genuine database and the overall feature representation of the first video, and detect the target genuine video that matches the video to be distributed based on the similarity calculation results.

7. The method according to claim 6, wherein: The structure of the graph structure building module is: Determine the node feature representation corresponding to each video frame in the video to be detected M1: F t =[D t ,H t ,T t ,O t ,S t ,E t ] In the formula, F t Represents the graph node v of the tth frame t The corresponding node feature representation; D t represents the Euclidean distance between the t-th frame and the t-1-th frame, which is used to express the difference between frames; H t represents the color histogram of the tth frame; T t represents the texture pattern feature of the tth frame; t , S t and E t Respectively represent the object information, scene information, and event information of the tth frame detected using the pre-trained model; Construct the graph node set corresponding to the graph structure: V={v t |t=1,2,...,T} Where V represents the graph node set of the graph structure, T is the total number of video frames of the video M1 to be detected; Construct the edge connection set corresponding to the graph structure: E={(v t ,v t+1 ,w t,t+1 )|t=1,2,...,T-1} In the formula, each edge connects (v t ,v t+1 ) represents the relationship between the tth frame and the t+1th frame, and the edge weight w t,t+1 Defined according to the inter-frame difference corresponding to the t-th frame and the t+1-th frame; The structure of the TGN encoding module is: For each graph node in the graph structure, the relationship weight between the graph node and its neighboring graph nodes is calculated to capture the local dependency in the graph structure: e ij =LeakyReLU(a T [WF i ||WF j ]) In the formula, F i represents the feature representation of graph node i; e ij represents the attention weight between graph node i and its neighboring graph node j; a T Represents the weight vector of the attention mechanism; W represents the feature transformation matrix; Calculate the normalized attention coefficient: In the formula, α ij Represents the normalized attention coefficient between graph node i and graph node j; N i Represents the set of neighbor graph nodes of graph node i; Update node features of graph nodes based on multi-head attention mechanism: In the formula, H i represents the updated feature representation of graph node i, and G represents the number of attention heads; represents the attention coefficient of the hth attention head; W h represents the feature transformation matrix of the h-th attention head; Perform feature aggregation over multiple time steps to capture dynamic changes in the time dimension: In the formula, Represents the graph node v of the tth frame t The corresponding aggregate feature is represented; Δt represents the time window length; Aggregate(·) represents the aggregation function; The structure of the feature matching module is: Determine the overall feature representation of the video based on the aggregated feature representation of each graph node in the graph structure: In the formula, F M1 Represents the overall feature representation of the video corresponding to the video to be detected M1; Readout represents the global pooling operation of the global features of the read graph; The mean and standard deviation are calculated based on the aggregated features of each graph node in the graph structure to normalize the feature vectors of the overall feature representation of the video to be detected and the legitimate video: In the formula, and They represent the standardized overall video feature representation of the video to be detected M1 and the overall video feature representation of the legitimate video M2 respectively; μ and σ represent the mean and standard deviation of the feature vector respectively; Calculate the cosine similarity between the overall feature vectors of the videos corresponding to M1 and M2 after standardization: In the formula, express and The cosine similarity between ; Will The video M2 is compared with a preset similarity threshold to detect whether the legitimate video M2 is a target legitimate video that matches the video M1 to be distributed.

8. The method according to claim 7, wherein: The loss function of the video matching model is: L total =λ1L contrastive +λ2L triplet +λ3L center Where, L total Represents the comprehensive loss function of the video matching model; L contrastive , L triplet and L center They represent contrast loss term, triplet loss term and center loss term respectively, λ1, λ2 and λ3 represent the weight coefficients of the corresponding loss terms respectively; The expression of the contrast loss term is: Where N represents the total number of sample pairs in the data sample set; y i represents the label of sample pair i; d i represents the distance corresponding to sample pair i; m represents the boundary threshold, which is used to control the minimum distance of non-matching sample pairs; and Respectively represent the overall video feature representations corresponding to the two sample videos in sample pair i; The expression of the triple loss term is: In the formula, Represents the distance between the anchor sample and the positive sample, Represents the distance between the anchor sample and the negative sample; β represents the boundary threshold, which controls the minimum distance difference between the positive and negative sample pairs; Represents the overall feature representation of the video of anchor sample i, Represents the overall feature representation of the video of positive sample i, Represents the overall feature representation of the video of negative sample i; The expression of the center loss term is: In the formula, F i represents the overall feature representation of the video of sample i, Indicates the category y to which sample i belongs i center of j represents the center of category j; δ(y i =j) means that if sample i belongs to category j, it takes 1, otherwise it takes 0.

9. A copyright tracking system for unlicensed video distribution, applied to a CDN system, wherein the CDN system comprises a core node and a plurality of edge nodes, the system comprising a first device deployed at the core node and a second device deployed at each edge node respectively: The first device comprises: The watermark embedding unit is used to generate an identifier corresponding to each legitimate video in the legitimate database, and extract a low-level video feature group and a video semantic feature group corresponding to the legitimate video; the low-level video feature group includes: inter-frame difference, color histogram and texture pattern; the video semantic feature group includes: video object information, video scene information and video event information; for each first video frame in the legitimate video, calculate the color histogram difference between the first video frame and the adjacent second video frame; if the color histogram difference exceeds a preset threshold, determine the first video frame and / or the second video frame as a key frame, identify at least one redundant pixel area in each of the key frames, and randomly select a target redundant pixel area from each of the redundant pixel areas; the redundant pixel area includes a flat area, a complex texture area and an edge area; generate an encrypted digital watermark according to the identifier and the copyright information of the legitimate video, and embed the encrypted digital watermark into the pixel value of the target redundant pixel area; A model training unit, used for training a video matching model based on a low-level video feature group and a video semantic feature group corresponding to each legitimate video; A model deployment unit, configured to deploy each of the edge nodes based on the video matching model; The second device comprises: A video matching unit, configured to determine a low-level video feature group and a video semantic feature group corresponding to the video to be distributed when the edge node detects the video to be distributed, and call the video matching model to determine a target legitimate video that matches the video to be distributed; A frame region extraction unit, configured to screen at least one key frame to be detected according to a color histogram of each video frame in the video to be distributed, and extract at least one redundant pixel region to be detected in each key frame to be detected; A watermark identification unit, used for identifying whether there is an encrypted digital watermark in each of the redundant pixel regions to be detected; A decryption verification unit, configured to obtain the decrypted data corresponding to the encrypted digital watermark if it exists; perform matching verification on the decrypted data according to the identifier and copyright information corresponding to the target legitimate video; if there is no match, determine that the video to be distributed is an unlicensed video, and refuse to distribute the video to be distributed; The watermark updating unit is used to generate a real-time distribution record according to the device information of the edge node and the current system timestamp if a match is found, and to update the encrypted digital watermark using the real-time distribution record, and to perform distribution processing based on the video to be distributed with the updated encrypted digital watermark.

10. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images