Role copyright infringement detection method and system based on visual and semantic features

Abstract

The application discloses a role copyright infringement detection method and system based on visual and semantic features, the method comprises the following steps: S1, role target separation and feature extraction: input the content to be detected, separate the role target therein, and extract the role features, the role features comprising at least one of visual features, semantic features and time sequence features; S2, role feature correspondence and matching: the extracted role features are matched with the existing role features in the copyright content library; S3, role feature similarity analysis: when the similarity between the extracted role and the matched role exceeds a preset threshold, the role is determined as a potential infringing role. The application takes the visual and semantic features of the role as the core, constructs a whole-process detection framework from target separation to feature matching and then to similarity analysis, can effectively solve the low-efficiency problem of the prior art which depends on the manual comparison method, provides an automatic and intelligent solution for the role copyright infringement identification of large-scale digital content, and significantly improves the detection efficiency and the scientificity of the identification.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence and computer vision technology, specifically relating to a method and system for detecting copyright infringement of characters based on visual and semantic features. Background Technology

[0002] With the rapid development of the digital content industry, character images have gradually become an important core intellectual property. However, due to the high replicability of digital content, copyright infringement issues related to character images are becoming increasingly prominent. For example, unauthorized copying, modification, or impersonation of characters, and even unauthorized video copying and editing, frequently occur in virtual reality, game development, and film production, posing a serious threat to the rights of creators.

[0003] Currently, the determination of character copyright infringement mainly relies on legal standards and manual comparative analysis. For example, it involves comparing a character's visual appearance, personality traits, and their relationship to the core content of the work to determine whether substantial similarity exists. However, this method heavily depends on manual analysis, resulting in low efficiency and difficulty in handling today's environment of high-volume digital content production and rapid dissemination. With the generation of massive amounts of character images in a short period, traditional manual comparison methods can no longer quickly and accurately determine infringement. Furthermore, the increasingly diverse digital expressions of character images, such as virtual characters and generative AI creations, further increase the complexity of infringement determination. Therefore, existing methods are insufficient to meet the demands of the digital age for efficient and accurate infringement determination, and innovative technological means are urgently needed to support this.

[0004] Furthermore, existing character image infringement determinations are typically limited to the character's visual image itself, such as the graphic design of Bing Dwen Dwen or Mickey Mouse, while lacking a comprehensive definition of infringement for the character's multimodal characteristics. For example, the character's behavioral characteristics, action characteristics, plot characteristics, and multimodal expressions such as audio, video, and text have not yet been included in a systematic scope of determination.

[0005] To address this, this application proposes a character copyright infringement detection method based on visual and semantic features to tackle the diverse infringement challenges of digital content and improve the efficiency and accuracy of infringement determination. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a method and system for detecting copyright infringement of characters based on visual and semantic features. This invention takes the visual and semantic features of characters as its core and constructs a full-process detection framework from target separation to feature matching and then to similarity analysis. It can achieve efficient and accurate detection of character image infringement behaviors with multimodal features in complex and ever-changing content scenarios.

[0007] The technical solution adopted by this invention to solve its technical problem is as follows:

[0008] A method for detecting copyright infringement of characters based on visual and semantic features, including

[0009] S1 Character Target Separation and Feature Extraction

[0010] Input the content to be detected, separate the character targets, and extract the character features, which include at least one of visual features, semantic features, and temporal features;

[0011] S2 Character Traits Correspondence and Matching

[0012] The extracted character features are matched with existing character features in the copyright content library to find similar characters;

[0013] S3 Character Feature Similarity Analysis

[0014] When the similarity between the extracted character and the matched character exceeds a preset threshold, it can be identified as a potential infringing character.

[0015] Furthermore, to achieve character target separation and feature extraction, S1 includes...

[0016] S1.1 Multimodal Data Collection and Preprocessing

[0017] Preprocess the input multimodal data to be detected to ensure the consistency of subsequent model inputs and the completeness of multimodal feature extraction;

[0018] S1.11 Multimodal Data Reception

[0019] The multimodal data includes images, videos, and text;

[0020] Input data supports image formats such as JPEG and PNG, video formats such as MP4 and AVI, as well as subtitles or descriptive text data;

[0021] S1.12 Video Data Frame Decomposition

[0022] The input video is decoded into consecutive frames at a specified frame rate, and each frame is timestamped.

[0023] S1.13 Image and Video Standardization

[0024] Image standardization is performed on each input image / frame, including resolution adjustment, color normalization, and noise reduction.

[0025] S1.14 Text Data Processing

[0026] Text data that can be used for semantic feature extraction is generated through methods such as word segmentation and encoding;

[0027] S1.2 Role-Target Separation

[0028] S1.21 Instance Segmentation

[0029] A convolutional neural network model is used to generate segmentation masks for each image / frame, and cross-frame target tracking is optimized by temporal series consistency loss.

[0030] The formula for time series consistency loss is:

[0031]

[0032] in: It is the total number of frames in the video; Indicates the first Frame target segmentation mask; This indicates the intersection over union ratio of two masks.

[0033] S1.22 Enhanced Dynamic Hints

[0034] Combining point cues and mask cues improves segmentation accuracy in complex scenes;

[0035] The formula for the segmentation weight is:

[0036]

[0037] in: Indicates the splitting weight; Indicates the similarity between target features and background features; This indicates the region weights generated by the prompt; and Weighting coefficients for features and regions;

[0038] S1.23 Occlusion and Overlap Optimization

[0039] By introducing a dynamic memory module and optical flow estimation, the model's continuous target tracking capability is improved.

[0040] S1.3 Character Feature Extraction

[0041] Multimodal representations are generated by combining visual and semantic features for subsequent feature matching and similarity analysis;

[0042] S1.31 Visual Feature Extraction

[0043] High-level features of image data, including texture, edges, and shape, are extracted using a convolutional neural network model, and visual feature vectors of characters are generated.

[0044] S1.32 Semantic Feature Extraction

[0045] Generate semantic feature vectors from text data using a semantic representation model;

[0046] S1.33 Temporal Feature Enhancement

[0047] By modeling consecutive frames using an LSTM model, changes in the temporal dimension can be captured.

[0048]

[0049] in, Indicates time The captured temporal features, This represents visual features containing time-series information after being modeled using LSTM. Indicates the first The visual feature vector of a frame; Represents consecutive time frames. The size of the time window;

[0050] S1.34 Multimodal Feature Fusion

[0051] By integrating visual, semantic, and temporal features, a unified multimodal representation is generated.

[0052]

[0053] in, This represents a multimodal feature vector that integrates visual, semantic, and temporal features, used to comprehensively represent the target role; Represents the visual feature vector of the character; A semantic feature vector representing a character; Represents the temporal feature vector of a character; The fusion weights represent visual features and are used to adjust the influence of visual features on the final representation. The fusion weights represent the semantic features and are used to adjust the influence of semantic features on the final representation. The fusion weights represent the temporal features and are used to adjust the influence of temporal features on the final representation.

[0054] S1.4 Result Output

[0055] S1.41 Feature Storage

[0056] The generated character features are stored in a database for subsequent analysis;

[0057] S1.42 Visualization Results

[0058] Output visualizations of segmentation masks and feature embeddings for easy debugging;

[0059] S1.43 Output Format

[0060] Embedded vectors including visual features, semantic features, and temporal features.

[0061] Furthermore, to achieve character feature correspondence and matching, S2 includes

[0062] S2.1 Feature Representation and Storage

[0063] S2.11 Building a Copyright Content Library

[0064] S2.111 Character Target Separation: Extracting character instances from copyrighted content;

[0065] S2.112 Feature Extraction: Perform feature extraction on the extracted role instances, including...

[0066] Visual features: Extract key image information of the character (such as outline, color, texture, key points, etc.) and generate visual feature vectors;

[0067] Semantic features: Analyze the character's text description, semantic tags, behavioral features, plot connections, and other information, and generate semantic feature vectors;

[0068] Multimodal feature fusion: Integrating character features across different modalities to form a unified, multi-dimensional character feature representation. ;

[0069]

[0070] in: Indicates visual features; Represents semantic features; Indicates time-series characteristics (optional);

[0071] S2.113 Standardization and Storage

[0072] The extracted character features are stored in the copyright content library according to a unified standard; each character instance corresponds to a unique identifier in the library, which is associated with its visual features, semantic features and related metadata (such as copyright holder, registration time, copyright type, etc.).

[0073] S2.114 Dynamic Updates and Optimizations

[0074] The copyright content library supports dynamic updates, allowing the addition of new character data and regular optimization of the feature representation of existing data to maintain the comprehensiveness and accuracy of the content library;

[0075] S2.12 Representation of features to be matched

[0076] The modal features are characterized based on the role features output by S1;

[0077] Single-modal features: such as visual features only ;

[0078] Multimodal features: such as visual features and semantic features ;

[0079] S2.2 Feature Preprocessing

[0080] S2.21 Feature Alignment

[0081] For a single modal feature, calculate the similarity between the modal feature and the corresponding feature in the copyright content library;

[0082] For multimodal features, the similarity between each modal feature and the corresponding feature in the copyright content library is calculated separately;

[0083] S2.22 Feature Normalization

[0084] All modal features are normalized to ensure that the feature vectors have a consistent range;

[0085] S2.3 Matching Search Algorithm

[0086] S2.31 Intramodal Matching

[0087] Visual feature matching: Calculating cosine similarity or other similarities

[0088] Cosine similarity:

[0089] Or Euclidean distance (anti-similarity):

[0090] Or structural similarity (SSIM): ;

[0091] Semantic feature matching: Calculating cosine similarity or other similarities

[0092] Calculate cosine similarity:

[0093] Or Jaccard similarity:

[0094] Word Shift Distance (WMD): Calculates the overall semantic distance between two text embeddings;

[0095] Temporal feature matching: Calculate the dynamic time warping (DTW) distance or other metrics for time series features.

[0096] Calculate Dynamic Time Warping (DTW):

[0097] Or slice similarity: Divide the time series into multiple segments, calculate the cosine similarity of each segment and take the average value;

[0098] S2.32 Intermodal Fusion

[0099] Calculate the total similarity based on the weights of the character feature modalities:

[0100]

[0101] in: This represents the overall similarity score; , , Weights for visual, semantic, and temporal features;

[0102] S2.33 Multimodal Search Optimization

[0103] Perform fast nearest neighbor search on high-dimensional features;

[0104] S2.4 Matching Result Filtering

[0105] S2.41 Set threshold

[0106] Only retain role matching results with a similarity score higher than a set threshold. The threshold can be adjusted according to actual application needs.

[0107] S2.42 Candidate Role Ranking

[0108] Based on total similarity Sort the matching results and output the Top-K candidate roles and feature sets;

[0109]

[0110] Each matched role feature contains multiple modalities. .

[0111] Furthermore, to achieve character feature similarity analysis, S3 includes...

[0112] The Top-K character feature set and total similarity are obtained based on S2 matching. Conduct character feature similarity analysis;

[0113] S3.1 Advanced Analysis

[0114] S3.11 Multi-level Validation

[0115] For matched characters with high visual similarity, further examine semantic and temporal features;

[0116] S3.12 Conflict Resolution

[0117] If there is a conflict in the similarity scores between modalities, the result of the modality with the higher weight shall be given priority.

[0118] S3.2 Output Results

[0119] like Higher than the set threshold If a character is identified as a potential infringer, and the similarity score of at least one modality exceeds its specificity threshold, then the character is identified as a potential infringer.

[0120] Optionally, a report can be generated from the character feature similarity analysis results, including the overall similarity score, the similarity score for each modality, comparative evidence (such as visual matching diagrams and text comparison results), and a possible Top-N list of infringing characters.

[0121] The beneficial effects of this invention are as follows:

[0122] 1. High efficiency: By constructing a full-process detection framework from character target separation to feature matching and similarity analysis, this invention can automatically identify and analyze copyright infringement of characters, greatly improving detection efficiency, and is especially suitable for the rapid processing of large-scale digital content.

[0123] 2. Accuracy: Based on the combination of visual and semantic features, this invention can capture the multidimensional characteristics of a character, not limited to appearance similarity, but also covering complex features such as behavioral features and semantic associations, thereby significantly improving the accuracy of infringement determination.

[0124] 3. Applicability to multiple scenarios: The method of this invention is applicable to various digital content scenarios such as film and television, games, and animation, especially the complex application environment of virtual characters and generative artificial intelligence creation, which can meet the diverse needs of the digital content industry for copyright protection.

[0125] 4. Intelligentization: By using artificial intelligence technology to extract character characteristics and perform matching analysis, infringement determination is transformed from traditional manual analysis to an intelligent and automated method, which greatly reduces labor costs and time consumption.

[0126] 5. Scalability: The detection framework and copyright content library of this invention have good scalability. The feature extraction algorithm, matching method and analysis model can be continuously updated according to needs to adapt to the ever-changing content forms and infringement methods.

[0127] 6. Industry Value: This invention provides a scientific and systematic solution for detecting copyright infringement in the digital content industry. It not only effectively protects the rights and interests of creators, but also promotes the development of digital copyright protection technology and provides technical support for relevant laws and industry practices.

[0128] In summary, this invention not only improves the efficiency and accuracy of character copyright infringement detection, but also significantly expands the applicable scenarios and industry value of the detection method, providing strong technical support for digital content copyright protection. Attached Figure Description

[0129] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0130] Figure 1 This is a flowchart of the copyright infringement detection process for the character in this invention;

[0131] Figure 2 This is a flowchart of the process for constructing a copyright content library according to the present invention;

[0132] Figure 3 This is a flowchart of the character target separation and feature extraction in a specific implementation example of the present invention;

[0133] Figure 4 This is a flowchart of the role feature correspondence and matching in a specific implementation example of the present invention;

[0134] Figure 5 This is a flowchart of the role feature similarity analysis in a specific implementation example of the present invention. Detailed Implementation

[0135] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0136] Furthermore, the following description is for illustrative purposes and not for limitation, and sets forth specific details such as particular system architectures and data models to provide a thorough understanding of the embodiments of the present invention. However, those skilled in the art will understand that the present invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, models, and methods are omitted to avoid unnecessary detail that could obscure the description of the present invention.

[0137] This embodiment relates to a method for detecting copyright infringement of characters based on visual and semantic features, referring to... Figure 1 The method includes

[0138] S1 Character Target Separation and Feature Extraction

[0139] Input the content to be detected, separate the character targets, and extract the character features, which include at least one of visual features, semantic features, and temporal features;

[0140] Visual features: Describe the character's appearance (such as color and texture);

[0141] Semantic features: derived from the character's textual description (such as name, identity, and backstory);

[0142] Temporal characteristics: Describe the dynamic changes of a character in a video (such as movement trajectories);

[0143] S2 Character Traits Correspondence and Matching

[0144] The extracted character features are matched with existing character features in the copyright content library to find similar characters;

[0145] S3 Character Feature Similarity Analysis

[0146] When the similarity between the extracted character and the matched character exceeds a preset threshold, it can be identified as a potential infringing character.

[0147] In this embodiment, to achieve character target separation and feature extraction, refer to Figure 3 S1 specifically includes

[0148] S1.1 Multimodal Data Collection and Preprocessing

[0149] Preprocess the input multimodal data to be detected to ensure the consistency of subsequent model inputs and the completeness of multimodal feature extraction;

[0150] S1.11 Multimodal Data Reception

[0151] The multimodal data includes images, videos, and text;

[0152] Input data supports image formats such as JPEG and PNG, video formats such as MP4 and AVI, as well as subtitles or descriptive text data;

[0153] S1.12 Video Data Frame Decomposition

[0154] The input video is decoded into consecutive frames at a specified frame rate (e.g., 24 frames per second), and each frame is timestamped.

[0155] S1.13 Image and Video Standardization

[0156] Image normalization is performed on each input image / frame, including resolution adjustment (e.g., 512×512 pixels), color normalization, and noise reduction.

[0157] S1.14 Text Data Processing

[0158] Text data that can be used for semantic feature extraction is generated through methods such as word segmentation and encoding;

[0159] S1.2 Role-Target Separation

[0160] S1.21 Instance Segmentation

[0161] This embodiment uses an enhanced Mask Track R-CNN model to generate segmentation masks for each image / frame, and optimizes cross-frame target tracking through temporal consistency loss;

[0162] The formula for time series consistency loss is:

[0163]

[0164] in: It is the total number of frames in the video; Indicates the first Frame target segmentation mask; This indicates the intersection over union ratio of two masks.

[0165] S1.22 Enhanced Dynamic Hints

[0166] Combining point cues and mask cues improves segmentation accuracy in complex scenes;

[0167] The formula for the segmentation weight is:

[0168]

[0169] in: Indicates the splitting weight; Indicates the similarity between target features and background features; This indicates the region weights generated by the prompt; and Weighting coefficients for features and regions;

[0170] S1.23 Occlusion and Overlap Optimization

[0171] By introducing a dynamic memory module and optical flow estimation, the model's continuous target tracking capability is improved.

[0172] S1.3 Character Feature Extraction

[0173] Multimodal representations are generated by combining visual and semantic features for subsequent feature matching and similarity analysis;

[0174] S1.31 Visual Feature Extraction

[0175] This embodiment utilizes the ResNet (Residual Network) model to extract high-level features from image data, including texture, edges, and shape, and generates visual feature vectors for the character.

[0176]

[0177] in, Indicate target Visual feature vectors, This represents the global visual appearance features extracted using the ResNet model. Represents the visual data of the input image frame or character;

[0178] S1.32 Semantic Feature Extraction

[0179] Semantic representation models, such as BERT (Bidirectional Encoder Representations from Transformers), are used to generate semantic feature vectors from text data.

[0180]

[0181] in, Indicate target semantic feature vectors, This represents the semantic embedding of text generated by the BERT model. The input text data is usually descriptive text related to the character, such as subtitles, tags, etc.

[0182] S1.33 Temporal Feature Enhancement

[0183] By modeling consecutive frames using the LSTM (Long Short-Term Memory) model, information about changes in the temporal dimension can be captured.

[0184]

[0185] in, Indicates time The captured temporal features, This represents visual features containing time-series information after being modeled using LSTM. Indicates the first The visual feature vector of a frame; Represents consecutive time frames. The size of the time window;

[0186] S1.34 Multimodal Feature Fusion

[0187] By integrating visual, semantic, and temporal features, a unified multimodal representation is generated.

[0188]

[0189] in, This represents a multimodal feature vector that integrates visual, semantic, and temporal features, used to comprehensively represent the target role; Represents the visual feature vector of the character; A semantic feature vector representing a character; Represents the temporal feature vector of a character; The fusion weights represent visual features and are used to adjust the influence of visual features on the final representation. The fusion weights represent the semantic features and are used to adjust the influence of semantic features on the final representation. The fusion weights represent the temporal features and are used to adjust the influence of temporal features on the final representation.

[0190] S1.4 Result Output

[0191] S1.41 Feature Storage

[0192] The generated character features are stored in a database for subsequent analysis;

[0193] S1.42 Visualization Results

[0194] Output visualizations of segmentation masks and feature embeddings for easy debugging;

[0195] S1.43 Output Format

[0196] Embedded vectors including visual features, semantic features, and temporal features.

[0197] This step, through multimodal data input and preprocessing, target separation, extraction of visual and semantic features, and multimodal feature fusion, enables accurate segmentation and efficient feature generation of character targets, providing strong support for subsequent character infringement detection. This step has significant advantages in occlusion, dynamic scenes, and the combination of multimodal features, and is applicable to infringement analysis tasks in various complex scenarios.

[0198] Furthermore, in this embodiment, to achieve role feature correspondence and matching, refer to... Figure 4 S2 specifically includes

[0199] S2.1 Feature Representation and Storage

[0200] S2.11 Building a Copyright Content Library ( Figure 2 (shown), including

[0201] S2.111 Character Target Separation: Extracting character instances from copyrighted content;

[0202] S2.112 Feature Extraction: Perform feature extraction on the extracted role instances, including...

[0203] Visual features: Extract key image information of the character (such as outline, color, texture, key points, etc.) and generate visual feature vectors;

[0204] Semantic features: Analyze the character's text description, semantic tags, behavioral features, plot connections, and other information, and generate semantic feature vectors;

[0205] Multimodal feature fusion: Integrating character features across different modalities to form a unified, multi-dimensional character feature representation. ;

[0206]

[0207] in: Indicates visual features; Represents semantic features; Indicates time-series characteristics (optional);

[0208] S2.113 Standardization and Storage

[0209] The extracted character features are stored in the copyright content library according to a unified standard; each character instance corresponds to a unique identifier - data tag - in the library, and is associated with its visual features, semantic features and related metadata (such as copyright holder, registration time, copyright type, etc.).

[0210] S2.114 Dynamic Updates and Optimizations

[0211] The copyright content library supports dynamic updates, allowing the addition of new character data and regular optimization of the feature representation of existing data to maintain the comprehensiveness and accuracy of the content library;

[0212] The implementation methods for role target separation and feature extraction in S2.11 of this embodiment for constructing the copyright content library can refer to the relevant algorithms in step S1.

[0213] S2.12 Representation of features to be matched

[0214] The modal features are characterized based on the role features output by S1;

[0215] Single-modal features: such as visual features only ;

[0216] Multimodal features: such as visual features and semantic features ;

[0217] S2.2 Feature Preprocessing

[0218] S2.21 Feature Alignment

[0219] For a single modal feature, calculate the similarity between that modal feature and the corresponding feature in the copyright content library; for example, if only visual features are available. At that time, calculate its visual features compared with those in the copyright database. Similarity;

[0220] For multimodal features, the similarity between each modal feature and the corresponding feature in the copyright content library is calculated separately;

[0221] S2.22 Feature Normalization

[0222] All modal features are normalized to ensure that the feature vectors have a consistent range;

[0223] This embodiment uses L2 normalization: ;

[0224] S2.3 Matching Search Algorithm

[0225] S2.31 Intramodal Matching

[0226] Visual feature matching: Calculate cosine similarity or other similarities. The similarity score represents the visual features.

[0227] Cosine similarity:

[0228] Or Euclidean distance (anti-similarity):

[0229] Or structural similarity (SSIM): ;

[0230] Semantic feature matching: Calculate cosine similarity or other similarities. The similarity score represents the semantic features.

[0231] Calculate cosine similarity:

[0232] Or Jaccard similarity:

[0233] Word Shift Distance (WMD): Calculates the overall semantic distance between two text embeddings;

[0234] Temporal feature matching: Calculate the dynamic time warping (DTW) distance or other metrics of time series features. The similarity score represents the temporal features.

[0235] Calculate Dynamic Time Warping (DTW):

[0236] Or slice similarity: Divide the time series into multiple segments, calculate the cosine similarity of each segment and take the average value;

[0237] S2.32 Intermodal Fusion

[0238] Calculate the total similarity based on the weights of the character feature modalities:

[0239]

[0240] in: This represents the overall similarity score; , , The weights for visual, semantic, and temporal features can be set according to the importance of the task. If a certain modality feature is missing, the corresponding weight is 0.

[0241] S2.33 Multimodal Search Optimization

[0242] This embodiment uses an ANN (Approximate Nearest Neighbor) algorithm, such as FAISS, to perform fast nearest neighbor search on high-dimensional features;

[0243] S2.4 Matching Result Filtering

[0244] S2.41 Set threshold

[0245] Only retain role matching results with a similarity score higher than a set threshold. The threshold can be adjusted according to actual application needs.

[0246] S2.42 Candidate Role Ranking

[0247] Based on total similarity Sort the matching results and output the Top-K candidate roles and feature sets;

[0248]

[0249] Each matched role feature contains multiple modalities. , Multimodal embeddings representing character characteristics.

[0250] Furthermore, in this embodiment, to achieve character feature similarity analysis, refer to... Figure 5 S3 specifically includes

[0251] The Top-K roles and feature sets, as well as the total similarity, are obtained based on S2 matching. Conduct character feature similarity analysis;

[0252] S3.1 Advanced Analysis

[0253] S3.11 Multi-level Validation

[0254] For matched characters with high visual similarity, further examine semantic and temporal features;

[0255] S3.12 Conflict Resolution

[0256] If there is a conflict in the similarity scores between modalities, the result of the modality with the higher weight shall be given priority.

[0257] S3.2 Output Results

[0258] like Higher than the set threshold If a character is identified as a potential infringer, and the similarity score of at least one modality exceeds its specificity threshold, then the character is identified as a potential infringer.

[0259] Optionally, a report can be generated from the character feature similarity analysis results, including the overall similarity score, the similarity score for each modality, comparative evidence (such as visual matching diagrams and text comparison results), and a possible Top-N list of infringing characters.

[0260] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0261] Another embodiment of the present invention relates to a character copyright infringement detection system based on visual and semantic features, including a database and

[0262] The character target separation and feature extraction module is used to separate the character target from the input content to be detected and extract the character features, which include at least one of visual features, semantic features, and temporal features.

[0263] The character feature mapping and matching module is used to map and match the extracted character features with the registered copyright character features in the copyright content library of the database in order to find similar characters;

[0264] The character feature similarity analysis module is used to determine a character as a potential infringer when the similarity between the extracted character and the matched character exceeds a preset threshold.

[0265] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0266] Another embodiment of the present invention relates to an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the character copyright infringement detection method based on visual and semantic features. Alternatively, when the processor executes the computer program, it implements the functions of each module in the above-described embodiment of the character copyright infringement detection system based on visual and semantic features.

[0267] For example, a computer program can be divided into one or more modules / units, one or more of which are stored in memory and executed by a processor to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in an electronic device.

[0268] The electronic device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The electronic device may include, but is not limited to, processors and memory; for example, it may also include input / output devices, network access devices, buses, etc.

[0269] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0270] The memory can be an internal storage unit of an electronic device, such as a hard drive or RAM. It can also be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory can include both internal and external storage units. The memory is used to store computer programs and other programs and data required by the electronic device, and can also be used to temporarily store data that has been output or will be output.

[0271] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the character copyright infringement detection method based on visual and semantic features. Alternatively, when executed by a processor, the computer program implements the functions of each module in the above-described embodiment of the character copyright infringement detection system based on visual and semantic features.

[0272] Computer programs include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0273] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting copyright infringement of characters based on visual and semantic features, characterized in that: include S1 Character Target Separation and Feature Extraction Input the content to be detected, separate the character target, and extract the character features. The character features include at least one of visual features, semantic features, and temporal features. The temporal features describe the dynamic changes of the character in the video. S2 Character Traits Correspondence and Matching The extracted character features are matched with existing character features in the copyright content library to find similar characters; S3 Character Feature Similarity Analysis When the similarity between the extracted character and the matched character exceeds a preset threshold, it can be identified as a potential infringing character; To achieve character feature correspondence and matching, S2 includes S2.1 Feature Representation and Storage S2.11 Building a Copyright Content Library S2.12 Representation of features to be matched The modal features are characterized based on the role features output by S1; S2.2 Feature Preprocessing S2.21 Feature Alignment Calculate the similarity between this modal feature and the corresponding feature in the copyright content library; S2.22 Feature Normalization All modal features are normalized to ensure that the feature vectors have a consistent range; S2.3 Matching Search Algorithm S2.31 Intramodal Matching For visual feature matching: calculate cosine similarity or other similarities; For semantic feature matching: calculate cosine similarity or other similarities; For time series feature matching: calculate the dynamic time-warped distance or other metrics of the time series features: S2.32 Intermodal Fusion Calculate the total similarity based on the weights of the character feature modalities: in: This represents the overall similarity score; , , The weights are assigned to visual, semantic, and temporal features; if a modality is missing, the corresponding weight is 0. S2.4 Matching Result Filtering S2.41 Set threshold Only retain character matching results with a similarity score higher than a set threshold; S2.42 Candidate Role Ranking Based on total similarity Sort the matching results and output the Top-K candidate roles and feature sets; To achieve character feature similarity analysis, S3 includes The Top-K character feature set and total similarity are obtained based on S2 matching. Conduct character feature similarity analysis; S3.1 Advanced Analysis S3.11 Multi-level Validation For matched characters with high visual similarity, further examine semantic and temporal features; S3.12 Conflict Resolution If there is a conflict in the similarity scores between modalities, the result of the modality with the higher weight shall be given priority. S3.2 Output Results like Higher than the set threshold If a character is identified as a potential infringer, and the similarity score of at least one modality exceeds its specificity threshold, then the character is identified as a potential infringer.

2. The method for detecting copyright infringement of characters according to claim 1, characterized in that: To achieve character target separation and feature extraction, S1 includes S1.1 Multimodal Data Collection and Preprocessing Preprocess the input multimodal data to be detected; S1.11 Multimodal Data Reception The multimodal data includes images, videos, and text; S1.12 Video Data Frame Decomposition The input video is decoded into consecutive frames at a specified frame rate, and each frame is timestamped. S1.13 Image and Video Standardization Image standardization is performed on each input image / frame, including resolution adjustment, color normalization, and noise reduction. S1.14 Text Data Processing Generate text data that can be used for semantic feature extraction; S1.2 Role-Target Separation S1.21 Instance Segmentation A convolutional neural network model is used to generate segmentation masks for each image / frame, and cross-frame target tracking is optimized by temporal series consistency loss. S1.22 Enhanced Dynamic Hints Combining point cues and mask cues improves segmentation accuracy in complex scenes; The formula for the segmentation weight is: in: Indicates the splitting weight; Indicates the similarity between target features and background features; This indicates the region weights generated by the prompt; and Weighting coefficients for features and regions; S1.23 Occlusion and Overlap Optimization By introducing a dynamic memory module and optical flow estimation, the model's continuous target tracking capability is improved. S1.3 Character Feature Extraction Multimodal representations are generated by combining visual and semantic features for subsequent feature matching and similarity analysis; S1.31 Visual Feature Extraction High-level features of image data, including texture, edges, and shape, are extracted using a convolutional neural network model, and visual feature vectors of characters are generated. S1.32 Semantic Feature Extraction Generate semantic feature vectors from text data using a semantic representation model; S1.33 Temporal Feature Enhancement By modeling continuous frames using an LSTM model, the temporal dependence features of dynamic characters can be captured. S1.34 Multimodal Feature Fusion By integrating visual, semantic, and temporal features, a unified multimodal representation is generated. in, This represents a multimodal feature vector that integrates visual, semantic, and temporal features, used to comprehensively represent the target role; Represents the visual feature vector of the character; A semantic feature vector representing a character; Represents the temporal feature vector of a character; The fusion weights represent visual features and are used to adjust the influence of visual features on the final representation. The fusion weights represent the semantic features and are used to adjust the influence of semantic features on the final representation. The fusion weights represent the temporal features and are used to adjust the influence of temporal features on the final representation. S1.4 Result Output The generated character features are stored in the database. The character features are embedded vectors that include visual features, semantic features, and temporal features.

3. The method for detecting copyright infringement of characters according to claim 1, characterized in that: The S2.11 construction of the copyright content library includes S2.111 Character Target Separation: Extracting character instances from copyrighted content; S2.112 Feature Extraction: Perform feature extraction on the extracted role instances, including... Visual features: Extract key image information of the character and generate visual feature vectors; Semantic features: Analyze the character's text description, semantic tags, behavioral features, plot connections, and other information, and generate semantic feature vectors; Multimodal feature fusion: Integrating character features across different modalities to form a unified, multi-dimensional character feature representation. ; in: Indicates visual features; Represents semantic features; Indicates temporal characteristics; S2.113 Standardization and Storage The extracted character features are stored in the copyright content library according to a unified standard; each character instance corresponds to a unique identifier in the library, which is associated with its visual features, semantic features and related metadata. S2.114 Dynamic Updates and Optimizations The copyright content library supports dynamic updates, allowing the addition of new character data and regular optimization of the feature representation of existing data to maintain the comprehensiveness and accuracy of the content library.

4. The method for detecting copyright infringement of characters according to claim 1, characterized in that: The results of the character feature similarity analysis will be used to generate a report, including the overall similarity score, the similarity score of each modality, comparative evidence, and a list of possible Top-N infringing characters.

5. A character copyright infringement detection system based on visual and semantic features, characterized in that: The system is used to implement the character copyright infringement detection method according to any one of claims 1 to 4, including a database and The character target separation and feature extraction module is used to separate the character target from the input content to be detected and extract the character features. The character features include at least one of visual features, semantic features, and temporal features. The temporal features are: Describe the dynamic changes of the characters in the video; The character feature mapping and matching module is used to map and match the extracted character features with the registered copyright character features in the copyright content library in order to find similar characters; The character feature similarity analysis module is used to determine a character as a potential infringer when the similarity between the extracted character and the matched character exceeds a preset threshold.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the detection method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the detection method as described in any one of claims 1 to 4.

Images