A character image infringement early warning method, device, equipment and medium

Abstract

The application discloses a character image infringement early warning method and device, equipment and medium, and relates to the technical field of computer vision, which comprises the following steps: frame extraction is performed on a to-be-detected video, character detection is performed on the extracted target frame, and detection information is generated; a to-be-detected vector and a to-be-detected character visual constituent element are generated based on the detection information, the to-be-detected vector is matched with a registered character, and a candidate registered character is determined; a target score is determined based on the registered character vector and the to-be-detected vector, and an element combination constraint score of the to-be-detected character image and the candidate registered character is determined; a comprehensive credibility score is determined based on the target score and the element combination constraint score, the comprehensive credibility score is aggregated, and an aggregated score is obtained; a candidate evidence package is constructed according to the aggregated score, the target score, the element combination constraint score and a registered reference graph; the candidate evidence package is reviewed, and an infringement early warning result is determined according to the aggregated score and a structured review result. The character image after modification can be accurately identified.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus, device, and medium for early warning of character image infringement. Background Technology

[0002] With the rapid growth of short dramas, derivative videos, animated adaptations, and generative content, well-known character images are being modified and used in increasingly diverse ways. The well-known character images or intellectual property (IP) mentioned here refer to character images with stable visual identifiers in film, short dramas, animation, comics, games, and other content, including live-action characters, virtual characters, comic book characters, and characters modified through derivative works. Infringers may not directly copy original video clips, but rather retain the character's core visual characteristics, such as hairstyle outlines, clothing color schemes, headdresses, props, or signature poses, while altering age, makeup, clothing details, camera backgrounds, or image quality, making traditional video copy detection methods difficult to detect.

[0003] Existing technologies for video copyright detection typically include methods such as keyframe extraction, image fingerprinting, depth visual vector retrieval, continuous similar frame identification, face recognition, character trajectory clustering, and cross-shot character atlas construction. These methods are suitable for detecting scenarios where original video clips have been copied, cropped, compressed, watermarked, or sped up, or for identifying the facial identities of the same actor. However, their stability and interpretability are insufficient when character images have been reshot, re-costume, re-created, cartoonized, or partially retained as iconic elements.

[0004] Specifically, existing solutions can be summarized into the following categories. The first category is copyright detection solutions based on video keyframes or image vectors. These solutions extract frames from both the copyrighted video and the video to be detected, generate visual vectors or image fingerprints, calculate cosine similarity or nearest neighbor distance, and determine whether infringement is suspected based on the number of similar frames or the proportion of consecutive frames. The second category is video character detection solutions based on face recognition. These solutions detect and extract features from faces appearing in the video, compare them with a sample face database, and output suspected characters based on matching degree and frequency of occurrence. The third category is character recognition solutions based on scene context, character location, and trajectory maps. These solutions segment the video by shot or scene, identify character trajectories, and construct cross-shot character maps based on appearance similarity, spatial relationships, co-occurrence frequency, and behavioral continuity, thereby clustering or identifying the same character. The fourth category is content detection solutions based on multimodal weighted similarity. These solutions extract features from images, text, audio, or other modalities, learn or set different modal weights, calculate comprehensive similarity, and then issue warnings. The common feature of the above schemes is that the comparison objects are usually complete frames, complete characters, faces of the same actor, video clips, or cross-shot trajectories, rather than breaking down the character image into visually identifiable elements and their combinations. However, they have drawbacks such as lacking element-level expression of the character image, being unable to effectively handle character transformation, and being prone to bias towards the main character or frequently appearing objects in the video.

[0005] Therefore, how to reliably identify the substantial visual similarity between a character and a registered well-known character after the character has been modified by changing clothes, hairstyle, color, comic book style, low-resolution compression, partial occlusion, short drama adaptation, or AI redrawing, avoiding missed detections and false detections, and providing an interpretable chain of evidence is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method, device, equipment, and medium for early warning of character image infringement, which can reliably identify the substantial similarity between a character and a registered character image after the character has been dressed up, recolored, cartoonized, compressed to low resolution, partially obscured, or modified through secondary creation, thereby reducing missed detections and false detections. The specific solution is as follows: Firstly, this application discloses a method for early warning of character image infringement, including: The video to be detected is frame-by-frame extracted, and human detection is performed on the extracted target frames to determine the human cropping region and generate the detection information corresponding to the human cropping region. Based on the detection information, a detection vector and visual constituent elements of the character to be detected in the video to be detected are generated. The detection vector is then matched with registered characters in a preset registered character library to determine candidate registered characters. The target score is determined based on the similarity between the registered character vector of the candidate registered character and the vector to be detected. The element combination constraint score between the image of the character to be detected and the candidate registered character is determined based on the preset visual composition element map corresponding to the candidate registered character and the visual composition elements of the character to be detected. A comprehensive credibility score is determined based on the target score and the element combination constraint score. Multiple comprehensive credibility scores corresponding to the same candidate registration role are aggregated to obtain an aggregate score. Based on the aggregate score, the target score, the element combination constraint score, and the registration reference graph corresponding to the candidate registered role, construct the candidate evidence package corresponding to the candidate registered role; The candidate evidence package is reviewed to obtain the corresponding structured review results. Based on the aggregated score and the structured review results, the infringement warning result corresponding to the character image to be detected is determined.

[0007] Optionally, before performing frame extraction on the video to be detected, the method further includes: Receive the registration reference image or registration reference video of the character to be protected, and configure metadata including character identifier, character name, work name, reference source and rights holder information; Perform person detection on the registration reference image or registration reference video to determine the target person cropping area; The cropped area of ​​the target character is enhanced, and a target vector corresponding to the character to be protected is generated based on the enhanced area. The enhancement process includes any one or a combination of several of the following: scale transformation, cropping perturbation, brightness perturbation, contrast perturbation, sharpness perturbation, slight rotation, and local occlusion simulation. The target vector includes a global visual vector, a facial vector, and a head region vector. A preset registered role library is constructed based on the target vector corresponding to the role to be protected, the registration reference image or registration reference video, the cropping frame, and the metadata. The visual constituent elements of the character to be protected are extracted by a preset element extraction model or manual annotation, and element nodes are determined based on the visual constituent elements of the character to be protected. The visual constituent elements of the character include any one or a combination of several of the following: facial contour, eyebrow, eye, nose and mouth structure, hairstyle contour, hair color, clothing silhouette, clothing color scheme, headdress, accessories, props, action posture and body proportion. The element nodes include element type, element features, visibility statistics, stability weight, discrimination weight and conflict threshold. The visual constituent elements of the character to be protected are defined as nodes, and the combination constraints, substitutability relationships, missing tolerance relationships, or conflict relationships between the visual constituent elements of the character to be protected are defined as edges, so as to construct a preset visual constituent element map.

[0008] Optionally, determining the element combination constraint score between the image of the character to be detected and the candidate registered character based on the preset visual composition element map corresponding to the candidate registered character and the visual composition elements of the character to be detected includes: The visibility and quality of each visual element in the visual elements of the character to be detected are determined, and the stability and distinguishability of the corresponding visual elements in the preset visual element map are obtained. The visibility is used to characterize the observability of the visual elements in the detection information. The quality is used to characterize the sharpness, brightness, compression distortion, motion blur, and effective pixel ratio of the visual element region. The stability is used to characterize the degree to which the visual elements remain stable in different registration reference images, different plot stages, or different style versions. The distinguishability is used to characterize the ability of the visual elements to distinguish the characters to be protected. Determine the cosine similarity between the element feature vector of each character visual element in the visual elements of the character to be detected and the element feature vector of the corresponding character visual element in the preset visual elements map, and determine the cosine similarity as the single element similarity. The product of the visibility, quality, stability and discriminability is determined as a weighting coefficient. Based on the weighting coefficient, the single-element similarity of the visual components of each character is weighted and summed to obtain the element node score. The element combination constraint score is determined based on the element node score.

[0009] Optionally, determining the element combination constraint score based on the element node score includes: Determine whether the visibility, quality, and single-element similarity of each character's visual elements corresponding to each combination constraint in the preset visual element map meet the first threshold. If all conditions are met, the target product between visibility, quality, and single-element similarity is determined, and the minimum value among the target products of the visual constituent elements of each role corresponding to the combined constraints is determined as the combined satisfaction. The combined satisfaction is weighted and summed based on the weights of each combined constraint to obtain the combined enhancement score; If there is a target character visual element in the visual elements of the character to be detected that meets the second threshold in both visibility and quality, but whose single element similarity is less than the preset conflict threshold, then the ratio of the sum of the conflict weights corresponding to the target character visual element to the sum of the conflict weights corresponding to each character visual element in the visual elements of the character to be detected is determined as the conflict penalty score. The element node score and the combined enhancement score are weighted based on the first configurable weight to obtain a weighted result. The difference between the weighted result and the conflict penalty score after being weighted according to the second configurable weight is determined, and the element combined constraint score is obtained by limiting the difference to a first preset value range.

[0010] Optionally, the target score includes a global visual score, a face portion, and a head portion; Accordingly, determining the comprehensive credibility score based on the target score and the element combination constraint score includes: The global visual score in the target score is determined as the basic visual score; If the face portion in the target score is greater than or equal to a preset face portion threshold, then a first difference between the face portion and the preset face portion threshold is determined, and a first product between the first difference and a preset face gain coefficient is determined. The sum of the basic visual score and the first product is determined as the current target visual score; If the head portion of the target score is greater than or equal to a preset head portion threshold, then a second difference between the head portion and the preset head portion threshold is determined, and a second product between the second difference and a preset head gain coefficient is determined. Update the sum between the current target visual score and the second product to the current target visual score; The current target visual score and the element combination constraint score are weighted and fused, and the comprehensive credibility score is determined by controlling the corresponding fusion result within a second preset value range.

[0011] Optionally, the aggregation of multiple comprehensive credibility scores corresponding to the same candidate registered role to obtain an aggregate score includes: The information stability weights of each detection information corresponding to the candidate registered roles are determined, and the top target number of detection information with the highest information stability weights are selected. The information stability weights are determined based on at least one of the following: character detection confidence, image quality, consistency between adjacent frames, whether they belong to the same tracking trajectory, whether they contain core elements, and whether they are in the main position of the image. The detection information includes a character cropped image, original frame, timestamp, frame number, character detection box, character detection confidence, image quality index, and cropping source information. Based on the aforementioned information stability weights, the comprehensive credibility scores corresponding to the number of detection information items for the previous target are weighted and summed to obtain the aggregate score.

[0012] Optionally, the step of reviewing the candidate evidence package to obtain a corresponding structured review result, and determining the infringement warning result corresponding to the character image to be detected based on the aggregated score and the structured review result, includes: The candidate evidence package is input into a preset review model to obtain the corresponding structured review results. The structured review results include review conclusions, review scores, confidence levels, reasons for uncertainty, a list of matching elements, and a list of conflicting elements. The review conclusions are: match, pending review, or no match. If the aggregate score is greater than or equal to the preset hit threshold, and the review conclusion is a match, then the infringement warning result corresponding to the character image to be detected is determined as a hit; If the review conclusion is pending review, or if the aggregate score is greater than or equal to the preset manual review threshold and less than the preset hit threshold, then the infringement warning result corresponding to the character image to be detected will be determined as manual review. If the aggregate score is less than the preset manual review threshold, or the review conclusion is a mismatch, then the infringement warning result corresponding to the character image to be detected will be determined as a miss. Otherwise, the infringement warning result corresponding to the character image to be detected will be determined as a miss.

[0013] Secondly, this application discloses a character image infringement early warning device, including: The detection information generation module is used to extract frames from the video to be detected, perform human detection on the extracted target frames to determine the human cropping area, and generate detection information corresponding to the human cropping area. The matching module is used to generate a detection vector and visual constituent elements of the character to be detected in the video to be detected based on the detection information, and to match the detection vector with the registered characters in the preset registered character library to determine the candidate registered characters. The element combination constraint score determination module is used to determine the target score based on the similarity between the registered role vector of the candidate registered role and the vector to be detected, and to determine the element combination constraint score between the image of the character to be detected and the candidate registered role according to the preset visual composition element map corresponding to the candidate registered role and the visual composition elements of the character to be detected. The aggregation score determination module is used to determine a comprehensive credibility score based on the target score and the element combination constraint score, and to aggregate multiple comprehensive credibility scores corresponding to the same candidate registration role to obtain an aggregation score; The candidate evidence package construction module is used to construct the candidate evidence package corresponding to the candidate registration role based on the aggregate score, the target score, the element combination constraint score and the registration reference graph corresponding to the candidate registration role; The infringement warning result determination module is used to review the candidate evidence package, obtain the corresponding structured review result, and determine the infringement warning result corresponding to the character image to be detected based on the aggregation score and the structured review result.

[0014] Thirdly, this application discloses an electronic device, including: Memory, used to store computer programs; A processor is used to execute computer programs to implement the steps of the aforementioned character image infringement early warning method.

[0015] Fourthly, this application discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the aforementioned character image infringement warning method.

[0016] This application first extracts frames from the video to be tested, performs character detection on the extracted target frames to determine the character cropping region, and generates detection information corresponding to the character cropping region; based on the detection information, it generates a target vector and visual constituent elements corresponding to the character image in the video to be tested, and matches the target vector with registered characters in a preset registered character library to determine candidate registered characters; it determines a target score based on the similarity between the registered character vector of the candidate registered character and the target vector, and determines the target score based on the preset visual constituent element map corresponding to the candidate registered character and the visual constituent elements of the character to be tested. The following steps are taken: 1. Analyze the element combination constraint score between the character image to be detected and the candidate registered character; 2. Determine a comprehensive credibility score based on the target score and the element combination constraint score; 3. Aggregate multiple comprehensive credibility scores corresponding to the same candidate registered character to obtain an aggregated score; 4. Construct a candidate evidence package corresponding to the candidate registered character based on the aggregated score, the target score, the element combination constraint score, and the registration reference graph corresponding to the candidate registered character; 5. Review the candidate evidence package to obtain a corresponding structured review result; 6. Determine the infringement warning result corresponding to the character image to be detected based on the aggregated score and the structured review result. As can be seen, this application achieves efficient, accurate, and interpretable infringement warning by constructing a multi-dimensional character image comparison mechanism: On the one hand, it quickly identifies candidate objects by matching the similarity between the vector to be detected and the vector of the registered character, and calculates the element combination constraint score by combining the visual composition element map, thereby taking into account both the overall style and the compliance judgment of local elements, significantly improving the accuracy of detection and anti-interference ability; on the other hand, by aggregating the comprehensive credibility scores of multiple frames and generating a candidate evidence package containing review information, it avoids the randomness of single-frame misjudgment and provides a structured basis for manual review. Finally, it combines the automated quantitative scoring with the qualitative review results to form a reliable infringement judgment output, thereby ensuring the efficiency of the warning, effectively reducing the risk of false alarms and false negatives, and supporting the closed-loop management of subsequent rights protection or compliance handling. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This application discloses a flowchart of a character image infringement early warning method. Figure 2 This is a schematic diagram of a character visual composition element map disclosed in this application; Figure 3 This application discloses a flowchart of a specific character image infringement early warning method. Figure 4 This is a schematic diagram of the structure of a character image infringement early warning device disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Existing technologies typically compare complete frames, complete characters, the same actor's face, video clips, or cross-camera trajectories, rather than breaking down a character's image into visually recognizable elements and their combinations. Therefore, at least the following drawbacks exist: First, there is a lack of element-level representation of character image. Existing methods usually only provide overall visual similarity or facial similarity, which is difficult to describe fine-grained situations such as similar hairstyles but different clothing, highly consistent headdress and clothing color schemes but low facial clarity, and similar clothes but obvious conflicts in hairstyle or head outline.

[0021] Secondly, it cannot effectively handle modified characters. For scenarios such as changing clothes, hair color, age changes, comic book style, low-resolution compression, and side profile obscuring, single vector or face comparison is prone to judging truly similar modified characters as dissimilar, or misjudging non-target characters with similar backgrounds and clothing colors as similar.

[0022] Third, there is a lack of combinatorial constraints and conflict penalties. The identification of certain roles is not determined by a single element, but by the combined relationships of multiple elements; existing methods lack explicit modeling for situations where multiple weak elements combine to form strong evidence, or where a conflict in a core element leads to a reduction in weight.

[0023] Fourth, the interpretability of the evidence chain is insufficient. General similarity systems can only return a number of candidates and scores, and cannot clearly indicate which elements are similar, which elements conflict, which frames support a hit, and which frames require manual review.

[0024] Fifth, it tends to favor the main character or frequently appearing objects in the video. When performing overall frame retrieval or video-level vector aggregation, frequently appearing characters, backgrounds, or camera styles may dominate the results, causing genuinely suspected characters to be overlooked due to their short appearance time.

[0025] To address the aforementioned technical issues, this application discloses a method, device, equipment, and medium for early warning of character image infringement. This method can reliably identify the substantial similarity between a character and a registered character image after the character has been dressed up, recolored, cartoonized, compressed to low resolution, partially obscured, or modified through secondary creation, thereby reducing missed detections and false detections.

[0026] To facilitate understanding, some terms involved in this invention will be explained first. Visual constituent elements refer to the observable visual parts used to constitute character image recognition, including but not limited to facial contours, eyebrows, eyes, nose, mouth, hairstyle, hair color, clothing silhouette, clothing color scheme, headdress, accessories, props, posture, and body proportions. Element map refers to a structured representation of a character image with visual constituent elements as nodes and stable combination relationships, strong and weak constraint relationships, substitutable relationships, and conflict relationships between elements as edges. Combination constraint refers to a constraint that has strong recognizability only when multiple elements appear simultaneously or maintain a specific relationship; for example, when a specific hairstyle silhouette, headdress, and clothing color scheme are jointly satisfied, the character similarity is significantly enhanced. Visibility score refers to the degree to which an element can be reliably observed in query video evidence, affected by factors such as occlusion, side profile, blurriness, and image size. Quality score refers to the impact of evidence image clarity, compression distortion, brightness, and the size of the character area on the reliability of element judgment. An evidence package refers to a collection of cropped images of several characters, original frames, timestamps, candidate reference images, element scores, and intermediate system results selected from the video to be detected, all for a given candidate IP. The image-text comparison visual model refers to a model used to convert images into visual vectors and calculate similarity; it can be implemented using a Contrastive Language-Image Pre-training (CLIP) model or other equivalent visual semantic pre-training models. The verification model refers to a model capable of simultaneously receiving image and text evidence and outputting structured conclusions, such as a visual language model. In this scheme, it is used for evidence verification rather than directly providing legal conclusions. Uncertainty reasons refer to the categories of reasons given by the system for evidence that failed to directly match or exclude it, such as low quality, local perspective, ambiguous identity features, or mixed evidence.

[0027] Furthermore, the solution of this invention is generally divided into a registration phase and a detection phase. The registration phase is used to establish a preset registered character library and a preset visual composition element map; the detection phase is used to extract query evidence from the video to be detected and complete candidate recall, element constraint scoring, multi-frame aggregation, structured review, and early warning output. As shown in Table 1, the system can generally include a character registration module, a character detection and cropping module, a visual vector module, an element extraction module, an element map construction module, a candidate recall module, a combination constraint scoring module, an evidence package construction module, a structured review module, and an early warning output module. Each module can be deployed on the same server, edge device, or cloud platform, or can be completed collaboratively by multiple service modules through interfaces; each module can be implemented by software programs, or by hardware accelerators, graphics processors, neural network processors, or cloud inference services, and can be updated independently to adapt to different types of character images such as animated characters, comic characters, game characters, or derivative characters.

[0028] Table 1. Functional Description of Each Module of the Character Image Infringement Early Warning System

[0029] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a method for early warning of character image infringement, including: Step S11: Extract frames from the video to be detected, perform human detection on the extracted target frames to determine the human cropping area, and generate detection information corresponding to the human cropping area.

[0030] This embodiment is used to obtain query evidence from a video to be detected. Specifically, frames of the video to be detected can be extracted according to a preset time interval, shot change, keyframe strategy, or motion change strategy to obtain a query frame sequence; then, multi-person detection is performed on each query frame, retaining a maximum of a preset number of person cropping regions, and generating detection information for each person cropping region. The detection information may include person cropping image, original frame, timestamp, frame number, person detection box, person detection confidence, image quality index, and cropping source information.

[0031] As for the various implementation methods for this step, the frame extraction strategy can adopt any one or a combination of the following: The first is equal-interval frame extraction, such as extracting one frame every half second or every several frames, which is simple to implement and provides uniform coverage; the second is frame extraction based on shot boundary detection, which first detects shot transition points and then extracts representative frames within each shot, thereby reducing redundancy within the same shot; the third is adaptive frame extraction based on motion amplitude, which encrypts frames in segments with rapid motion or large changes in image, and sparsely extracts frames in static segments; the fourth is keyframe extraction, which directly utilizes keyframes in video encoding. Person detection can be implemented using any of the following: object detection model, instance segmentation model, human keypoint model, manual selection, or video tracking bounding box. For example, for a one-minute short drama segment, the system extracts 120 frames at two frames per second, retaining a maximum of three character cropping regions in each frame, thus generating a maximum of 360 detection information.

[0032] In this way, by performing multi-character detection on each frame and retaining multiple character cropping regions, the comparison unit is refined from the entire frame to individual characters. Since the background and subtitles are excluded from the character cropping regions, the false similarity caused by background similarity is significantly weakened; and since multiple characters are retained per frame instead of a single character, supporting characters who appear briefly or occupy a small portion of the screen are also included in the candidate evidence, thereby reducing the risk of missing detection of characters who appear briefly but have obvious IP characteristics.

[0033] In addition, before extracting frames from the video to be detected, a registration reference image or video of the character to be protected is received, and metadata including character identifier, character name, work name, reference source, and rights holder information is configured; character detection is performed on the registration reference image or video to determine the target character cropping region; the target character cropping region is enhanced, and a target vector corresponding to the character to be protected is generated based on the enhanced region; the enhancement process includes any one or a combination of scale transformation, cropping perturbation, brightness perturbation, contrast perturbation, sharpness perturbation, slight rotation, and local occlusion simulation; the target vector includes a global visual vector, a facial vector, and a head region vector; based on the target vector corresponding to the character to be protected, the registration reference image or video, The cropping box and the metadata are used to construct a preset registered role library; the visual constituent elements of the role to be protected are extracted by a preset element extraction model or manual annotation, and element nodes are determined based on the visual constituent elements of the role to be protected; the visual constituent elements of the role include any one or a combination of several of the following: facial contour, eyebrow, eye, nose and mouth structure, hairstyle contour, hair color, clothing silhouette, clothing color scheme, headdress, accessories, props, action posture and body proportion; the element nodes include element type, element features, visibility statistics, stability weight, distinguishability weight and conflict threshold; the visual constituent elements of the role to be protected are determined as nodes, and the combination constraints, substitutability relationships, missing tolerance relationships or conflict relationships between the visual constituent elements of the role to be protected are determined as edges, so as to construct a preset visual constituent element graph.

[0034] Step S12: Based on the detection information, generate the detection vector and visual constituent elements of the character to be detected in the video to be detected, and match the detection vector with the registered characters in the preset registered character library to determine the candidate registered characters.

[0035] In this embodiment, a detection vector is generated for the cropped area of ​​the character corresponding to each detection information. The detection vector may include one or more of the following: global visual vector, facial vector, and head region vector. At the same time, the visual constituent elements of the character to be detected are extracted through element extraction. Then, based on the similarity between the detection vector and the registered character vectors of registered characters in the preset registered character library, several registered characters with the highest similarity are recalled as candidate registered characters.

[0036] As for the various implementation methods of this step, candidate recall can be accomplished solely by global visual vectors, or it can be achieved by fusing facial vectors, head region vectors, or element summary vectors. For example, for animated or comic book characters, since facial details are unstable or even unavailable, the weight of facial vector recall can be reduced, while the recall of head region vectors and element summary vectors can be strengthened. Vector retrieval can be implemented using an approximate nearest neighbor index to quickly return candidates from a large-scale registered character database. Global visual vectors can be generated by image-text comparison visual models, or by other image retrieval models, self-supervised visual models, or multimodal embedding models.

[0037] Step S13: Determine the target score based on the similarity between the registered role vector of the candidate registered role and the vector to be detected. Determine the element combination constraint score between the image of the character to be detected and the candidate registered role based on the preset visual composition element map corresponding to the candidate registered role and the visual composition elements of the character to be detected.

[0038] In this embodiment, a target score is determined based on the similarity between the registered role vector of the candidate registered role and the vector to be detected. The target score is used to characterize the similarity at the overall vector level, and may include the global visual score, the face part, and the head part; the element combination constraint score is used to characterize the similarity at the level of the visual constituent elements of the role, and comprehensively considers element node similarity, element visibility, image quality, element distinguishability, combination constraints, and conflict constraints.

[0039] When determining the element combination constraint score between the image of the character to be detected and the candidate registered character based on the preset visual element map corresponding to the candidate registered character and the visual element of the character to be detected, the visibility and quality of each character visual element in the visual element of the character to be detected are determined, and the stability and distinguishability of the corresponding character visual elements in the preset visual element map are obtained; the visibility is used to characterize the observability of the character visual element in the detection information; the quality is used to characterize the sharpness, brightness, compression distortion, motion blur and effective pixel ratio of the character visual element area; the stability is used to characterize the character visual element in different registration reference maps, The degree to which stability is maintained across different plot stages or different style versions; the distinguishability is used to characterize the ability of the visual elements of a character to distinguish the character to be protected; the cosine similarity between the element feature vector of each visual element of the character to be detected and the element feature vector of the corresponding visual element in the preset visual element map is determined, and the cosine similarity is determined as the single element similarity; the product of visibility, quality, stability and distinguishability is determined as a weighting coefficient, and the single element similarity of each visual element of the character is weighted and summed based on the weighting coefficient to obtain the element node score; the element combination constraint score is determined based on the element node score. Simultaneously, the element combination constraint score is determined based on the element node score, including: determining whether the visibility, quality, and single-element similarity of each character's visual constituent element corresponding to each combination constraint in the preset visual constituent element map meet a first threshold; if all are met, then the target product between visibility, quality, and single-element similarity is determined, and the minimum value of the target product of each character's visual constituent element corresponding to the combination constraint is determined as the combination satisfaction; the combination satisfaction is weighted and summed based on the weight of each combination constraint to obtain the combination enhancement score; if there is a visual constituent element of the character to be detected with both visibility and quality... If a target character visual element satisfies the second threshold but its single-element similarity is less than a preset conflict threshold, then the ratio of the sum of the conflict weights corresponding to the target character visual element to the sum of the conflict weights corresponding to each character visual element in the detected character visual element is determined as the conflict penalty score. The element node score and the combination enhancement score are weighted based on the first configurable weight to obtain a weighted result. The difference between the weighted result and the conflict penalty score after weighting according to the second configurable weight is determined, and the element combination constraint score is obtained by limiting the difference to a first preset value range.

[0040] In one specific embodiment, let the set of visual constituent elements of a registered role R be: E R={e1,e2,…,e m}; Each element e i At least includes: element type t i Element eigenvector a i Element stable weight w i Element Discrimination d i A set of elements that can be replaced i Element conflict threshold θ i The element's occurrence statistics across different reference maps are also included. The elemental spectrum is represented as G. R =(V R C R ), where V R For a set of element nodes, C R This is a set of element relations, including strong constraints, weak constraints, composite reinforcement relations, substitutable relations, conflict relations, and missing-tolerance relations, as detailed below. Figure 2 As shown. Figure 2 In this system, facial contours, eyebrows, eyes, nose, mouth, hairstyle and hair color, clothing silhouette, clothing color scheme, headwear and accessories, and posture are all nodes; the edges between nodes represent the combination relationships in character recognition. For example, for a certain character, "red clothing" alone is not enough to determine similarity, but the combination of "specific high ponytail hairstyle, gold headwear, red cape, and specific weapon" can trigger combination enhancement. Conversely, if the hairstyle and head structure of the character in the query evidence conflict with the core elements of the registered character, even if the clothing colors are similar, the final score should be reduced. The visual composition element map differs from existing cross-camera character trajectory maps. The nodes in this invention are not the character trajectories in the video, but rather the visual composition elements in the character image; the edges are not co-occurrence relationships or temporal relationships between trajectories, but rather combination constraints, substitutability relationships, and conflict relationships between visual elements. Therefore, this map is more suitable for identifying similar character images that have been modified, re-dressed, re-colored, partially occluded, or re-created.

[0041] Determining the element combination constraint score may include the following sub-steps: Sub-step one involves determining the visibility and quality of each visual element within the visual elements of the character to be detected, and obtaining the stability and discriminability of the corresponding visual elements in the preset visual element atlas. Wherein, visibility v i (q) is used to characterize the observability of the i-th element in the query evidence item q in the detection information, and its value ranges from 0 to 1. Obstruction, back view, side view, small figure, and truncation of the image edge will reduce visibility; quality k i(q) is used to characterize the sharpness, brightness, compression distortion, motion blur, and effective pixel ratio of the i-th element region, with a value ranging from 0 to 1. Low sharpness, severe compression, excessive darkness, overexposure, or strong motion blur will reduce the quality value; stability w i (R) is used to characterize the degree to which the element remains stable across different registration reference images, different story stages, or different style versions; Discrimination d i (R) is used to characterize the ability of an element to distinguish the registered role (the role to be protected) R. The more distinctive the element, the higher the weight. Examples include special hairstyles, headdresses, weapons, clothing color schemes, and iconic props. Common elements such as ordinary black hair and ordinary white shirts have lower distinguishability. As for the various ways to achieve this, visibility and quality can be calculated from image detection results and image statistical features, or they can be output as structured values ​​by a visual language model.

[0042] Sub-step two: Determine the element feature vector x of each visual element of the character to be detected. i (q), and the element feature vector a of the corresponding character visual element in the preset visual element map. i The cosine similarity between (R) is calculated, and the cosine similarity Sim is... i (q,R) is determined as the single-element similarity: Sim i (q,R)=cos(x i (q),a i (R)); The element feature vector can be output by an image encoder, a local visual model, an instance segmentation model, a pose estimation model, a face feature model, a head region model, or a visual language model.

[0043] Sub-step three involves determining the product of visibility, quality, stability, and discriminability as a weighting coefficient, and then weighting and summing the single-element similarity of each character's visual constituent elements based on this weighting coefficient to obtain the element node score. S node (q,R)=[Σ i w i (R)·d i (R)·v i (q)·k i (q)·Sim i (q,R)] / [Σ i w i (R)·d i (R)·v i (q)·k i [(q)+ε]; Here, ε is a minimal constant to prevent the denominator from being zero. The above formula distinguishes between "invisible" and "visible but dissimilar" cases. Invisible or low-quality elements are not directly penalized to avoid misjudgments due to occlusion, side profiles, low resolution, or compression; however, when a core element with high visibility, high quality, and high distinguishability is clearly dissimilar, the final score will be reduced through subsequent conflict penalties. Therefore, under the complex observation conditions of character modification, this embodiment can more accurately reflect the degree of similarity at the character image level compared to a simple averaging scheme.

[0044] Sub-step four involves determining the element combination constraint score based on the element node score. Character portrayal is typically not determined by a single element, but rather by the combination relationships of multiple elements. Let the set of combination constraints be... Each combination constraint It can be represented as: ; in, This represents the subset of elements participating in the combination constraint. Indicates the relation type, This indicates the combined constraint weights. This represents the combined trigger threshold. Determining the element combination constraint score may include the following steps.

[0045] Determine whether the visibility, quality, and single-element similarity of each character's visual elements corresponding to each combination constraint in the preset visual element map meet the first threshold. That is, for each combination constraint... For each element in the array, determine whether its single-element similarity is not less than the corresponding trigger threshold τ. i .

[0046] If all conditions are met, then the target product between visibility, quality, and single-element similarity is determined, and the minimum value among the target products of each visual component element corresponding to the combined constraints is determined as the combined satisfaction. : ; Where 1(·) is an indicator function, which takes the value 1 when the condition inside the parentheses is true, and takes the value 0 otherwise. i (q) represents visibility, k i (q) represents quality. The purpose of using the minimum value instead of the average value is that the combination constraint requires all key elements to reach the corresponding threshold. If any key element is not visible, has too low quality, or is dissimilar, the combination satisfaction will be reduced, thus reflecting the integrity of the combination.

[0047] The combined satisfaction scores are weighted and summed based on the weights of each combined constraint to obtain the combined enhancement score. : ; If there is a target character visual element in the visual elements of the character to be detected that satisfies both the visibility and quality of the second threshold, but whose single-element similarity is less than a preset conflict threshold, then the ratio of the sum of the conflict weights corresponding to the target character visual element to the sum of the conflict weights corresponding to each character visual element in the visual elements of the character to be detected is determined as the conflict penalty score: ; in, Let be the conflict weight of the i-th type of element. and These are the visibility threshold and the quality threshold (i.e., the second threshold), respectively. This is the conflict threshold for the element.

[0048] The element node score and the combination enhancement score are weighted based on a first configurable weight to obtain a weighted result. The difference between the weighted result and the conflict penalty score after weighting according to a second configurable weight is determined, and the element combination constraint score is obtained by limiting the difference to a first preset value range. : ; Where α and β are the first configurable weights, γ is the second configurable weight, and clip(·,0,1) represents the first preset value range that restricts the result to between 0 and 1.

[0049] The above formula enables the system to differentiate between "deficiency tolerance" and "conflict penalty": elements that are not clearly visible are not penalized directly; only elements that are clearly visible and conflict with the core elements of the registered character are penalized. This mechanism reduces false positives caused by low resolution, obscuring, side profiles, makeup changes, or color alterations, while suppressing false positives based solely on similar clothing colors or backgrounds. Therefore, this embodiment can simultaneously suppress two typical types of false positives: one is missed detection due to low resolution, obscuring, side profiles, makeup changes, or color alterations; the other is false positives based solely on similar clothing colors or backgrounds. For example, when a queried person's clothing color is similar to a registered character, but their hairstyle outline is clearly visible and significantly inconsistent with the registered character, the hairstyle outline element triggers a conflict penalty, significantly lowering the element combination constraint score and thus avoiding false positives.

[0050] Furthermore, it should be noted that, in addition to the aforementioned formulaic weighting, combination enhancement, and conflict penalty, the combined constraint scoring can also be implemented using learned ranking models, gradient boosting trees, neural network fusion models, or Bayesian scoring models. In these models, the visibility, quality, stability, discriminative power, and single-element similarity of each element serve as feature inputs, and the combined constraint score is the output. Combined satisfaction can be achieved using minimum value aggregation, or by employing the geometric mean or soft minimum aggregation of the product of the objective values ​​of each element. Conflict penalties can be implemented using ratios, or by employing a monotonically increasing function of the number of conflicting elements or the form of the maximum conflict term.

[0051] Step S14: Determine the comprehensive credibility score based on the target score and the element combination constraint score, and aggregate multiple comprehensive credibility scores corresponding to the same candidate registration role to obtain an aggregate score.

[0052] In this embodiment, the global visual score in the target score is determined as the basic visual score; the target score includes the global visual score, the face portion, and the head portion; if the face portion in the target score is greater than or equal to a preset face portion threshold, a first difference between the face portion and the preset face portion threshold is determined, and a first product between the first difference and a preset face gain coefficient is determined; the sum of the basic visual score and the first product is determined as the current target visual score; if the head portion in the target score is greater than or equal to a preset head portion threshold, a second difference between the head portion and the preset head portion threshold is determined, and a second product between the second difference and a preset head gain coefficient is determined; the sum of the current target visual score and the second product is updated as the current target visual score; the current target visual score and the element combination constraint score are weighted and fused, and the comprehensive credibility score is determined by controlling the corresponding fusion result within a second preset value range. Specifically, the target score includes the global visual score S. visual Surface part S face and the head part S head To balance speed and accuracy, this invention first employs multi-path candidate recall, followed by fine-grained scoring of the candidates. Multi-path recall can include global visual vector recall, facial vector recall, head region vector recall, and element summary recall. Determining the overall credibility score may include the following sub-steps.

[0053] Sub-step one: Determine the global visual score from the target score as the basic visual score: ; Sub-step two: If the surface portion of the target score is greater than or equal to a preset surface portion threshold T faceThen, a first difference is determined between the face portion and the preset face portion threshold, and the first difference is determined to be related to a preset facial gain coefficient η. face The first product between the base visual score and the first product is used to determine the current target visual score. like ; but .

[0054] Sub-step three: If the head portion of the target score is greater than or equal to a preset head portion threshold T head Then, a second difference is determined between the head portion and the preset head portion threshold, and the second difference is determined relative to the preset head gain coefficient η. head The second product between the two products updates the current target visual score by summing the current target visual score with the sum of the two products: like ; but .

[0055] Sub-step four involves weighted fusion of the current target visual score and the element combination constraint score, and determining the comprehensive credibility score (i.e., single-evidence pre-score) by controlling the corresponding fusion result within a second preset value range. ; in, and η is the trigger threshold. face and η head ρ is the gain coefficient, ρ is the fusion weight of the basic visual score and the element constraint score, and clip(·,0,1) represents the second preset value range that restricts the result to between 0 and 1.

[0056] The threshold-triggered fusion mechanism aims to prevent facial or head regions from having an excessive impact on the final result when their quality is low or their scores are insufficient; conversely, it uses facial or head regions as enhancements to boost candidate character scores when they achieve high similarity. This approach utilizes stable facial information within the character while also being compatible with scenarios where facial details are unstable or unavailable, such as animation, comics, and short drama adaptations. Therefore, this embodiment, while leveraging stable facial information, also accommodates scenarios where facial details are unstable or unavailable, demonstrating better cross-genre adaptability compared to fixed-weight fusion schemes.

[0057] In addition, facial features can be provided by face detection and facial feature models, or by head contour models, local facial feature models, or facial attribute models. For cartoon characters, facial weights can be reduced. Head region features can be obtained by expanding and cropping the head region according to the character's bounding box and encoding it, or by hairstyle segmentation, headdress segmentation, or head and shoulder region encoding. In addition to using linear gains for the portion exceeding the threshold, enhancement terms can also use a smooth soft threshold function to make the score transition smoothly around the threshold and avoid jitter near the threshold.

[0058] In this embodiment, for the same candidate role R, there may be multiple pieces of query evidence in the video to be detected. Since a character may only appear in a few frames in a short video, it is not possible to simply average across all frames; otherwise, the target evidence would be diluted by a large number of irrelevant frames. Aggregation may include the following sub-steps.

[0059] Sub-step one involves determining the information stability weight of each detection information corresponding to the candidate registered role, and selecting the top target number of detection information with the highest information stability weight. The information stability weight is determined based on at least one of the following: character detection confidence, image quality, consistency between adjacent frames, whether they belong to the same tracking trajectory, whether they contain core elements, and whether they are located in the main subject position of the image. The detection information includes the character cropped image, original frame, timestamp, frame number, character detection box, character detection confidence, image quality index, and cropping source information.

[0060] Sub-step two involves weighted summation of the comprehensive credibility scores corresponding to the top number of detection pieces of information based on the information stability weights, to obtain an aggregate score. Preferably, the top M stable pieces of evidence for each candidate role are retained. Stable polymerization fraction is used: ; in, The evidence stability weight, i.e., the aforementioned information stability weight, can be determined by factors such as the confidence level of person detection, image quality, consistency between adjacent frames, whether they belong to the same tracking trajectory, whether they contain core elements, and whether they are located in the main position of the image. As a variety of implementation methods, soft maximum aggregation can also be used when it is necessary to emphasize a single strong piece of evidence. ; Here, λ is a control parameter; the larger λ is, the closer the aggregation result is to taking the maximum evidence score. Furthermore, multi-frame evidence aggregation can incorporate diversity constraints to avoid evidence all coming from the same adjacent frame or the same similar scene. For example, it can prioritize evidence from different time periods, different poses, different cropping frames, or different shots to improve the representativeness of the reviewed evidence. Besides the weighted aggregation and soft maximum aggregation mentioned above, aggregation methods can also employ maximum value aggregation, trajectory-level aggregation, or shot-level aggregation.

[0061] In this way, because aggregation is performed only on a small amount of highly stable evidence, a large number of low-quality, low-confidence, or irrelevant frames are not included in the aggregation, and the target evidence is not diluted. Since the stability weight comprehensively considers factors such as whether it contains core elements and whether it is in the main position of the frame, key evidence containing distinctive elements receives higher weight. Therefore, even if a suspected character only appears briefly in the video, as long as their key evidence is stable and reliable, the aggregation score can still reach the level required for discrimination. Thus, this embodiment significantly reduces the risk of missing detection of characters who appear briefly but have obvious IP characteristics.

[0062] Step S15: Construct a candidate evidence package corresponding to the candidate registered role based on the aggregated score, the target score, the element combination constraint score, and the registration reference graph corresponding to the candidate registered role.

[0063] In this embodiment, for each candidate role, the system selects several pieces of evidence based on score, stability, and diversity to form a candidate-level evidence package. The candidate evidence package preferably includes the following: a query of the person's cropped image; a query of the original frame; a query of the video name, timestamp, and frame number; person detection bounding boxes, head region bounding boxes, and face region bounding boxes; person detection confidence and image quality indicators; candidate role identifiers, candidate role names, and candidate rankings; a registration reference image of the candidate role; global visual score, face portion score, head portion score, element node score, combination enhancement score, conflict penalty score, and video-level aggregation score; and an element similarity table, a combination constraint trigger table, and a conflict element table.

[0064] Step S16: Review the candidate evidence package to obtain the corresponding structured review result, and determine the infringement warning result corresponding to the character image to be detected based on the aggregation score and the structured review result.

[0065] In this embodiment, the candidate evidence package is input into a preset review model to obtain the corresponding structured review result. The structured review result includes a review conclusion, a review score, confidence level, cause of uncertainty, a list of matching elements, and a list of conflicting elements. The review conclusion is either a match, pending review, or no match. If the aggregate score is greater than or equal to a preset hit threshold, and the review conclusion is a match, then the infringement warning result corresponding to the character image to be detected is determined as a hit. If the review conclusion is pending review, or the aggregate score is greater than or equal to a preset manual review threshold and less than the preset hit threshold, then the infringement warning result corresponding to the character image to be detected is determined as a manual review. If the aggregate score is less than the preset manual review threshold, or the review conclusion is a no match, then the infringement warning result corresponding to the character image to be detected is determined as a no hit. Otherwise, the infringement warning result corresponding to the character image to be detected is determined as a no hit. Specifically, after receiving the evidence package, the structured review module does not directly output a legally significant infringement conclusion, but instead outputs a technical judgment result. The review result can be expressed as: ; Among them, MATCH indicates that the candidate role matches the query evidence, UNCERTAIN indicates that the evidence is insufficient or ambiguous and requires manual review, and NO_MATCH indicates that there is no match.

[0066] The structured review module's output fields preferably include: review conclusion, review score, confidence level, reason for uncertainty, judgment rationale, list of matching elements, and list of conflicting elements. Reasons for uncertainty preferably include low quality, local perspective, identity feature ambiguity, and mixed evidence. The final judgment rule can be expressed as: ; in, The hit threshold, This is the threshold for manual review.

[0067] The structured verification mechanism described above differs from simply calling a large model. Its input is constrained by candidate recall and element graph scoring, and its output must be backfilled with structured fields, which, together with the system score, determine the warning result. This reduces the instability caused by the free judgment of the large model and forms a traceable and interpretable chain of evidence.

[0068] Furthermore, the final warning output module can generate an evidence chain report, which includes at least: the name of the video to be detected, the detection time, the frame extraction strategy, the maximum number of frames extracted, and the maximum number of characters to be cropped per frame; the candidate character identifier, character name, candidate ranking, source of the registration reference image, and image library version; the global visual score, face portion, head portion, element node score, combination enhancement score, conflict penalty score, and final video score for each candidate character; the triggered combination constraints, such as the hairstyle outline, headdress, and clothing color combination being satisfied; the triggered conflict items, such as the core hairstyle outline being clear but significantly inconsistent with the registered character; the structured review conclusion, review score, confidence level, reasons for uncertainty, and Chinese reasons; the key evidence frames that support the hit and the evidence frames that require manual review; and the cropped image, original frame, timestamp, and cropping box corresponding to each evidence frame. This evidence chain report can be used for internal platform review, rights holder monitoring, manual review and sorting, and subsequent evidence collection and retention. Because the report contains interpretable and conflicting elements, reviewers can not only see the conclusion of "similar or dissimilar," but also the specific visual constituent elements on which the system's judgment is based.

[0069] In this way, the structured review mechanism of this embodiment, whose input is constrained by candidate recall and element spectrum scoring, must output structured fields and jointly determine the warning result with the system score. Since the review model only works on the constrained candidate evidence package and must output structured fields, its conclusions can be cross-validated by the system score, thereby reducing the instability caused by the free judgment of large models. Since the final judgment depends on both the aggregated score and the review conclusion, and introduces a three-state manual review, uncertain situations such as low quality, local perspectives, identity ambiguity, or mixed evidence are no longer forcibly binarized, but are guided to manual review, thereby reducing misjudgments while forming a traceable and interpretable chain of evidence. Therefore, compared with binarization or pure large model solutions, this embodiment has higher discrimination robustness and stronger interpretability, and is more in line with the copyright review process.

[0070] As can be seen, this application achieves efficient, accurate, and interpretable infringement warning by constructing a multi-dimensional character image comparison mechanism: On the one hand, it quickly identifies candidate objects by matching the similarity between the vector to be detected and the vector of the registered character, and calculates the element combination constraint score by combining the visual composition element map, thereby taking into account both the overall style and the compliance judgment of local elements, significantly improving the accuracy of detection and anti-interference ability; on the other hand, by aggregating the comprehensive credibility scores of multiple frames and generating a candidate evidence package containing review information, it avoids the randomness of single-frame misjudgment and provides a structured basis for manual review. Finally, it combines the automated quantitative scoring with the qualitative review results to form a reliable infringement judgment output, thereby ensuring the efficiency of the warning, effectively reducing the risk of false alarms and false negatives, and supporting the closed-loop management of subsequent rights protection or compliance handling.

[0071] This embodiment refines and expands the registration stage described above, which occurs before frame extraction of the video to be detected. The registration stage is used to establish a preset registered role library and a preset visual composition element map, such as... Figure 3 As shown, the steps may include the following.

[0072] The system receives the registration reference image or video of the character to be protected and configures metadata including character identifier, character name, work title, reference source, and rights holder information. In this step, the consistency of the same character identifier and character name can be verified simultaneously to prevent the same character from being entered into the database repeatedly or to avoid name conflicts. As for various implementation methods, the registration reference can be a single high-definition poster, several stills, a master video, or a combination thereof; the metadata can be stored in a relational database, key-value database, or file system.

[0073] The registered reference image or video is subjected to person detection to determine the target person's cropping area. If person detection fails, the entire image can be used as a fallback reference according to the configuration; if multiple people are detected, the target person is selected based on rules such as manual specification, central region, detection confidence, or association with role identifiers. For example, for a poster containing both main characters and supporting characters, the person with the highest detection confidence and located in the center of the image can be prioritized as the target person.

[0074] The cropped region of the target character is enhanced, and a target vector corresponding to the character to be protected is generated based on the enhanced region. The enhancement process includes any one or a combination of several of the following: scale transformation, cropping perturbation, brightness perturbation, contrast perturbation, sharpness perturbation, slight rotation, and local occlusion simulation. The target vector includes a global visual vector, a facial vector, and a head region vector. The enhancement process improves the system's robustness to low-resolution images, compression, angle changes, and style changes. As for various implementation methods, multiple enhanced versions can be generated for the same cropped region of the target character and encoded separately, then averaged or multiple vectors can be retained; alternatively, only the original region can be encoded, and enhancement can be performed during testing in the detection phase.

[0075] A preset registered role library is constructed based on the target vector corresponding to the role to be protected, the registration reference image or registration reference video, the cropping box, and the metadata. Each registered role in the registered role library is associated with at least its global visual vector, facial vector, head region vector, reference image, cropping box, detection score, role identifier, role name, and source path.

[0076] The visual constituent elements of the protected character are extracted using a preset element extraction model or with manual annotation. Element nodes are then determined based on these visual constituent elements. Character visual constituent elements include any one or a combination of several of the following: facial contour, eyebrow, eye, nose, and mouth structure, hairstyle outline, hair color, clothing silhouette, clothing color scheme, headdress, accessories, props, posture, and body proportions. Element nodes include element type, element features, visibility statistics, stability weight, discriminative weight, and conflict threshold. As for various implementation methods, element extraction can be achieved by outputting structured elements from a visual language model, or by using a segmentation model with a classifier, an attribute recognition model, semi-automatic manual annotation, or rule templates.

[0077] The visual constituent elements of the character to be protected are defined as nodes, and the combination constraints, substitutability relationships, missing tolerance relationships, or conflict relationships between the visual constituent elements of the character to be protected are defined as edges, so as to construct a preset visual constituent element graph. For example... Figure 2 As shown, facial contours, eyebrows, eyes, nose, mouth, hairstyle and hair color, clothing silhouette, clothing color scheme, headwear and accessories, and posture are all nodes. The edges between nodes represent the combination relationships in character recognition. For example, for a certain character, red clothing alone is not enough to determine similarity, but a combination of a specific high ponytail hairstyle, gold headwear, red cape, and a specific weapon can trigger combination enhancement; conversely, if the hairstyle and head structure of the character in the query evidence conflict with the core elements of the registered character, even if the clothing colors are similar, the final score should be reduced. As for various implementation methods, the element graph can be stored using a graph structure with nodes, edges, weights, and thresholds, or it can be stored using relational tables, constraint rule bases, knowledge graphs, JSON (JavaScript Object Notation), or database tables.

[0078] It is important to note that the visual composition element map of this invention differs fundamentally from existing cross-camera character trajectory maps. In cross-camera character trajectory maps, nodes represent character trajectories in a video, and edges represent co-occurrence or temporal relationships between trajectories, aiming to cluster the same character within the same video or scene. In contrast, in the visual composition element map of this invention, nodes represent visual composition elements within a character's image, and edges represent combination constraints, substitutability, and conflict relationships between visual elements, aiming to identify similar character images after modification, costume changes, color alterations, partial occlusion, or secondary creation. The derivation of its technical effect is as follows: Because the map uses stable appearance elements rather than volatile video trajectories as basic units, when a character is reshot, redrawn, or the actor is changed, the trajectory completely changes while the appearance elements and their combination relationships are still preserved. Therefore, the element map of this invention possesses a stable recognition capability in modified character scenes that existing trajectory maps lack.

[0079] It should be noted that this invention is not limited to a specific model or implementation method. As long as the technical concept of visual element graphs combined with combination constraints, multi-evidence aggregation, and structured verification is employed, the purpose of this invention can be achieved. Table 2 below shows the preferred and alternative implementations of each technical step. Those skilled in the art can freely choose or combine them according to role type, content category, and computing power conditions.

[0080] Table 2. Schematic diagram of the preferred and alternative implementations of each technical step.

[0081] Furthermore, the existing engineering foundation can support the registration, recall, evidence construction, and review processes of this invention. For example, existing character image storage, image library vector storage, and character name conflict verification can correspond to the registration stage; it is recommended to add element graph fields, combination constraint fields, and element version numbers to the image library metadata. Existing character detection, character cropping, and reference image enhancement can reduce background interference; it is recommended to add element region cropping such as head, hairstyle, clothing, and accessory regions. Existing image-text comparison visual vector recall can be positioned as a recall channel. Existing facial vectors and head region vectors can be fused using threshold triggering to avoid misleading low-quality faces. Existing candidate evidence frame output can be used to construct candidate evidence packages and supplement element similarity tables, combination constraint trigger tables, and conflict tables. Existing multimodal review interfaces can be limited to structured evidence review rather than direct infringement judgment. Existing result webpage display can be enhanced with element graph editing, constraint rule configuration, and evidence chain export.

[0082] See Figure 4 As shown, this embodiment of the invention discloses a character image infringement early warning device, comprising: The detection information generation module 11 is used to extract frames from the video to be detected, perform human detection on the extracted target frames to determine the human cropping area, and generate detection information corresponding to the human cropping area. Matching module 12 is used to generate a detection vector and visual constituent elements of the character to be detected in the video to be detected based on the detection information, and to match the detection vector with the registered characters in the preset registered character library to determine the candidate registered characters. The element combination constraint score determination module 13 is used to determine the target score based on the similarity between the registered role vector of the candidate registered role and the vector to be detected, and to determine the element combination constraint score between the image of the character to be detected and the candidate registered role according to the preset visual composition element map corresponding to the candidate registered role and the visual composition elements of the character to be detected. The aggregation score determination module 14 is used to determine a comprehensive credibility score based on the target score and the element combination constraint score, and to aggregate multiple comprehensive credibility scores corresponding to the same candidate registration role to obtain an aggregation score; The candidate evidence package construction module 15 is used to construct the candidate evidence package corresponding to the candidate registration role based on the aggregate score, the target score, the element combination constraint score and the registration reference graph corresponding to the candidate registration role; The infringement warning result determination module 16 is used to review the candidate evidence package, obtain the corresponding structured review result, and determine the infringement warning result corresponding to the character image to be detected based on the aggregation score and the structured review result.

[0083] For details on the specific implementation methods and technical effects of each module in the device described in this embodiment, please refer to the corresponding content of the foregoing method embodiments, which will not be repeated here.

[0084] As can be seen, this application achieves efficient, accurate, and interpretable infringement warning by constructing a multi-dimensional character image comparison mechanism: On the one hand, it quickly identifies candidate objects by matching the similarity between the vector to be detected and the vector of the registered character, and calculates the element combination constraint score by combining the visual composition element map, thereby taking into account both the overall style and the compliance judgment of local elements, significantly improving the accuracy of detection and anti-interference ability; on the other hand, by aggregating the comprehensive credibility scores of multiple frames and generating a candidate evidence package containing review information, it avoids the randomness of single-frame misjudgment and provides a structured basis for manual review. Finally, it combines the automated quantitative scoring with the qualitative review results to form a reliable infringement judgment output, thereby ensuring the efficiency of the warning, effectively reducing the risk of false alarms and false negatives, and supporting the closed-loop management of subsequent rights protection or compliance handling.

[0085] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0086] Figure 5 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the character image infringement warning method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0087] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0088] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0089] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the character image infringement warning method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0090] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned character image infringement early warning method. The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0091] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0092] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0093] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0094] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0095] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for early warning of character image infringement, characterized in that, include: The video to be detected is frame-by-frame extracted, and human detection is performed on the extracted target frames to determine the human cropping region and generate the detection information corresponding to the human cropping region. Based on the detection information, a detection vector and visual constituent elements of the character to be detected in the video to be detected are generated. The detection vector is then matched with registered characters in a preset registered character library to determine candidate registered characters. The target score is determined based on the similarity between the registered character vector of the candidate registered character and the vector to be detected. The element combination constraint score between the image of the character to be detected and the candidate registered character is determined based on the preset visual composition element map corresponding to the candidate registered character and the visual composition elements of the character to be detected. A comprehensive credibility score is determined based on the target score and the element combination constraint score. Multiple comprehensive credibility scores corresponding to the same candidate registration role are aggregated to obtain an aggregate score. Based on the aggregate score, the target score, the element combination constraint score, and the registration reference graph corresponding to the candidate registered role, construct the candidate evidence package corresponding to the candidate registered role; The candidate evidence package is reviewed to obtain the corresponding structured review results. Based on the aggregate score and the structured review results, the infringement warning result corresponding to the character image to be detected is determined. The step of determining the element combination constraint score between the image of the character to be detected and the candidate registered character based on the preset visual composition element map corresponding to the candidate registered character and the visual composition elements of the character to be detected includes: The visibility and quality of each visual element in the visual elements of the character to be detected are determined, and the stability and distinguishability of the corresponding visual elements in the preset visual element map are obtained. The visibility is used to characterize the observability of the visual elements in the detection information. The quality is used to characterize the sharpness, brightness, compression distortion, motion blur, and effective pixel ratio of the visual element region. The stability is used to characterize the degree to which the visual elements remain stable in different registration reference images, different plot stages, or different style versions. The distinguishability is used to characterize the ability of the visual elements to distinguish the characters to be protected. Determine the cosine similarity between the element feature vector of each character visual element in the visual elements of the character to be detected and the element feature vector of the corresponding character visual element in the preset visual elements map, and determine the cosine similarity as the single element similarity. The product of the visibility, quality, stability and discriminability is determined as a weighting coefficient. Based on the weighting coefficient, the single-element similarity of the visual components of each character is weighted and summed to obtain the element node score. The element combination constraint score is determined based on the element node score.

2. The character image infringement early warning method according to claim 1, characterized in that, Before performing frame extraction on the video to be detected, the process also includes: Receive the registration reference image or registration reference video of the character to be protected, and configure metadata including character identifier, character name, work name, reference source and rights holder information; Perform person detection on the registration reference image or registration reference video to determine the target person cropping area; The cropped area of ​​the target character is enhanced, and a target vector corresponding to the character to be protected is generated based on the enhanced area. The enhancement process includes any one or a combination of several of the following: scale transformation, cropping perturbation, brightness perturbation, contrast perturbation, sharpness perturbation, slight rotation, and local occlusion simulation. The target vector includes a global visual vector, a facial vector, and a head region vector. A preset registered role library is constructed based on the target vector corresponding to the role to be protected, the registration reference image or registration reference video, the cropping frame, and the metadata. The visual constituent elements of the character to be protected are extracted by a preset element extraction model or manual annotation, and element nodes are determined based on the visual constituent elements of the character to be protected. The visual constituent elements of the character include any one or a combination of several of the following: facial contour, eyebrow, eye, nose and mouth structure, hairstyle contour, hair color, clothing silhouette, clothing color scheme, headdress, accessories, props, action posture and body proportion. The element nodes include element type, element features, visibility statistics, stability weight, discrimination weight and conflict threshold. The visual constituent elements of the character to be protected are defined as nodes, and the combination constraints, substitutability relationships, missing tolerance relationships, or conflict relationships between the visual constituent elements of the character to be protected are defined as edges, so as to construct a preset visual constituent element map.

3. The character image infringement early warning method according to claim 1, characterized in that, Determining the element combination constraint score based on the element node score includes: Determine whether the visibility, quality, and single-element similarity of each character's visual elements corresponding to each combination constraint in the preset visual element map meet the first threshold. If all conditions are met, the target product between visibility, quality, and single-element similarity is determined, and the minimum value among the target products of the visual constituent elements of each role corresponding to the combined constraints is determined as the combined satisfaction. The combined satisfaction is weighted and summed based on the weights of each combined constraint to obtain the combined enhancement score; If there is a target character visual element in the visual elements of the character to be detected that meets the second threshold in both visibility and quality, but whose single element similarity is less than the preset conflict threshold, then the ratio of the sum of the conflict weights corresponding to the target character visual element to the sum of the conflict weights corresponding to each character visual element in the visual elements of the character to be detected is determined as the conflict penalty score. The element node score and the combined enhancement score are weighted based on the first configurable weight to obtain a weighted result. The difference between the weighted result and the conflict penalty score after being weighted according to the second configurable weight is determined, and the element combined constraint score is obtained by limiting the difference to a first preset value range.

4. The character image infringement early warning method according to claim 1, characterized in that, The target score includes the global visual score, the face score, and the head score. Accordingly, determining the comprehensive credibility score based on the target score and the element combination constraint score includes: The global visual score in the target score is determined as the basic visual score; If the face portion in the target score is greater than or equal to a preset face portion threshold, then a first difference between the face portion and the preset face portion threshold is determined, and a first product between the first difference and a preset face gain coefficient is determined. The sum of the basic visual score and the first product is determined as the current target visual score; If the head portion in the target score is greater than or equal to a preset head portion threshold, then a second difference between the head portion and the preset head portion threshold is determined, and a second product between the second difference and a preset head gain coefficient is determined. Update the sum between the current target visual score and the second product to the current target visual score; The current target visual score and the element combination constraint score are weighted and fused, and the comprehensive credibility score is determined by controlling the corresponding fusion result within a second preset value range.

5. The character image infringement early warning method according to claim 1, characterized in that, The aggregation of multiple comprehensive credibility scores corresponding to the same candidate registered role to obtain an aggregate score includes: The information stability weights of each detection information corresponding to the candidate registered roles are determined, and the top target number of detection information with the highest information stability weights are selected. The information stability weights are determined based on at least one of the following: character detection confidence, image quality, consistency between adjacent frames, whether they belong to the same tracking trajectory, whether they contain core elements, and whether they are in the main position of the image. The detection information includes a character cropped image, original frame, timestamp, frame number, character detection box, character detection confidence, image quality index, and cropping source information. Based on the aforementioned information stability weights, the comprehensive credibility scores corresponding to the number of detection information items for the previous target are weighted and summed to obtain the aggregate score.

6. The character image infringement early warning method according to any one of claims 1 to 5, characterized in that, The process of reviewing the candidate evidence package to obtain a corresponding structured review result, and determining the infringement warning result corresponding to the character image to be detected based on the aggregate score and the structured review result, includes: The candidate evidence package is input into a preset review model to obtain the corresponding structured review results; the structured review results include review conclusion, review score, confidence level, cause of uncertainty, list of matching elements and list of conflicting elements, and the review conclusion is a match, pending review or no match; If the aggregate score is greater than or equal to the preset hit threshold, and the review conclusion is a match, then the infringement warning result corresponding to the character image to be detected is determined as a hit; If the review conclusion is pending review, or if the aggregate score is greater than or equal to the preset manual review threshold and less than the preset hit threshold, then the infringement warning result corresponding to the character image to be detected will be determined as manual review. If the aggregate score is less than the preset manual review threshold, or the review conclusion is a mismatch, then the infringement warning result corresponding to the character image to be detected will be determined as a miss. Otherwise, the infringement warning result corresponding to the character image to be detected will be determined as a miss.

7. A character image infringement early warning device, characterized in that, include: The detection information generation module is used to extract frames from the video to be detected, perform human detection on the extracted target frames to determine the human cropping area, and generate detection information corresponding to the human cropping area. The matching module is used to generate a detection vector and visual constituent elements of the character to be detected in the video to be detected based on the detection information, and to match the detection vector with the registered characters in the preset registered character library to determine the candidate registered characters. The element combination constraint score determination module is used to determine the target score based on the similarity between the registered role vector of the candidate registered role and the vector to be detected, and to determine the element combination constraint score between the image of the character to be detected and the candidate registered role according to the preset visual composition element map corresponding to the candidate registered role and the visual composition elements of the character to be detected. The aggregation score determination module is used to determine a comprehensive credibility score based on the target score and the element combination constraint score, and to aggregate multiple comprehensive credibility scores corresponding to the same candidate registration role to obtain an aggregation score; The candidate evidence package construction module is used to construct the candidate evidence package corresponding to the candidate registration role based on the aggregate score, the target score, the element combination constraint score and the registration reference graph corresponding to the candidate registration role; The infringement warning result determination module is used to review the candidate evidence package, obtain the corresponding structured review result, and determine the infringement warning result corresponding to the character image to be detected based on the aggregation score and the structured review result. The element combination constraint score determination module is used to determine the visibility and quality of each character visual element in the visual elements of the character to be detected, and to obtain the stability and distinguishability of the corresponding character visual elements in the preset visual element map; the visibility is used to characterize the observability of the character visual elements in the detection information; the quality is used to characterize the sharpness, brightness, compression distortion, motion blur, and effective pixel ratio of the character visual element region; the stability is used to characterize the degree to which the character visual elements remain stable in different registration reference images, different plot stages, or different style versions; the distinguishability... This is used to characterize the ability of visual elements to distinguish the characters to be protected; determine the cosine similarity between the element feature vector of each visual element in the visual elements to be detected and the element feature vector of the corresponding visual element in the preset visual element map, and determine the cosine similarity as the single element similarity; determine the product of visibility, quality, stability and distinguishability as a weighting coefficient, and perform a weighted summation of the single element similarities of each visual element based on the weighting coefficient to obtain the element node score; determine the element combination constraint score based on the element node score.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing a computer program to implement the steps of the character image infringement early warning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on a computer-readable storage medium, and when executed by a processor, the computer program implements the steps of the character image infringement early warning method as described in any one of claims 1 to 6.

Images