Method, device and medium for detecting copyright risk of black-box generative model

Abstract

The application discloses a copyright risk detection method and device of a black box generative model, equipment and medium, relates to the field of copyright risk detection, and comprises the following steps: obtaining target copyright asset characteristics from a target copyright asset library and generating a target feature vector; generating a current model prompt word according to a current prompt word generation strategy and the target feature vector, inputting the current model prompt word into a to-be-detected black box model by using a preset model access interface, and obtaining a current output sample of the to-be-detected black box model; judging whether the current model test budget is exhausted, if not, updating the prompt word for multiple rounds and outputting a similarity feedback result of each round; and if the current model test budget is exhausted, performing copyright risk detection on the to-be-detected black box model based on the similarity feedbacks. By automatically updating the prompt word and performing multiple rounds of iteration, the problem that it is difficult to cover enough detection paths is solved, and the detection result accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of copyright risk detection, and in particular to a method, apparatus, equipment, and medium for copyright risk detection of black-box generative models. Background Technology

[0002] In third-party AI platforms, commercial API models, enterprise procurement models, or closed-system deployment models, testers typically cannot access the training set, model weights, model parameters, and internal inference state. They can only submit prompts and receive model output samples through a black-box interface. Under these conditions, if the model has memorized specific copyrighted assets during training or fine-tuning, ordinary manual testing is unlikely to systematically detect this risk.

[0003] Current methods for detecting model copyright risks involve testing the model's output using a fixed test set to determine the model's copyright risk. However, this method struggles to cover a sufficient number of detection paths, leading to inaccurate detection results. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for copyright risk detection using a black-box generative model. This method, through automatic updating of prompts and multiple iterations, solves the problem of insufficient coverage of detection paths and improves the accuracy of detection results. The specific solution is as follows: Firstly, this application provides a method for copyright risk detection of black-box generative models, including: Obtain the characteristics of the target copyright assets from the target copyright asset library, and generate the target feature vector corresponding to the characteristics of the target copyright assets; The current prompt word is generated based on the current prompt word generation strategy and the target feature vector. The current model prompt word is input into the black box model to be detected using a preset model access interface, and the current output sample of the black box model to be detected is obtained. The prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library. Determine whether the current model test budget has been exhausted. If the current model test budget has not been exhausted, compare the current output sample with the copyright assets in the target copyright asset library to obtain the current similarity feedback, and update the current prompt word generation strategy according to the current similarity feedback to obtain the corresponding updated prompt word generation strategy. The updated prompt word generation strategy is determined as the new current prompt word generation strategy, and the process jumps to the step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector; If the current model test budget has been exhausted, then copyright risk detection is performed on the black-box model to be detected based on the similarity feedback corresponding to each round.

[0005] Optionally, obtaining the characteristics of the target copyright asset from the target copyright asset library includes: The target data type corresponding to the copyright assets in the target copyright asset library is determined, and feature extraction is performed on the copyright assets according to the target data type to obtain the features of the target copyright assets; wherein, the data type of the copyright assets includes text type, image type, video type and audio type, and the copyright asset features include plot features, character relationship features, scene features, camera language features, music rhythm features, keyword features, text fragment features and multimodal embedding features.

[0006] Optionally, generating the current model prompt word based on the current prompt word generation strategy and the target feature vector includes: According to the current prompt word generation strategy, the feature subset corresponding to the current round is extracted from the target feature vector; the feature subset is a set of features selected from different copyright asset features contained in the target feature vector; Based on the model type of the black-box model to be detected, select the target prompt word template corresponding to the model type from the preset template library; The target abstraction level is determined from the preset abstraction levels; the preset abstraction levels are the levels that measure the degree of abstraction of the prompt words. Based on the feature subset, the target prompt word template, and the target abstraction level, a candidate prompt word set corresponding to the current round is generated, and the current model prompt word is determined from the candidate prompt word set.

[0007] Optionally, determining the current model prompt word from the candidate prompt word set includes: A set of deduplicated prompts is selected from the candidate prompts set based on the priority of each candidate prompt. The priority is determined based on the target indicators corresponding to each prompt. The target indicators include the number and importance of the copyright asset features covered by the candidate prompts, the cost required to call the candidate prompts, and the benefits of the candidate prompts determined based on historical feedback results. The similarity of each prompt word in the set of prompt words to be deduplicated is compared. If the similarity between any two prompt words exceeds a preset prompt word similarity threshold, the prompt word with lower priority among the two prompt words is deleted to obtain the current model prompt word.

[0008] Optionally, the step of comparing the current output sample with the copyright assets in the target copyright asset library to obtain current similarity feedback includes: The current output sample is compared with the copyright assets in the target copyright asset library in a multidimensional similarity comparison to obtain the corresponding comparison results. The similarity comparison process includes text segment similarity comparison, plot structure similarity comparison, character relationship similarity comparison, scene similarity comparison, camera language similarity comparison, music rhythm similarity comparison, keyword similarity comparison, and multimodal embedding feature similarity comparison. Obtain the common expression feature hit status of the current output sample; the common expression feature hit status represents the situation in which the current output sample hits a common expression feature, wherein the common expression feature is a feature that appears more than a preset threshold in the public corpus and does not belong to the unique expression of the copyright asset; The current similarity feedback is generated based on the comparison results and the hit rate of the common expression features.

[0009] Optionally, the copyright risk detection of the black-box model to be detected based on the similarity feedback corresponding to each round includes: Based on the similarity feedback, target rounds with a target similarity not less than a preset data similarity threshold are selected from each round; wherein, the target similarity characterizes the similarity between the model output sample and the copyright asset; Different perturbation processes are applied to the target model prompt words corresponding to the target round to obtain perturbation prompt words, and each perturbation prompt word is output to the black box model to be detected to obtain perturbation output samples corresponding to each perturbation prompt word. Each of the perturbation output samples is compared with the copyright asset to obtain the corresponding perturbation similarity feedback, and the risk detection result corresponding to the black box model to be detected is determined based on the perturbation similarity feedback.

[0010] Optionally, after performing copyright risk detection on the black-box model to be detected based on the similarity feedback corresponding to each round, the method further includes: Generate a detection evidence package corresponding to the black box model to be detected; wherein, the detection evidence package includes the asset identifier of the target copyright asset library, the model identifier of the black box model to be detected, the model prompt word sequence and corresponding hash value corresponding to each round, the similarity feedback corresponding to each round, and the risk detection result of the black box model to be detected.

[0011] Secondly, this application provides a copyright risk detection device for a black-box generative model, comprising: The vector generation module is used to obtain the characteristics of the target copyright assets from the target copyright asset library and generate the target feature vector corresponding to the characteristics of the target copyright assets; The output sample acquisition module is used to generate the current model prompt word according to the current prompt word generation strategy and the target feature vector, input the current model prompt word into the black box model to be detected using a preset model access interface, and acquire the current output sample of the black box model to be detected; wherein, the prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library; The strategy update module is used to determine whether the current model test budget has been exhausted. If the current model test budget has not been exhausted, the current output sample is compared with the copyright assets in the target copyright asset library to obtain the current similarity feedback, and the current prompt word generation strategy is updated according to the current similarity feedback to obtain the corresponding updated prompt word generation strategy. The step jump module is used to determine the updated prompt word generation strategy as the new current prompt word generation strategy, and jump to the step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector; The risk detection module is used to perform copyright risk detection on the black-box model to be tested based on the similarity feedback corresponding to each round if the current model test budget has been exhausted.

[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the aforementioned copyright risk detection method for black-box generative models.

[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned copyright risk detection method for a black-box generative model.

[0014] This application first obtains the features of target copyright assets from the target copyright asset library and generates target feature vectors corresponding to the features of the target copyright assets. Then, it generates current model prompts based on the current prompt word generation strategy and the target feature vectors. The current model prompts are input into the black-box model to be detected using a preset model access interface, and the current output samples of the black-box model to be detected are obtained. The prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library. Then, it is determined whether the current model test budget is exhausted. If the current model test budget is not exhausted, the current output samples are compared with the copyright assets in the target copyright asset library to obtain the current similarity feedback. The current prompt word generation strategy is updated based on the current similarity feedback to obtain the corresponding updated prompt word generation strategy. The updated prompt word generation strategy is then determined as the new current prompt word generation strategy, and the process jumps to the step of generating current model prompts based on the current prompt word generation strategy and the target feature vectors. Finally, if the current model test budget is exhausted, copyright risk detection is performed on the black-box model to be detected based on the similarity feedback corresponding to each round. Therefore, this application solves the problem of low coverage of manually written prompt words by extracting features from the target copyright asset library and generating corresponding target feature vectors; it solves the problem of unstable prompt word quality by driving the generation of model prompt words through a prompt word generation strategy; and it solves the problem of difficulty in covering a sufficient number of detection paths and improves the accuracy of detection results by obtaining similarity feedback by comparing the current output sample with the copyright assets, updating the prompt word generation strategy based on the feedback, and then jumping to the generation step to form an iterative loop. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a schematic diagram of the copyright risk detection method for a black-box generative model disclosed in this application; Figure 2 This application discloses a flowchart for copyright risk detection of a black-box generative model. Figure 3 This application discloses a flowchart of adaptive optimization of prompt words; Figure 4 This is a schematic diagram of the structure of a copyright risk detection system disclosed in this application; Figure 5 A timing diagram for generating an evidence package as disclosed in this application; Figure 6 This is a schematic diagram of the copyright risk detection device for a black-box generative model disclosed in this application; Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] See Figure 1 As shown in the figure, this invention discloses a copyright risk detection method for black-box generative models, including: Step S11: Obtain the target copyright asset features from the target copyright asset library and generate the target feature vector corresponding to the target copyright asset features.

[0019] The overall process for copyright risk detection of the black-box model in this embodiment is as follows: Figure 2 As shown, it includes: S100: Obtain copyright asset and black-box generative model information. The system receives the copyright asset library to be probed (i.e., the target copyright asset library), the black-box generative model access interface, model type, output modality, calling parameters, test budget, and risk threshold.

[0020] S200: Extract copyright asset features (i.e., target copyright asset features). The system extracts plot features, character relationship features, scene features, camera language features, music rhythm features, keyword features, text fragment features, and multimodal embedding features from the copyright asset database, and forms a copyright asset feature vector (i.e., target feature vector).

[0021] S300: Generate candidate prompt words (i.e., model prompt words for the current round). The system generates probe prompt words (i.e., model prompt words) based on the target feature vector and model type, including: explicit probe prompt words, implicit probe prompt words, structural probe prompt words, continuation probe prompt words, style probe prompt words, scene probe prompt words, and music rhythm probe prompt words.

[0022] S400: Invoke the black-box generative model and collect model output samples. The system submits probe prompts through the black-box access interface and records model output samples, invocation time, model version, parameter configuration, response status, and randomness information. In other words, in this step, the system invokes the black-box generative model through the black-box access interface. The system supports automated web page invocation, batch API invocation, enterprise model gateway invocation, or plugin invocation. Each invocation generates a invocation record, including model identifier, model version, invocation time, probe prompts, invocation parameters, response status, model output samples, and output hash value.

[0023] S500: Calculate similarity feedback. The system compares the similarity of the model's output samples with the copyright asset library in terms of text, plot structure, character relationships, scenes, cinematography, musical rhythm, keywords, and multimodal embedding features to obtain similarity feedback.

[0024] S600: Update the adaptive detection strategy (i.e., the prompt word generation strategy). Based on similarity feedback, output stability, reasons for detection failure, and test budget, the system updates the feature selection, prompt word template, level of abstraction, constraint strength, and perturbation parameters for the next round of detection prompt words.

[0025] S700: Performs stability retesting and risk scoring. The system performs repeated calls, perturbation calls, and cross-suggestion calls on highly similar output paths to calculate the copyright memory risk score.

[0026] S800: Generates a detection evidence package. The system outputs a sequence of detection prompt words, model output samples, similar segments, a similarity matrix, a risk score, an iteration trajectory, a hash value, a timestamp, and an audit summary.

[0027] The above steps S100 to S800 form a closed-loop detection process. S100 to S300 are responsible for generating executable detection prompts from the copyright asset library; S400 and S500 are responsible for collecting model output samples under black-box conditions and generating similarity feedback; S600 updates the adaptive detection strategy based on the similarity feedback; S700 reduces false alarms caused by accidental similarity through stability retesting; and S800 solidifies the complete test trajectory into a detection evidence package. This process does not involve inputting prompts and comparing outputs all at once, but rather uses similarity feedback as a control signal for the generation of prompts in the next round, enabling the detection process to gradually converge within the test budget.

[0028] The inputs, outputs, and core processing flow of each step above are shown in Table 1: Table 1

[0029] In this embodiment, obtaining target copyright asset features from the target copyright asset library includes: determining the target data type corresponding to the copyright asset in the target copyright asset library, and extracting features from the copyright asset based on the target data type to obtain target copyright asset features; wherein, the data type of the copyright asset includes text type, image type, video type and audio type, and the copyright asset features include plot features, character relationship features, scene features, camera language features, music rhythm features, keyword features, text fragment features and multimodal embedding features.

[0030] Specifically, for text-based copyright assets, the system extracts chapter titles, paragraph summaries, highly informative phrases, character names, character relationships, event chains, conflict nodes, dialogue excerpts, keywords, and narrative order. Instead of directly using the complete original text as detection prompts, the system transforms it into composable copyright asset features to reduce the reliance of test prompts on direct copying of the original work.

[0031] For drama or short drama copyright assets, the system extracts character function points, character relationships, key conflicts, identity reversals, misunderstandings being resolved, evidence being revealed, plot progression order, and episode structure. These features are used to generate structure detection prompts to test whether the model can reproduce similar plot structures without providing the original text.

[0032] For copyrighted image or video assets, the system extracts scene layout, subject relationships, camera language, character design, key props, color combinations, camera movement, and keyframe semantics. These features are used to generate scene detection cues or camera detection cues.

[0033] For music or audio copyright assets, the system extracts rhythm patterns, tempo, accent distribution, melody outline, chord progression, timbre tags, and lyric keywords. These features are used to generate music rhythm detection prompts.

[0034] Target feature vector It can be represented as: ; in, This represents the target feature vector of the j-th copyright asset; Indicates plot characteristics; Indicates the characteristics of the role relationship; Indicate scene characteristics; Indicates the characteristics of camera language; Indicates the characteristics of musical rhythm; Indicate keyword features; Representing text fragment features; This represents multimodal embedding features.

[0035] It should be noted that, to facilitate batch detection and evidence verification, the system can save each copyright asset feature as a structured record. The meanings of the fields included in the structured record are shown in Table 2: Table 2

[0036] Rarity score is determined by a combination of factors: frequency in public corpus, frequency of occurrence within copyright assets, frequency of occurrence across works, and weighted manual annotation. If a plot point, character relationship combination, or scene structure is common in public corpus, its rarity score is low; conversely, if it is strongly correlated with a specific copyright asset and has a low frequency of occurrence across works, its rarity score is high. In other words, the system calculates the rare feature hit score based on the rarity score in the copyright asset feature record. If the model output samples only hit common expressions, Lower; if the output sample hits multiple high-rarity features and these features come from the same copyright asset, then Increase. This mechanism is used to reduce false positives caused by similarity in common themes.

[0037] Step S12: Generate the current model prompt word according to the current prompt word generation strategy and the target feature vector, input the current model prompt word into the black box model to be detected using the preset model access interface, and obtain the current output sample of the black box model to be detected; wherein, the prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library.

[0038] It should be noted that the detection prompts generated by the system in this embodiment are not limited to one template, but include the following types.

[0039] Explicit detection cues are used to test the model’s response to titles, character names, or keywords, but such cues are not the only detection method.

[0040] Implicit detection cues are used to remove the title of the work and the name of the character, retaining only the plot relationship, scene structure, character function position or music rhythm description, in order to test whether the model outputs content that is highly similar to the copyrighted assets under implicit conditions.

[0041] The continuation detection prompts are used to provide short snippets, summaries, or rewritten snippets to observe whether the model can continue writing content similar to the copyrighted assets.

[0042] Structure detection prompts are used to provide constraints on character relationships, conflict nodes, and event chains to test whether the model can reproduce similar plot progression structures.

[0043] Perturbation detection prompts are used to rewrite prompts describing the same copyright asset features with different levels of abstraction, different orders, different language styles, or different modalities, in order to verify whether the output similarity is stable.

[0044] The system maintains a detection state vector for each round of detection. : ; in, This indicates the detection prompt for round t; This represents the output sample of the model in round t; This represents the similarity feedback in round t; This represents the copyright memory risk score for round t; Indicates historical exploration trajectories; This indicates the remaining test budget.

[0045] It should be noted that black-box calls may encounter issues such as rejection, timeout, output truncation, incorrect output format, content security interception, frequency limitation, or abnormal interface return. The system generates an exception label for each type of exception and feeds the exception label back to the detection prompt word adaptive optimization module.

[0046] If a response is rejected or content security is blocked, the system reduces the proportion of explicit work titles, original text excerpts, and sensitive words, and uses structural or implicit detection prompts instead. If output truncation occurs, the system reduces output length constraints or splits the output into multiple segments. If frequency limitations occur, the system adjusts the calling rhythm or reduces the concurrency of the candidate queue. If the model output format is inconsistent with the expected modality, the system reselects a detection template that matches the model's capabilities. The system considers low feedback, rejection, off-topic output, excessively short output, incorrect output modality, and duplicate output as different failure reasons. For low feedback, the system increases rare feature coverage; for rejection, the system reduces explicitness; for off-topic output, the system increases task constraints and output format requirements; for excessively short output, the system adjusts the generation length; for incorrect output modality, the system switches detection templates; for duplicate output, the system adds perturbation parameters or switches feature subsets.

[0047] In this embodiment, generating the current model prompt word based on the current prompt word generation strategy and the target feature vector includes: extracting a feature subset corresponding to the current round from the target feature vector according to the current prompt word generation strategy; the feature subset is a set of features selected from different copyright asset features contained in the target feature vector; selecting a target prompt word template corresponding to the model type from a preset template library according to the model type of the black box model to be detected; determining the target abstraction level from a preset abstraction level; the preset abstraction level is a level that measures the degree of abstraction of the prompt word; generating a candidate prompt word set corresponding to the current round based on the feature subset, the target prompt word template, and the target abstraction level, and determining the current model prompt word from the candidate prompt word set.

[0048] The process of determining the current model prompt word from the candidate prompt word set includes: filtering a set of prompt words to be deduplicated based on the priority of each candidate prompt word; wherein the priority is determined according to the target indicators corresponding to each prompt word, and the target indicators include the number and importance of copyright asset features covered by the candidate prompt words, the cost required to call the candidate prompt words, and the benefits of the candidate prompt words determined based on historical feedback results; comparing the similarity of each prompt word in the set of prompt words to be deduplicated, and if the similarity between any two prompt words exceeds a preset prompt word similarity threshold, then the prompt word with the lower priority between the two prompt words is deleted to obtain the current model prompt word.

[0049] In other words, in this embodiment, the system maintains a candidate queue of detection prompt words (i.e., a candidate prompt word queue) before each round of detection. Each candidate in the queue includes a probe prompt word, prompt word type, selected feature set, level of abstraction, expected trigger weight, compliance constraint flag, historical similarity feedback, and call cost. The system does not randomly select prompt words; instead, it chooses the next round of prompt words from the candidate queue based on priority.

[0050] Candidate prompt priority It can be represented as: ; in, This indicates the number and importance of copyright asset features covered by the candidate suggestions; Indicates the strength of rare features covered by candidate prompts; This represents the gain predicted based on historical similarity feedback. Indicates the cost of invocation; This indicates the penalty items that trigger refusal to answer, security blocking, or repeated testing; to This represents the weighting coefficient.

[0051] The specific process of generating and filtering prompt words is as follows: The system is based on the target feature vector Generate candidate suggestion words (i.e., candidate suggestion words) Candidate suggestions can be obtained using the following function: ; Where G represents the prompt word generation function; M represents the target feature vector; L represents the black-box generative model type; Cfg represents the output modality; Cfg represents the test configuration, including detection intensity, whether to allow explicit work names, whether to allow fragment continuation, maximum number of calls, and risk threshold.

[0052] The prompt word generation function G can perform the following sub-steps: S310: Select a feature subset. The system selects one or more feature subsets from the target feature vector, such as "character relationships + conflict nodes", "scenes + camera language", "rhythm + keywords", and "high-information text fragments + continuation instructions".

[0053] S320: Select a probe template. The system selects a text generation template, image generation template, video generation template, music generation template, or multimodal generation template based on the model type.

[0054] S330: Determine the level of abstraction. The system selects the level of abstraction for the prompt words from three levels: explicit, semi-implicit, and implicit. The explicit level can include the title of the work or the name of the character; the semi-implicit level includes key features but removes the name; the implicit level only includes structural relationships and scene constraints.

[0055] S340: Generate a candidate set of probe hints. The system generates multiple candidate probe hints and sorts them according to coverage, difference, and compliance constraints.

[0056] S350: Select candidate suggestion words. The system prioritizes suggestion words that cover key copyright asset characteristics and do not directly copy large sections of the original text as candidate suggestion words.

[0057] It should be noted that the detection prompt templates can be configured according to the output modality and feature type. For example, text-based models can use "plot structure generation templates," "character relationship continuation templates," and "high-information phrase completion templates"; image-based models can use "scene layout generation templates," "character design weakening templates," and "key prop combination templates"; video-based models can use "camera movement generation templates" and "keyframe transition templates"; and music-based models can use "rhythm generation templates" and "melody outline generation templates." Each template contains replaceable slots. Slots can include asset type, character function, conflict node, scene description, camera language, music rhythm, keywords, level of abstraction, and prohibited items. By using slot-based templates, the system can avoid directly copying the complete copyrighted work into the prompts while retaining structural information that could trigger the model's memory risk.

[0058] The system then performs deduplication, similarity filtering, and compliance control on the candidate probe prompts. If two prompts differ only in their synonyms and cover the same features, the one with higher priority is retained. If a prompt contains excessively long original text fragments or significantly exceeds the limits allowed by the test configuration, the system downgrades its weight or removes it. If the black-box generative model has repeatedly rejected a certain type of explicit prompt, the system reduces the explicitness and switches to implicit probe prompts.

[0059] It should be noted that the system generates a call record for each model call. The call log includes probe prompts, model output samples, model identifier, version number, call time, interface address summary, call parameters, response status, exception tags, and output hash value. The call log does not rely on manual screenshots and can serve as a traceable data source in the detection evidence package.

[0060] Step S13: Determine whether the current model test budget is exhausted. If the current model test budget is not exhausted, compare the current output sample with the copyright assets in the target copyright asset library to obtain the current similarity feedback, and update the current prompt word generation strategy according to the current similarity feedback to obtain the corresponding updated prompt word generation strategy.

[0061] The prompt word adaptive optimization process in this embodiment is as follows: Figure 3 As shown, this illustrates the relationship between the target feature vector, the candidate queue of detection prompts, the detection state vector, similarity feedback, the prompt update function, and the next round of detection prompts.

[0062] To avoid mistaking common themes or generic plot points for copyrighted content, the system filters or downgrades common expressions during the feature extraction stage. Common expressions include common themes, common character relationships, common scenes, common camera language, common rhythm patterns, and industry-standard keywords. Instead of directly deleting common expressions, the system marks them as low-rarity features and includes them as part of the common expression penalty in subsequent similarity feedback.

[0063] Accordingly, in this embodiment, the current output sample is compared with the copyright assets in the target copyright asset library to obtain current similarity feedback. This includes: performing multi-dimensional similarity comparison between the current output sample and the copyright assets in the target copyright asset library to obtain corresponding comparison results; wherein, the similarity comparison process includes text segment similarity comparison, plot structure similarity comparison, character relationship similarity comparison, scene similarity comparison, camera language similarity comparison, music rhythm similarity comparison, keyword similarity comparison, and multimodal embedding feature similarity comparison; obtaining the common expression feature hit status of the current output sample; the common expression feature hit status represents the situation where the current output sample hits a common expression feature, which is a feature whose frequency of occurrence in the public corpus exceeds a preset threshold and does not belong to the unique expression of the copyright asset; generating the current similarity feedback based on the comparison results and the common expression feature hit status.

[0064] Specifically, the system will output samples from the model. A multi-dimensional similarity comparison is performed with the copyright asset database to obtain similarity feedback (i.e., similarity feedback). : ; in, Indicates the similarity between text segments; Indicates the similarity of plot structure; Indicates the similarity of character relationships; Indicates scene similarity; Indicates the similarity of camera language; Indicates the similarity of musical rhythms; Indicates keyword similarity; This represents the similarity of multimodal embedding features.

[0065] Overall similarity feedback score It can be represented as: ; in, This represents the weight of the i-th similarity dimension; Let represent the score of the i-th similarity dimension in the t-th round; This represents the interaction weight between two similarity dimensions. The interaction term is used to identify cases where "single-item similarity is not high, but plot, character relationships, and keywords are all similar."

[0066] After each round of detection, the system provides feedback based on the similarity score. Update candidate queue If a feature subset covered by a certain prompt word generates high feedback, the system generates neighboring prompt words for that feature subset and increases their queue priority; if a prompt word continuously generates low feedback or rejection, the system lowers its template priority or removes it from the queue.

[0067] Queue rearrangement is not simply about selecting the most similar cue words, but about maintaining a balance between exploration and exploitation. The system can retain a certain proportion of broad-based exploration cue words to discover other possible memory paths; at the same time, more testing budget can be allocated to high-feedback paths for stability retesting and evidence solidification.

[0068] In addition, to distinguish between ordinary public expression and protected expression, the system also calculates a penalty for public expression. If the output samples only contain general scenes, general character relationships, or common music rhythms, the similarity feedback will be reduced; if the output samples hit high-information segments, specific character relationship combinations, rare plot sequences, or multimodal combination features, the similarity feedback will be increased.

[0069] In addition, the system can generate a similarity matrix. The rows of the similarity matrix represent segments, frames, audio windows, or structural nodes in the model's output samples, while the columns represent copyright asset features. Matrix elements represent the similarity between the corresponding output unit and the copyright asset feature. Through the similarity matrix, the system can not only provide an overall score but also pinpoint which output segments are similar to which copyright asset features.

[0070] Similarity location results This can include text character ranges, paragraph numbers, video frame ranges, audio time windows, image regions, character relationship edges, or plot node numbers. It will be included in the detection evidence package to explain the source of the risk score.

[0071] In this embodiment, the adaptive optimization process for prompt words is as follows: The system updates the next round of detection prompts based on similarity feedback. The prompt update function can be expressed as: ; Where U represents the prompt word update function; Indicates the current detection prompt; This indicates similarity feedback; Indicates historical exploration trajectories; Indicates the remaining test budget; This indicates the current prompt word strategy parameters.

[0072] The system adopts different update strategies based on different feedback types.

[0073] If the text fragments have low similarity but high similarity in plot structure, the system adds plot nodes and character relationship constraints, reduces original text fragment prompts, and generates structure-enhanced detection prompts.

[0074] If the keyword similarity is high but the output is unstable, the system generates multiple synonym rewriting suggestions to test whether similar outputs appear repeatedly under different expressions.

[0075] If the model refuses to answer or triggers a security policy, the system reduces the proportion of explicit work titles and original text excerpts, and switches to implicit detection prompts.

[0076] If the music rhythms are highly similar but the melodies are not, the system adds a rhythm window and accent description; if the melodies are highly similar, it adds melody outline detection prompts.

[0077] If the feedback is low in multiple rounds, the system expands the feature coverage or switches the detection template; if the feedback continues to rise in multiple rounds, the system enters the stability retesting stage.

[0078] Prompt word strategy parameters It may include: ; in, Indicates the degree of explicitness; Indicates feature coverage; Indicates the level of abstraction; Indicates the modal combination method; Indicates the number of candidate prompts in each round; This indicates a constraint on the output length or generation time.

[0079] Step S14: Determine the updated prompt word generation strategy as the new current prompt word generation strategy, and proceed to the step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector.

[0080] In this embodiment, the updated prompt word generation strategy needs to be determined as the new current prompt word generation strategy, and a new round of prompt word generation is performed. That is, the model detection in this embodiment is a multi-round closed-loop process, which enables this embodiment to adaptively update the detection strategy in each iteration, thereby improving the depth and breadth of the exploration path.

[0081] It should be noted that the system can divide the detection process into four states: initialization, breadth exploration, similarity enhancement, stability retesting, evidence consolidation, and termination. The initialization state is used to generate candidate prompts; the breadth exploration state is used to cover different copyright asset features; the similarity enhancement state is used to generate more refined detection prompts around high-feedback paths; the stability retesting state is used to verify whether the output similarity is repeatable; the evidence consolidation state is used to generate detection evidence packages; and the termination state is used to stop the test when the test budget is exhausted, the risk is low enough, or the evidence is sufficient.

[0082] The system uses the remaining test budget The number of calls is allocated among breadth exploration, similarity enhancement, and stability retesting. If the current highest copyright memory risk score is low, the system prioritizes breadth exploration; if the overall similarity feedback for a certain path continues to rise, the system prioritizes similarity enhancement; if the risk score is close to the high-risk threshold, the system prioritizes stability retesting to form more reliable evidence.

[0083] It should be noted that, as Figure 4 As shown, the system in this embodiment includes a copyright asset access module, a copyright asset feature extraction module, a detection prompt word generation module, a black-box generative model invocation module, a similarity feedback module, a prompt word adaptive optimization module, a stability retesting module, a copyright memory risk scoring module, and a detection evidence package generation module, wherein: The copyright asset access module is used to receive the copyright asset library and asset metadata; The copyright asset feature extraction module is used to extract plot, character relationships, scenes, camera language, music rhythm, keywords, text fragments, and multimodal embedding features; The detection prompt word generation module is used to generate candidate prompt words and a set of candidate detection prompt words based on the characteristics of copyright assets; The black-box generative model invocation module is used to invoke the black-box generative model through the black-box access interface and collect the model's output samples; The similarity feedback module is used to perform multi-dimensional similarity comparisons between the model's output samples and the copyright asset library; The prompt word adaptive optimization module is used to update the next round of detection prompt words based on similarity feedback; The stability retest module is used to perform repeated calls, perturbation calls, and cross-template calls on highly similar paths; The copyright memory risk scoring module is used to calculate the copyright memory risk score; The detection evidence package generation module is used to generate verifiable detection evidence packages.

[0084] Step S15: If the current model test budget has been exhausted, then perform copyright risk detection on the black-box model to be detected based on the similarity feedback corresponding to each round.

[0085] In this embodiment, the system can stop probing based on the following conditions: test budget exhausted; all candidate prompts are below the low-risk threshold; a certain path reaches the high-risk threshold and stability retesting passes; black-box access interface continuously malfunctions and exceeds the retry threshold; user or audit strategy requires manual intervention. The stopping conditions are written into the detection evidence package to prevent auditors from being unable to determine why the test ended.

[0086] In this embodiment, copyright risk detection is performed on the black-box model to be detected based on the similarity feedback corresponding to each round, including: selecting target rounds from each round with a target similarity not less than a preset data similarity threshold based on the similarity feedback; wherein, the target similarity characterizes the similarity between the model output sample and the copyright asset; performing different perturbation processing on the target model prompt words corresponding to the target rounds to obtain perturbed prompt words, and outputting each perturbed prompt word to the black-box model to be detected to obtain perturbed output samples corresponding to each perturbed prompt word; comparing each perturbed output sample with the copyright asset to obtain corresponding perturbed similarity feedback, and determining the risk detection result corresponding to the black-box model to be detected based on each perturbed similarity feedback.

[0087] In other words, to reduce errors caused by model randomness, the system can perform multiple samplings of the same detection prompt word, or use different random seeds, temperatures, generation lengths, image sizes, video durations, and audio durations for perturbation sampling. For black-box generative models that do not allow setting random parameters, the system obtains multiple model output samples through repeated calls and prompt word perturbation.

[0088] Specifically, the system performs stability retesting on highly similar paths. Stability retesting includes repeated calls with the same prompt word, prompt word perturbation calls, parameter perturbation calls, and cross-template calls. If the output under different calling conditions consistently produces content highly similar to the same copyrighted asset, it indicates that the risk is more likely due to model memory than accidental generation.

[0089] Copyright Memory Risk Score It can be represented as: ; in, This represents the highest overall similarity feedback score in multiple rounds of detection; This represents the average score of highly similar rounds; This indicates the output stability score; This represents the hit score for rare features; This indicates penalties for public expression; This represents the weighting coefficient.

[0090] Output stability score It can be represented as: ; in, This indicates the number of times the same highly similar feature of a copyrighted asset is repeatedly hit during perturbation detection; This indicates the total number of stability retests.

[0091] The system allows setting a low-risk threshold. Medium risk threshold and high risk threshold .when When the system outputs a low-risk result, it does so. When the system outputs the results to be reviewed, the system will output the results to be reviewed. When this occurs, the system outputs a high-risk warning and suggests manual review; when At that time, the system outputs a high-risk copyright memory detection evidence package.

[0092] The risk level output by the system can include low risk, pending review, medium risk, high risk, and interface untestable. Interface untestable indicates scenarios where the black-box generative model refuses to respond, the interface is abnormal, or output is limited, making effective probing impossible. This level is different from high risk to prevent misinterpreting untestable conditions as no risk or high risk.

[0093] In addition, after performing copyright risk detection on the black-box model to be detected based on the similarity feedback corresponding to each round, the method further includes: generating a detection evidence package corresponding to the black-box model to be detected; wherein, the detection evidence package includes the asset identifier of the target copyright asset library, the model identifier of the black-box model to be detected, the model prompt word sequence and corresponding hash value corresponding to each round, the similarity feedback corresponding to each round, and the risk detection result of the black-box model to be detected.

[0094] The generation sequence of the detection evidence package in this embodiment is as follows: Figure 5 As shown, this includes the data flow between the copyright asset library, the detection system, the black-box generative model, the similarity feedback module, and the auditing end.

[0095] The evidence package includes: Copyright asset identifier, asset type, licensing status, and summary of copyright asset characteristics.

[0096] Black-box generative model identifier, model version, access interface, call time, and call parameters.

[0097] Each round of detection includes the prompt word, prompt word type, prompt word strategy parameters, and prompt word hash value.

[0098] Each round of model outputs samples, hash values, screenshots, or files.

[0099] Similar segments, similarity dimensions, similarity scores, common expression penalties, and rare feature hit information.

[0100] The iterative trajectory of the prompt words, the results of the stability retest, and the copyright memory risk score.

[0101] Audit summary and recommendations for manual review.

[0102] The evidence package is not a legal infringement conclusion, but rather technical evidence used to assist copyright holders, model purchasers, platform compliance personnel, or auditors in verifying whether black-box generative models have copyright memory risks.

[0103] The structure of the detection evidence package in this embodiment is shown in Table 3: Table 3

[0104] The system can calculate hash values ​​for probe prompts, model output samples, similarity results, and timestamps according to the order of invocation, and add the hash values ​​from the previous round to the hash calculation of the next round, forming a hash chain. Through the hash chain, auditors can verify whether the probe trajectory has been tampered with.

[0105] In one specific implementation, the text-generative model detects copyright memory: the copyright asset database stores chapter summaries, character relationships, key plot points, and several high-information phrases from a novel. The system first generates implicit detection prompts: "Write the beginning of a story about an adopted boy who discovers a family secret after entering a magic school and, together with two companions, investigates clues to a forbidden zone, avoiding the use of any known work titles or character names." The model's output samples are similar to the target copyright asset in terms of character relationships and plot structure, but the text fragment similarity is low. Based on this, the system generates a second round of structure-enhanced detection prompts, adding constraints such as "specific school enrollment scene, three-person group, forbidden zone clues, identity reversal." If multiple rounds of output consistently match specific plot sequences and high-information phrases, the system calculates a copyright memory risk score and generates a detection evidence package.

[0106] In the second specific implementation, the video-generated model detects copyright memory: the copyright asset library stores the scene layout, camera movement, character positions, and key props of a film clip. The system generates scene detection prompts and camera language detection prompts, and calls the black-box video-generated model to obtain candidate video clips. The similarity feedback module finds that the candidate video clips are similar to the copyright assets in terms of camera movement paths, character positions, and prop combinations, but differ in color and dialogue. The system continues to generate perturbation detection prompts, changing the scene description order and character titles. If multiple candidate video clips still stably reproduce the same camera structure, a high-risk copyright memory detection evidence package is output.

[0107] In the third specific implementation, the music generative model detects copyright memory: the copyright asset database stores the rhythm pattern, melody outline, chord progression, and lyric keywords of a musical work. The system first generates rhythm detection prompts. If the output samples have similar rhythms but different melodies, the next round adds melody outlines and chord progression descriptions. If multiple rounds of output samples still hit the same melody outline and rhythm window under different prompt perturbations, the system calculates the copyright memory risk score and saves the audio segment, similarity matrix, and detection prompt sequence.

[0108] In addition, the replacement schemes for the above-mentioned copyright asset feature extraction and detection prompt word generation steps in this embodiment are shown in Table 4 below: Table 4

[0109] Therefore, this application solves the problem of low coverage of manually written prompt words by extracting features from the target copyright asset library and generating corresponding target feature vectors; it solves the problem of unstable prompt word quality by driving the generation of model prompt words through a prompt word generation strategy; and it solves the problem of difficulty in covering a sufficient number of detection paths and improves the accuracy of detection results by obtaining similarity feedback by comparing the current output sample with the copyright assets, updating the prompt word generation strategy based on the feedback, and then jumping to the generation step to form an iterative loop.

[0110] See Figure 6 As shown in the figure, an embodiment of the present invention discloses a copyright risk detection device for a black-box generative model, comprising: The vector generation module 11 is used to obtain the target copyright asset features from the target copyright asset library and generate the target feature vector corresponding to the target copyright asset features; The output sample acquisition module 12 is used to generate the current model prompt word according to the current prompt word generation strategy and the target feature vector, input the current model prompt word into the black box model to be detected using a preset model access interface, and acquire the current output sample of the black box model to be detected; wherein, the prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library; The strategy update module 13 is used to determine whether the current model test budget has been exhausted. If the current model test budget has not been exhausted, the current output sample is compared with the copyright assets in the target copyright asset library to obtain the current similarity feedback, and the current prompt word generation strategy is updated according to the current similarity feedback to obtain the corresponding updated prompt word generation strategy. Step jump module 14 is used to determine the updated prompt word generation strategy as the new current prompt word generation strategy, and jump to the step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector; The risk detection module 15 is used to perform copyright risk detection on the black-box model to be detected based on the similarity feedback corresponding to each round if the current model test budget has been exhausted.

[0111] In some specific embodiments, the vector generation module 11 may specifically include: The feature extraction unit is used to determine the target data type corresponding to the copyright assets in the target copyright asset library, and to extract features from the copyright assets according to the target data type to obtain the features of the target copyright assets; wherein, the data type of the copyright assets includes text type, image type, video type and audio type, and the copyright asset features include plot features, character relationship features, scene features, camera language features, music rhythm features, keyword features, text fragment features and multimodal embedding features.

[0112] In some specific embodiments, the output sample acquisition module 12 may specifically include: The feature subset extraction unit is used to extract the feature subset corresponding to the current round from the target feature vector according to the current prompt word generation strategy; the feature subset is a set of features selected from different copyright asset features contained in the target feature vector; The template selection unit is used to select a target prompt word template corresponding to the model type from a preset template library according to the model type of the black box model to be detected; A hierarchy determination unit is used to determine a target abstract hierarchy from a preset abstract hierarchy; the preset abstract hierarchy is a hierarchy that measures the degree of abstraction of the prompt words; The prompt word determination submodule is used to generate a set of candidate prompt words for the current round based on the feature subset, the target prompt word template and the target abstraction level, and to determine the current model prompt word from the set of candidate prompt words.

[0113] In some specific embodiments, the prompt word determination submodule may specifically include: The prompt word filtering unit is used to filter out a set of prompt words to be deduplicated from the candidate prompt word set according to the priority corresponding to each candidate prompt word in the candidate prompt word set; wherein, the priority is determined according to the target indicators corresponding to each prompt word, and the target indicators include the number and importance of copyright asset features covered by the candidate prompt words, the cost required to call the candidate prompt words, and the benefits of the candidate prompt words determined based on historical feedback results; The prompt word deduplication unit is used to compare the similarity of each prompt word in the set of prompt words to be deduplicated. If the similarity between any two prompt words exceeds a preset prompt word similarity threshold, the prompt word with lower priority among the two prompt words is deleted to obtain the prompt words of the current model.

[0114] In some specific embodiments, the policy update module 13 may specifically include: The similarity comparison unit is used to perform multi-dimensional similarity comparison between the current output sample and the copyright assets in the target copyright asset library to obtain the corresponding comparison results. The similarity comparison process includes text segment similarity comparison, plot structure similarity comparison, character relationship similarity comparison, scene similarity comparison, camera language similarity comparison, music rhythm similarity comparison, keyword similarity comparison, and multimodal embedding feature similarity comparison. The hit rate acquisition unit is used to acquire the hit rate of the common expression features of the current output sample; the hit rate of the common expression features represents the situation in which the current output sample hits the common expression features, and the common expression features are the features that appear more than a preset threshold in the public corpus and do not belong to the unique expression of the copyright asset; The feedback result acquisition unit is used to generate the current similarity feedback based on the comparison result and the hit status of the common expression feature.

[0115] In some specific embodiments, the risk detection module 15 may specifically include: The round selection unit is used to select target rounds from each round based on the similarity feedback, where the target similarity is not less than a preset data similarity threshold; wherein, the target similarity characterization model output sample and the copyright asset are similar. The sample acquisition unit is used to perform different perturbation processing on the target model prompt words corresponding to the target round to obtain perturbation prompt words, and output each perturbation prompt word to the black box model to be detected to obtain perturbation output samples corresponding to each perturbation prompt word. The detection result acquisition unit is used to compare each of the perturbation output samples with the copyright assets to obtain the corresponding perturbation similarity feedback, and to determine the risk detection result corresponding to the black box model to be detected based on each of the perturbation similarity feedback.

[0116] In some specific embodiments, the risk detection module 15 further includes: An evidence package generation unit is used to generate a detection evidence package corresponding to the black box model to be detected; wherein, the detection evidence package includes the asset identifier of the target copyright asset library, the model identifier of the black box model to be detected, the model prompt word sequence and corresponding hash value corresponding to each round, the similarity feedback corresponding to each round, and the risk detection result of the black box model to be detected.

[0117] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0118] Figure 7 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the copyright risk detection method for black-box generative models disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0119] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0120] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0121] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the copyright risk detection method for the black-box generative model executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0122] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned copyright risk detection method for a black-box generative model. The specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0124] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0125] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0126] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0127] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for copyright risk detection of a black-box generative model, characterized in that, include: Obtain the characteristics of the target copyright assets from the target copyright asset library, and generate the target feature vector corresponding to the characteristics of the target copyright assets; The current prompt word is generated based on the current prompt word generation strategy and the target feature vector. The current model prompt word is input into the black box model to be detected using a preset model access interface, and the current output sample of the black box model to be detected is obtained. The prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library. Determine whether the current model test budget has been exhausted. If the current model test budget has not been exhausted, compare the current output sample with the copyright assets in the target copyright asset library to obtain the current similarity feedback, and update the current prompt word generation strategy according to the current similarity feedback to obtain the corresponding updated prompt word generation strategy. The updated prompt word generation strategy is determined as the new current prompt word generation strategy, and the process jumps to the step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector; If the current model test budget has been exhausted, then the black-box model to be detected will be subjected to copyright risk detection based on the similarity feedback corresponding to each round. The step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector includes: According to the current prompt word generation strategy, the feature subset corresponding to the current round is extracted from the target feature vector; the feature subset is a set of features selected from different copyright asset features contained in the target feature vector; Based on the model type of the black-box model to be detected, select the target prompt word template corresponding to the model type from the preset template library; The target abstraction level is determined from the preset abstraction levels; the preset abstraction levels are the levels that measure the degree of abstraction of the prompt words. Based on the feature subset, the target prompt word template, and the target abstraction level, a candidate prompt word set corresponding to the current round is generated, and the current model prompt word is determined from the candidate prompt word set.

2. The copyright risk detection method for black-box generative models according to claim 1, characterized in that, The process of obtaining the characteristics of the target copyright assets from the target copyright asset library includes: The target data type corresponding to the copyright assets in the target copyright asset library is determined, and feature extraction is performed on the copyright assets according to the target data type to obtain the features of the target copyright assets; wherein, the data type of the copyright assets includes text type, image type, video type and audio type, and the copyright asset features include plot features, character relationship features, scene features, camera language features, music rhythm features, keyword features, text fragment features and multimodal embedding features.

3. The copyright risk detection method for black-box generative models according to claim 1, characterized in that, Determining the current model prompt word from the candidate prompt word set includes: A set of deduplicated prompts is selected from the candidate prompts set based on the priority of each candidate prompt. The priority is determined based on the target indicators corresponding to each prompt. The target indicators include the number and importance of the copyright asset features covered by the candidate prompts, the cost required to call the candidate prompts, and the benefits of the candidate prompts determined based on historical feedback results. The similarity of each prompt word in the set of prompt words to be deduplicated is compared. If the similarity between any two prompt words exceeds a preset prompt word similarity threshold, the prompt word with lower priority among the two prompt words is deleted to obtain the current model prompt word.

4. The copyright risk detection method for black-box generative models according to claim 1, characterized in that, The step of comparing the current output sample with the copyright assets in the target copyright asset library to obtain current similarity feedback includes: The current output sample is compared with the copyright assets in the target copyright asset library in a multidimensional similarity comparison to obtain the corresponding comparison results; wherein, the similarity comparison process includes text segment similarity comparison, plot structure similarity comparison, character relationship similarity comparison, scene similarity comparison, camera language similarity comparison, music rhythm similarity comparison, keyword similarity comparison, and multimodal embedding feature similarity comparison. Obtain the common expression feature hit status of the current output sample; the common expression feature hit status represents the situation in which the current output sample hits a common expression feature, wherein the common expression feature is a feature that appears more than a preset threshold in the public corpus and does not belong to the unique expression of the copyright asset; The current similarity feedback is generated based on the comparison results and the hit rate of the common expression features.

5. The copyright risk detection method for black-box generative models according to claim 1, characterized in that, The copyright risk detection of the black-box model to be detected based on the similarity feedback corresponding to each round includes: Based on the similarity feedback, target rounds with a target similarity not less than a preset data similarity threshold are selected from each round; wherein, the target similarity characterizes the similarity between the model output sample and the copyright asset; Different perturbation processes are applied to the target model prompt words corresponding to the target round to obtain perturbation prompt words, and each perturbation prompt word is output to the black box model to be detected to obtain perturbation output samples corresponding to each perturbation prompt word. Each of the perturbation output samples is compared with the copyright asset to obtain the corresponding perturbation similarity feedback, and the risk detection result corresponding to the black box model to be detected is determined based on the perturbation similarity feedback.

6. The copyright risk detection method for black-box generative models according to any one of claims 1 to 5, characterized in that, After performing copyright risk detection on the black-box model to be detected based on the similarity feedback corresponding to each round, the method further includes: Generate a detection evidence package corresponding to the black box model to be detected; wherein, the detection evidence package includes the asset identifier of the target copyright asset library, the model identifier of the black box model to be detected, the model prompt word sequence and corresponding hash value corresponding to each round, the similarity feedback corresponding to each round, and the risk detection result of the black box model to be detected.

7. A copyright risk detection device for a black-box generative model, characterized in that, include: The vector generation module is used to obtain the characteristics of the target copyright assets from the target copyright asset library and generate the target feature vector corresponding to the characteristics of the target copyright assets; The output sample acquisition module is used to generate the current model prompt word according to the current prompt word generation strategy and the target feature vector, input the current model prompt word into the black box model to be detected using a preset model access interface, and acquire the current output sample of the black box model to be detected; wherein, the prompt word generation strategy is a strategy that makes the model output converge to the copyright assets in the target copyright asset library; The strategy update module is used to determine whether the current model test budget has been exhausted. If the current model test budget has not been exhausted, the current output sample is compared with the copyright assets in the target copyright asset library to obtain the current similarity feedback, and the current prompt word generation strategy is updated according to the current similarity feedback to obtain the corresponding updated prompt word generation strategy. The step jump module is used to determine the updated prompt word generation strategy as the new current prompt word generation strategy, and jump to the step of generating the current model prompt word based on the current prompt word generation strategy and the target feature vector; The risk detection module is used to perform copyright risk detection on the black-box model to be tested based on the similarity feedback corresponding to each round if the current model test budget has been exhausted. The output sample acquisition module includes: The feature subset extraction unit is used to extract the feature subset corresponding to the current round from the target feature vector according to the current prompt word generation strategy; the feature subset is a set of several features selected from different copyright asset features contained in the target feature vector; The template selection unit is used to select a target prompt word template corresponding to the model type from a preset template library according to the model type of the black box model to be detected; A hierarchy determination unit is used to determine a target abstract hierarchy from a preset abstract hierarchy; the preset abstract hierarchy is a hierarchy that measures the degree of abstraction of the prompt words; The prompt word determination submodule is used to generate a set of candidate prompt words for the current round based on the feature subset, the target prompt word template and the target abstraction level, and to determine the current model prompt word from the set of candidate prompt words.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the copyright risk detection method for a black-box generative model as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the copyright risk detection method for a black-box generative model as described in any one of claims 1 to 6.

Images