Method for verifying copyright of neural network model based on watermark embedding

Abstract

The present application relates to the field of digital watermark embedding, and particularly relates to a copyright verification method and device based on a neural network model, an equipment and a storage medium, comprising: obtaining a watermark image dataset; extracting a plurality of watermark images in the watermark image dataset as an original trigger set, performing singular value decomposition processing on the original trigger set to obtain trigger set samples; extracting classification label data corresponding to the watermark images in the original trigger set, and corresponding the classification label data with the trigger set samples one by one to obtain a trigger set; inputting the watermark image dataset and the trigger set into a neural network model to be embedded with a watermark, training the neural network model for several times to obtain a target neural network model; in response to a model copyright verification instruction of a user, inputting the trigger set into the target neural network model to obtain a classification result, and according to the watermark images in the trigger set, the corresponding classification label data and the classification result, obtaining a verification result output by the target neural network model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital watermark embedding, and in particular to a copyright verification method and device based on a neural network model with watermark embedding, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of the field of deep learning, neural network models are increasingly used among enterprises. However, attackers tamper with other people's neural network models to make profits by illegally authorizing others, which infringes the copyright of the model creator and causes huge economic losses to the model creator. Therefore, it is necessary to protect the copyright of the neural network model.

[0003] With the wide use of digital watermark technology, the neural network model can be protected by embedding a watermark in the neural network model. However, the existing neural network watermark embedding method affects the function of the neural network model itself, resulting in problems such as decreased model task classification accuracy, insufficient watermark generality or decreased robustness. SUMMARY

[0004] Therefore, the purpose of the present application is to provide a copyright verification method and device based on a neural network model with watermark embedding, equipment and a storage medium. By singular value decomposition of the watermark image dataset, a trigger set is constructed, and a watermark embedding framework is formed according to the watermark image dataset and the trigger set, which improves the efficiency of watermark embedding, reduces the impact of the watermark on the function of the neural network model itself, and makes the embedding effect more stable. The watermark is not easy to be removed or forged, which improves the generality and robustness of the watermark.

[0005] In a first aspect, the present application provides a copyright verification method based on a neural network model with watermark embedding, comprising the following steps:

[0006] Obtain a watermark image dataset, wherein the watermark image dataset includes a plurality of watermark images, and each watermark image has corresponding classification label data;

[0007] Extract a plurality of watermark images in the watermark image dataset as an original trigger set, perform singular value decomposition on the original trigger set, and obtain a trigger set sample;

[0008] Extract the classification label data corresponding to the watermark images in the original trigger set, and correspond the classification label data and the trigger set sample one by one to obtain a trigger set;

[0009] Input the watermark image dataset and the trigger set into a neural network model to be embedded with a watermark, train the neural network model for several times, and obtain a target neural network model;

[0010] In response to a model copyright verification instruction of a user, the model copyright verification instruction including the trigger set, inputting the trigger set into the target neural network model, obtaining a classification result, and according to the watermark image in the trigger set, the corresponding classification label data and the classification result, obtaining a verification result output by the target neural network model.

[0011] In a second aspect, the embodiments of the present application provide a device for copyright verification of a neural network model based on watermark embedding, comprising:

[0012] A obtaining module is configured to obtain a watermark image dataset, wherein the watermark image dataset includes a plurality of watermark images, and each watermark image has corresponding classification label data.

[0013] A first extracting module is configured to extract a plurality of watermark images in the watermark image dataset as an original trigger set, perform singular value decomposition processing on the original trigger set, and obtain a trigger set sample.

[0014] A second extracting module is configured to extract the classification label data corresponding to the watermark images in the original trigger set, and correspond the classification label data with the trigger set sample one by one, and obtain a trigger set.

[0015] A training-embedding module is configured to input the watermark image dataset and the trigger set into a neural network model to be embedded with a watermark, perform a plurality of times of training on the neural network model, and obtain a target neural network model.

[0016] A verification module is configured to, in response to a model copyright verification instruction of a user, input the trigger set into the target neural network model, obtain a classification result, and according to the watermark image in the trigger set, the corresponding classification label data and the classification result, obtain a verification result output by the target neural network model.

[0017] In a third aspect, the embodiments of the present application provide a device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the copyright verification method of the neural network model based on watermark embedding as described in the first aspect are implemented.

[0018] In a fourth aspect, the embodiments of the present application provide a storage medium, the storage medium storing a computer program, when the computer program is executed by a processor, the steps of the copyright verification method of the neural network model based on watermark embedding as described in the first aspect are implemented.

[0019] In this embodiment, by performing singular value decomposition on the watermark image dataset and constructing a trigger set, the watermark embedding framework composed of the watermark image dataset and the trigger set is constructed, which improves the efficiency of watermark embedding, reduces the impact of the watermark on the function of the neural network model itself, and makes the embedding effect more stable. The watermark is not easy to erase or forge, thus improving the universality and robustness of the watermark.

[0020] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0021] Figure 1 A flowchart illustrating a copyright verification method based on a neural network model with watermark embedding, provided in one embodiment of this application;

[0022] Figure 2 This is a schematic diagram of a process for obtaining a trigger set according to an embodiment of this application;

[0023] Figure 3 A schematic diagram illustrating the process of obtaining a target neural network model according to one embodiment of this application;

[0024] Figure 4 This is a flowchart illustrating step S4 of a copyright verification method based on a watermark-embedded neural network model provided in one embodiment of this application.

[0025] Figure 5 A schematic diagram illustrating the process of model copyright verification provided in one embodiment of this application;

[0026] Figure 6 This is a flowchart illustrating step S5 of a copyright verification method based on a watermark-embedded neural network model provided in one embodiment of this application.

[0027] Figure 7 A schematic diagram of the structure of a copyright verification device based on a neural network model with watermark embedding provided in one embodiment of this application;

[0028] Figure 8 This is a schematic diagram of the structure of a device provided in one embodiment of this application. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0032] Please see Figure 1 , Figure 1 This is a flowchart illustrating a copyright verification method based on a neural network model with watermark embedding, provided in one embodiment of this application. The method includes the following steps:

[0033] S1: Obtain the watermarked image dataset, wherein the watermarked image dataset includes several watermarked images, and each watermarked image has corresponding classification label data.

[0034] The execution subject of the copyright verification method based on the watermark embedding neural network model is the copyright verification device of the neural network model (hereinafter referred to as the verification device). In an optional embodiment, the verification device can be a computer device, a server, or a server cluster composed of multiple computer devices.

[0035] The watermarked image dataset includes several watermarked images, comprising several watermark training images and watermark verification images. Each watermarked image contains subject contour information, and each watermarked image has classification label data corresponding to the subject contour information. However, the human eye and a pre-trained neural network model cannot accurately classify the label of the watermarked image. For example, if the subject contour information of the watermarked image is a cat, its classification label data is "cat".

[0036] In this embodiment, the verification device acquires a watermark image dataset input by the user, which is used to embed the watermark into the neural network model.

[0037] Please see Figure 2 , Figure 2 This is a schematic diagram of the process for obtaining a trigger set according to one embodiment of this application.

[0038] S2: Extract several watermarked images from the watermarked image dataset as the original trigger set, perform singular value decomposition on the original trigger set, and obtain trigger set samples.

[0039] Singular values ​​are a concept in matrix theory, generally obtained through the singular value decomposition theorem. Let X be an m*n matrix, q = min(m,n). The arithmetic square roots of the q non-negative eigenvalues ​​of A*A are called the singular values ​​of A. Singular value decomposition is an important matrix decomposition method in linear algebra and matrix theory, applicable to fields such as signal processing and statistics.

[0040] Generally speaking, singular values ​​correspond to the main information in a matrix, so singular value decomposition can be used for data analysis to extract the main information from the matrix.

[0041] The singular value decomposition algorithm is as follows:

[0042] A = U × S × V T

[0043] In the formula, A represents the singular value decomposition sample data of the watermark image; S is a matrix whose diagonal elements are singular value elements and whose other elements are all 0; U is the expression for A. T The matrix constructed from the eigenvectors of A; V is a matrix with respect to AA. T The matrix constructed from the eigenvectors.

[0044] In this embodiment, the verification device extracts several watermark images from the watermark image dataset as the original trigger set. Based on a preset singular value decomposition algorithm, it performs singular value decomposition on the watermark images in the original trigger set to obtain singular value decomposition sample data. Then, based on the singular value decomposition sample data, it constructs the trigger set sample A', where A' = U × S1 × V. T +U×S2×V T +……+U×S n ×V T , {S i |i∈n}.

[0045] In the formula, {S i |i∈n} is a matrix whose diagonal elements are singular values ​​and all other elements are 0. The maximum value of n is the number of pixels in the width of sample A in the watermark image.

[0046] In an optional embodiment, the verification device selects the first two trigger set samples obtained after processing by the singular value decomposition algorithm, which are optimal because they retain the main contour information of the image and the pre-trained and clean neural network model cannot identify its correct classification label. Since the training of the trigger set should not interfere with the original classification task, the number of watermarked images in the trigger set samples is generally controlled to within 200.

[0047] S3: Extract the classification label data corresponding to the watermark images in the original trigger set, and match the classification label data with the trigger set samples one by one to obtain the trigger set.

[0048] In this embodiment, the verification device extracts the classification label data corresponding to the watermark images in the original trigger set, and matches the classification label data with the trigger set samples one by one to obtain the trigger set.

[0049] Please see Figure 3 , Figure 3 This is a schematic diagram of the process for obtaining a target neural network model according to one embodiment of this application.

[0050] S4: Input the watermark image dataset and the trigger set into the neural network model to be embedded with the watermark, and train the neural network model several times to obtain the target neural network model.

[0051] The neural network model to be embedded with the watermark is a convolutional neural network structure, including ResNet series neural network structures, VGG series neural network structures, LeNet series neural network structures, and AlexNet series neural network structures, etc.

[0052] Because ResNet and VGG series neural network structures are deeper and more complex than LeNet and AlexNet series neural network structures, in an optional embodiment, when the neural network model to be embedded with a watermark is a ResNet or VGG series neural network structure, the verification device usually adopts a fine-tuning training method. That is, by modifying a small number of weight parameters of the ResNet or VGG series neural network structure, the neural network model to be embedded with a watermark is trained to ensure that the watermark embedding effect does not easily affect the structure and function of the neural network model itself.

[0053] In another optional embodiment, when the neural network model to be embedded with the watermark is a LeNet series or AlexNet series neural network structure, the verification device usually adopts the method of training the model from scratch, that is, by modifying all the weight parameters of the LeNet series or AlexNet series neural network structure, the neural network model to be embedded with the watermark is trained to ensure that the watermark embedding effect is more stable.

[0054] The verification device obtains the loss data output by the neural network model based on the watermark image dataset, the trigger set, and the corresponding loss calculation algorithm in the neural network model. Based on the loss data, the neural network model is trained to obtain a trained neural network model. In an optional embodiment, the verification device mixes the watermark image dataset and the trigger set at a ratio of 16:1 and trains the neural network model to obtain a trained neural network model.

[0055] Please see Figure 4 , Figure 4 This is a flowchart illustrating step S4 of a copyright verification method based on a watermark-embedded neural network model provided in one embodiment of this application, including steps S401 to S403:

[0056] S401: Based on the watermarked image dataset and the first loss calculation algorithm, obtain the first loss data related to the watermarked image dataset.

[0057] The first loss calculation algorithm is typically the Cross Entropy Error Function, as follows:

[0058]

[0059] In the formula, Loss1 is the first loss data; I is the number of classification task categories in the watermarked image dataset; y i p represents an indicator variable representing category i; i This represents the predicted probability that the label belongs to category i.

[0060] In this embodiment, the verification device inputs the watermark image dataset into the neural network model, obtains the prediction result of the neural network model for the watermark image dataset, obtains a first prediction probability based on the prediction result and the classification label data corresponding to the watermark images in the watermark image dataset, and obtains a first loss data related to the watermark image dataset based on the first prediction probability and the first loss calculation algorithm.

[0061] S402: Based on the trigger set and the second loss calculation algorithm, obtain the second loss data related to the trigger set.

[0062] The second loss calculation algorithm is as follows:

[0063]

[0064] In the formula, Loss1 is the second loss data; M is the number of classification task categories in the trigger set; y i p represents an indicator variable representing category i; i This represents the predicted probability that the predicted label belongs to category i; k is the watermark embedding weight parameter, used to prevent deviation from the model decision boundary.

[0065] In this embodiment, the verification device inputs the trigger set into the neural network model, obtains the prediction result of the neural network model for the trigger set, obtains a second prediction probability based on the prediction result and the classification label data corresponding to the watermark image in the trigger set, and obtains a second loss data related to the trigger set based on the second prediction probability and the second loss calculation algorithm.

[0066] S403: Superimpose the first loss data and the second loss data to obtain the total loss data of the neural network model. Based on the loss data, train the neural network model several times until the neural network model converges to obtain the target neural network model.

[0067] In this embodiment, the verification device superimposes the first loss data and the second loss data to obtain the total loss data of the neural network model, wherein the total loss data is:

[0068] Loss all =Loss1+Loss2

[0069] The acquired total loss data is input into the corresponding optimizer of the neural network model, and the neural network model is trained several times until the neural network model converges, thereby obtaining the target neural network model.

[0070] Please see Figure 5 , Figure 5 This is a schematic diagram of a model copyright verification process provided in one embodiment of this application.

[0071] S5: In response to the user's model copyright verification instruction, the model copyright verification instruction includes the trigger set, the trigger set is input into the target neural network model, the classification result is obtained, and the verification result output by the target neural network model is obtained based on the watermark image, the corresponding classification label data and the classification result in the trigger set.

[0072] In this embodiment, the verification device receives and responds to the model copyright verification instruction sent by the user. The verification device obtains the trigger set in the model copyright verification instruction and inputs the trigger set into the target neural network model. Based on the watermark image in the trigger set and the target neural network model, the device obtains the classification result output by the target neural network model. Based on the watermark image in the trigger set, the corresponding classification label, and the classification result, the device obtains the verification result output by the target neural network model.

[0073] Please see Figure 6 , Figure 6 This is a flowchart illustrating step S5 of a copyright verification method based on a watermark-embedded neural network model provided in one embodiment of this application, including steps S501 to S502:

[0074] S501: Obtain error data based on the classification result, the classification label data corresponding to the watermark images in the trigger set, and the classification error algorithm.

[0075] The classification error algorithm is as follows:

[0076]

[0077] In the formula, E is the output result of the classification error algorithm; K is the total number of watermarked images in the trigger set, and Mp(x) i ) represents the classification result; z i Δ represents the classification label data corresponding to the watermarked images in the trigger set; Δ is a binary function.

[0078] In this embodiment, the verification device inputs the classification result and the classification label data corresponding to the watermark image in the trigger set into the classification error algorithm for comparison, and obtains several output results E, wherein the output results include 1 and 0, when Mp(x i )≠z i If the condition is met, the output result is 1; otherwise, the output result is 0. Based on the output result E, the proportion of the output result 1 in all output results is obtained and used as the error data.

[0079] S502: Based on the error data and the preset error threshold, if the error data is less than the error threshold, a successful verification result is obtained; if the error data is greater than the error threshold, a failed verification result is obtained.

[0080] S6: In response to the user's model copyright verification instruction, the model copyright verification instruction includes the trigger set, the trigger set is input into the target neural network model, the classification result is obtained, and the verification result output by the target neural network model is obtained based on the classification label data corresponding to the watermark image in the trigger set and the classification result.

[0081] In this embodiment, the verification device obtains the model copyright verification instruction sent by the user, inputs the trigger set in the model copyright verification instruction into the target neural network model, obtains the classification result output by the target neural network model, compares the classification label data corresponding to the watermark image in the trigger set with the classification result, if the classification label data is the same as the classification result, the verification is successful; if they are different, the verification fails, and the verification result output by the target neural network model is obtained.

[0082] In this embodiment, the verification device is preset with an error threshold. After the verification device acquires the error data, it compares the error data with the error threshold. If the error data is less than the error threshold, a verification success result is obtained; if the error data is greater than the error threshold, a verification failure result is obtained.

[0083] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of a copyright verification device based on a watermark-embedded neural network model according to an embodiment of this application. This device can be implemented in whole or in part through software, hardware, or a combination of both. The device 7 includes:

[0084] The acquisition module 71 is used to acquire a watermark image dataset, wherein the watermark image dataset includes several watermark images, and each watermark image has corresponding classification label data.

[0085] The first extraction module 72 is used to extract several watermark images from the watermark image dataset as the original trigger set, perform singular value decomposition on the original trigger set, and obtain trigger set samples.

[0086] The second extraction module 73 is used to extract the classification label data corresponding to the watermark images in the original trigger set, and to match the classification label data with the trigger set samples one by one to obtain the trigger set;

[0087] The training-embedding module 74 is used to input the watermark image dataset and the trigger set into the neural network model to be embedded with the watermark, and to train the neural network model several times to obtain the target neural network model.

[0088] The verification module 75 is used to respond to the user's model copyright verification command, the model copyright verification command including the trigger set, input the trigger set into the target neural network model, obtain the classification result, and obtain the verification result output by the target neural network model based on the watermark image, the corresponding classification label data and the classification result in the trigger set.

[0089] In this embodiment, an acquisition module acquires a watermark image dataset, wherein the watermark image dataset includes several watermark images, each with corresponding classification label data; a first extraction module extracts several watermark images from the watermark image dataset as an original trigger set, performs singular value decomposition on the original trigger set to obtain trigger set samples; a second extraction module inputs the watermark image dataset and the trigger set into a neural network model to be embedded with the watermark, trains the neural network model several times to obtain a target neural network model; a verification module, in response to a user's model copyright verification command, which includes the trigger set, inputs the trigger set into the target neural network model to obtain classification results, and obtains the verification result output by the target neural network model based on the watermark images in the trigger set, the corresponding classification label data, and the classification results. This application improves the efficiency of watermark embedding by constructing a watermark framework consisting of a watermark image dataset and a trigger set, while reducing the impact of the watermark on the neural network model itself and its functionality. Moreover, the embedding effect is more stable, and the watermark is not easily erased or forged, thus improving the universality and robustness of the watermark.

[0090] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of a device provided in one embodiment of this application. The device 5 includes: a processor 81, a memory 82, and a computer program 83 stored in the memory 82 and executable on the processor 81; the computer device can store multiple instructions, which are adapted to be loaded and executed by the processor 81. Figures 1 to 5 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 1 , Figure 4 as well as Figure 6 The specific details of the illustrated embodiments will not be elaborated here.

[0091] The processor 81 may include one or more processing cores. The processor 81 connects to various parts of the server using various interfaces and lines. It executes various functions and processes data of the copyright verification device 7 based on a watermark-embedded neural network model by running or executing instructions, programs, code sets, or instruction sets stored in memory 82, and by calling data from memory 82. Optionally, the processor 81 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 81 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 81.

[0092] The memory 82 may include random access memory (RAM) or read-only memory. Optionally, the memory 82 may include a non-transitory computer-readable storage medium. The memory 82 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 82 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 82 may also be at least one storage device located remotely from the aforementioned processor 81.

[0093] This application embodiment also provides a storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figure 1 , Figure 4 as well as Figure 6 The method steps of the illustrated embodiment can be found in the following documentation for detailed execution. Figure 1 , Figure 4 as well asFigure 6 The specific details of the illustrated embodiments will not be elaborated here.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0096] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0097] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0098] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0101] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. A copyright verification method based on a neural network model with watermark embedding, characterized in that, Includes the following steps: Obtain a watermarked image dataset, wherein the watermarked image dataset includes several watermarked images, and each watermarked image has corresponding classification label data; Several watermarked images are extracted from the watermarked image dataset as the original trigger set. Singular value decomposition (SVD) is performed on the original trigger set according to a preset SVD algorithm to obtain SVD sample data. Based on the SVD sample data, a trigger set sample is constructed. The SVD algorithm is as follows: A=U×S×V T In the formula, A represents the singular value decomposition sample data of the watermark image; S is a matrix whose diagonal elements are singular value elements and whose other elements are all 0; U is the expression for A. T The matrix constructed from the eigenvectors of A; V is a matrix with respect to AA. T The matrix constructed from the eigenvectors; Extract the classification label data corresponding to the watermark images in the original trigger set, and match the classification label data with the trigger set samples one by one to obtain the trigger set; The watermarked image dataset and the trigger set are input into the neural network model to which the watermark is to be embedded. Based on the watermarked image dataset and the first loss calculation algorithm, first loss data related to the watermarked image dataset is obtained, wherein the first loss calculation algorithm is: In the formula, Loss 1 represents the first loss data; I The number of classification task categories in the watermarked image dataset; Indicate category i Indicator variables; The predicted label belongs to the category. i The predicted probability; Based on the trigger set and the second loss calculation algorithm, second loss data related to the trigger set is obtained, wherein the second loss calculation algorithm is: In the formula, Loss 2 represents the second loss data; M The number of categories of the triggering centralized classification tasks; Indicate category i Indicator variables; The predicted label belongs to the category. i The predicted probability; k Weight parameters are embedded in the watermark; The first loss data and the second loss data are superimposed to obtain the total loss data of the neural network model. Based on the loss data, the neural network model is trained several times until the neural network model converges to obtain the target neural network model. In response to a user's model copyright verification command, the model copyright verification command includes the trigger set. The trigger set is input into the target neural network model to obtain a classification result. Based on the watermark image, the corresponding classification label data, and the classification result in the trigger set, the verification result output by the target neural network model is obtained.

2. The copyright verification method based on a neural network model with watermark embedding according to claim 1, characterized in that, The steps of responding to a user's model copyright verification command, wherein the model copyright verification command includes the trigger set, inputting the trigger set into the target neural network model, obtaining classification results, and obtaining the verification result output by the target neural network model based on the watermark image, corresponding classification label data, and classification results in the trigger set, include: Based on the classification results, the classification label data corresponding to the watermark images in the trigger set, and the classification error algorithm, error data is obtained, wherein the classification error algorithm is: In the formula, E The output of the classification error algorithm; K The total number of watermarked images in the trigger set. The classification result; The classification label data corresponding to the watermarked images in the trigger set; It is a binary function; Based on the error data and a preset error threshold, if the error data is less than the error threshold, a successful verification result is obtained; if the error data is greater than the error threshold, a failed verification result is obtained.

3. A copyright verification device based on a neural network model with watermark embedding, characterized in that, include: The acquisition module is used to acquire a watermark image dataset, wherein the watermark image dataset includes several watermark images, and each watermark image has corresponding classification label data. The first extraction module is used to extract several watermark images from the watermark image dataset as an original trigger set, perform singular value decomposition (SVD) processing on the original trigger set according to a preset singular value decomposition algorithm, obtain singular value decomposition sample data, and construct a trigger set sample based on the singular value decomposition sample data. The singular value decomposition algorithm is as follows: A=U×S×V T In the formula, A represents the singular value decomposition sample data of the watermark image; S is a matrix whose diagonal elements are singular value elements and whose other elements are all 0; U is the expression for A. T The matrix constructed from the eigenvectors of A; V is a matrix with respect to AA. T The matrix constructed from the eigenvectors; The second extraction module is used to extract the classification label data corresponding to the watermark images in the original trigger set, and to match the classification label data with the trigger set samples one by one to obtain the trigger set; The training-embedding module is used to input the watermark image dataset and the trigger set into the neural network model to which the watermark is to be embedded, and to obtain first loss data related to the watermark image dataset based on the watermark image dataset and a first loss calculation algorithm, wherein the first loss calculation algorithm is: In the formula, Loss 1 represents the first loss data; I The number of classification task categories in the watermarked image dataset; Indicate category i Indicator variables; The predicted label belongs to the category. i The predicted probability; Based on the trigger set and the second loss calculation algorithm, second loss data related to the trigger set is obtained, wherein the second loss calculation algorithm is: In the formula, Loss 2 represents the second loss data; M The number of categories of the triggering centralized classification tasks; Indicate category i Indicator variables; The predicted label belongs to the category. i The predicted probability; k Weight parameters are embedded in the watermark; The first loss data and the second loss data are superimposed to obtain the total loss data of the neural network model. Based on the loss data, the neural network model is trained several times until the neural network model converges to obtain the target neural network model. The verification module is used to respond to the user's model copyright verification command, which includes the trigger set. The trigger set is input into the target neural network model to obtain the classification result. Based on the watermark image, the corresponding classification label data, and the classification result in the trigger set, the verification result output by the target neural network model is obtained.

4. A device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor; the computer program, when executed by the processor, implements the steps of the copyright verification apparatus based on a watermark-embedded neural network model as described in any one of claims 1 or 2.

5. A storage medium, characterized in that: The storage medium stores a computer program that, when executed by a processor, implements the steps of the copyright verification device based on a watermark-embedded neural network model as described in claim 1 or 2.

Images