A model ownership verification method and device, a storage medium and an equipment

Abstract

The specification discloses a model ownership verification method and device, a storage medium and equipment. An original sample is input into a target tracking model to obtain first features output by a feature extraction subnetwork of the model. The first features are input into a target tracking subnetwork of the model to obtain a tracking result. A specified feature is added to the original sample to obtain an adjusted sample, which is input into the target tracking model to obtain second features output by the feature extraction subnetwork. The target tracking model is trained with an optimization target that the difference between the tracking result and the label of the original sample is minimum and the difference between the first features and the second features is maximum. When a verification request for a to-be-verified model is received, the original sample is input into the to-be-verified model to obtain a first tracking result, and the adjusted sample of the original sample is input into the to-be-verified model to obtain a second tracking result. Whether the to-be-verified model is obtained based on the target tracking model is determined according to the difference between the first tracking result and the second tracking result.

Description

Technical Field

[0001] This specification relates to the field of machine learning technology, and in particular to a method, apparatus, storage medium, and device for verifying model ownership. Background Technology

[0002] Currently, users are increasingly concerned about their privacy and are paying more attention to the copyright protection of models. Therefore, research on model ownership verification methods is essential in order to identify models copyrighted by users that have been stolen and to protect their rights.

[0003] However, current model ownership verification methods are mostly designed for classification models. There are no suitable verification methods for other models, such as visual object tracking (VOT) models.

[0004] Therefore, this specification provides a method for verifying model ownership applicable to VOT models. Summary of the Invention

[0005] This specification provides a model ownership verification method, apparatus, storage medium, and device to at least partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a method for verifying model ownership. The target tracking model consists of a feature extraction subnetwork and a target tracking subnetwork. The method includes:

[0008] Determine the original samples for training the target tracking model;

[0009] Add specified features to the original sample to obtain the adjusted sample;

[0010] The original sample is input into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet. The first feature is then input into the target tracking subnet to obtain the tracking result.

[0011] The adjusted sample is input into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet;

[0012] The target tracking model is trained with the optimization objective of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature;

[0013] When a verification request for the model to be verified is received, the original sample is input into the model to be verified to obtain a first tracking result, and the adjusted sample is input into the model to be verified to obtain a second tracking result;

[0014] Based on the difference between the first tracking result and the second tracking result, it is determined whether the model to be verified is obtained based on the target tracking model.

[0015] This specification provides a model ownership verification device. The target tracking model consists of a feature extraction subnetwork and a target tracking subnetwork. The device includes:

[0016] The first module for sample determination is used to determine the original samples for training the target tracking model;

[0017] The second module for sample determination is used to add specified features to the original sample to obtain an adjusted sample.

[0018] The first input module is used to input the original sample into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet, and input the first feature into the target tracking subnet to obtain the tracking result;

[0019] The second input module is used to input the adjusted sample into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet;

[0020] The training module is used to train the target tracking model with the optimization objectives of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature;

[0021] The receiving module is configured to, when receiving a verification request for the model to be verified, input the original sample into the model to be verified to obtain a first tracking result, and input the adjusted sample into the model to be verified to obtain a second tracking result;

[0022] The verification module is used to determine whether the model to be verified is obtained based on the target tracking model, based on the difference between the first tracking result and the second tracking result.

[0023] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described model ownership verification method.

[0024] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described model ownership verification method.

[0025] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0026] In the aforementioned model ownership verification method, the original samples for training the target tracking model are determined and input into the target tracking model to obtain the first feature of the original samples output by the model's feature extraction subnet. This first feature is then input into the model's target tracking subnet to obtain the tracking result. Specified features are added to the original samples to obtain adjusted samples, which are then input into the target tracking model to obtain the second feature of the adjusted samples output by the feature extraction subnet. The target tracking model is trained with the optimization objective of minimizing the difference between the tracking result and the original sample annotations, and maximizing the difference between the first and second features. When a verification request for the model to be verified is received, the original samples are input into the model to be verified to obtain the first tracking result, and the adjusted samples are input into the model to be verified to obtain the second tracking result. Based on the difference between the first and second tracking results, it is determined whether the model to be verified is based on the target tracking model. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0028] Figure 1 This is a flowchart illustrating one of the model ownership verification methods described in this specification.

[0029] Figure 2 This is a schematic diagram of a target tracking model provided in this specification;

[0030] Figure 3 This is a schematic diagram of a target tracking model provided in this specification;

[0031] Figure 4 This is a schematic diagram of a model ownership verification device provided in this specification;

[0032] Figure 5 This is a schematic diagram of an electronic device provided in this specification. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0034] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0035] In this specification, the target tracking model consists of a feature extraction subnet and a target tracking subnet. The target tracking model is the model whose copyright needs to be protected. This specification uses a model ownership verification method to verify whether a suspicious model held by another party is based on this target tracking model; for example, whether it has misappropriated the parameters of the target tracking model to obtain the suspicious model. This is to determine whether the holder of the suspicious model infringes on the copyright of the user holding the target tracking model.

[0036] Figure 1 This is a flowchart illustrating one model ownership verification method described in this specification, which specifically includes the following steps:

[0037] In this specification, steps S100 to S108 are the process of training the target tracking model. Steps S110 to S112 are the process of verifying the ownership of the model to be verified. The model to be verified is, i.e., the suspected model that needs to undergo ownership verification.

[0038] S100: Determine the original samples for training the target tracking model.

[0039] In this specification, the model ownership verification method can be executed by the server.

[0040] To determine whether a target tracking model is actually derived from a target tracking model (i.e., to verify ownership) based on the output of the target tracking model when its parameters are stolen from a target tracking model copyrighted by the user corresponding to the server, the target tracking model can be trained using both training samples with and without specified features. This ensures that the trained target tracking model outputs differently depending on whether the input has the specified features or not.

[0041] Therefore, based on the difference between the output of the model to be verified for input with the specified features added and input without the specified features added, it can be verified whether the model to be verified is based on the target tracking model.

[0042] In this specification, training samples without specified features are used as the original samples.

[0043] Therefore, in this specification, the server can determine the original samples for training the target tracking model when training the target tracking model.

[0044] S102: Add specified features to the original sample to obtain the adjusted sample.

[0045] Once the original samples are determined, the server can add specified features to the original samples to obtain training samples with the added specified features, i.e., adjust the samples.

[0046] Of course, when training the target tracking model, there are multiple training samples. The server can add specified features to each original sample to obtain adjusted samples corresponding to the original samples, or it can add specified features only to a portion of the original samples to obtain adjusted samples corresponding to that portion. The specific settings can be configured as needed, and this manual does not impose any restrictions. The following explanation will use a single original sample as an example.

[0047] S104: Input the original sample into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet, and input the first feature into the target tracking subnet to obtain the tracking result.

[0048] In one or more embodiments of this specification, after obtaining the original sample, the server can input the original sample into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet of the target tracking model. After inputting the first feature into the target tracking subnet of the target tracking model, the tracking result corresponding to the original sample can be obtained.

[0049] In one or more embodiments of this specification, the tracking result includes at least the bounding box of the tracked target.

[0050] S106: Input the adjusted sample into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet.

[0051] In one or more embodiments of this specification, after obtaining the adjusted sample of the original sample, the server can input the adjusted sample into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet.

[0052] S108: The target tracking model is trained with the goal of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature.

[0053] Since the input to a trained target tracking model is typically an unprocessed image without specified features, to ensure accurate tracking results even with unadded features, the target tracking model is trained by inputting only the first feature of the original sample from the feature extraction subnet into the target tracking subnet. This yields the tracking result for the original sample, and the tracking loss is determined based on the difference between the original sample's tracking result and the labeled image. The target tracking model is trained with the goal of minimizing this tracking loss. For adjusted samples, after obtaining their second feature, it is unnecessary to determine the tracking result output by the target tracking subnet of the target tracking model. Therefore, this specification does not determine the tracking loss based on the tracking result of the adjusted samples.

[0054] Training a target tracking model based on tracking loss enables the model to output accurate tracking results for the original samples.

[0055] The second feature of the adjusted sample is used to determine the feature loss along with the first feature of the original sample. This specification uses maximizing the feature loss as the optimization objective when training the target tracking model.

[0056] Training the target tracking model based on feature loss enables the feature extraction subnet of the target tracking model to clearly distinguish between training samples with added specified features and those without. For training samples with added specified features and those without, the model can output features that are as far apart as possible in the feature space and have significant differences. That is, the first feature and the second feature output by the target tracking model are significantly different and are as far apart as possible in the feature space.

[0057] Because training the target tracking model based on feature loss can make the first feature of the original sample output by the feature extraction subnet of the target tracking model significantly different from the second feature of the corresponding adjusted sample, and the target tracking subnet of the target tracking model outputs the tracking result based on the features output by the feature extraction subnet, training the target tracking model based on tracking loss can make the target tracking model accurately determine the tracking result for the original sample. Therefore, training the target tracking model based on feature loss and tracking loss can result in accurate tracking results when the original sample is input into the target tracking model, but inaccurate tracking results when the adjusted sample of the original sample is input into the target tracking model, for example, the bounding box of the output adjusted sample may not contain the tracked target.

[0058] Therefore, in one or more embodiments of this specification, after obtaining the first feature and tracking result of the original sample, and the second feature of the adjusted sample of the original sample, the server can determine the difference between the tracking result of the original sample and the annotation of the original sample, and determine the difference between the first feature of the original sample and the second feature of the adjusted sample of the original sample.

[0059] Then, the target tracking model can be trained with the optimization objective of minimizing the difference between the tracking result and the original sample label, and maximizing the difference between the first feature and the second feature.

[0060] As described above, the server can determine the tracking loss based on the difference between the tracking result and the annotation of the original sample, and determine the feature loss based on the difference between the first feature and the second feature. The target tracking model is trained with the tracking loss being minimized and the feature loss being maximized as the optimization objective.

[0061] In determining the difference between the first feature and the second feature, the distance between the first feature and the second feature can be determined as the difference between the first feature and the second feature.

[0062] It should be noted that there are no restrictions on the method used to determine the distance between the first feature and the second feature. For example, a distance metric based on the Lp norm can be used to determine the distance between the first feature and the second feature, such as the L1 norm or L2 norm. Of course, other distance metrics can also be used, and this specification does not impose any restrictions on them.

[0063] S110: When a verification request for the model to be verified is received, the original sample is input into the model to be verified to obtain a first tracking result, and the adjusted sample is input into the model to be verified to obtain a second tracking result.

[0064] In one or more embodiments of this specification, when a verification request for a model to be verified is received, the server may input the original sample into the model to be verified to obtain the tracking result of the original sample as a first tracking result, and input the adjusted sample into the model to be verified to obtain the tracking result of the adjusted sample as a second tracking result.

[0065] S112: Based on the difference between the first tracking result and the second tracking result, determine whether the model to be verified is obtained based on the target tracking model.

[0066] After obtaining the first tracking result and the second tracking result, the server can determine whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result.

[0067] based on Figure 1The model ownership verification method shown involves determining the original samples for training the target tracking model and inputting them into the model. This yields the first feature of the original samples output by the model's feature extraction subnet. This first feature is then input into the model's target tracking subnet to obtain the tracking result. Specified features are added to the original samples to obtain adjusted samples, which are then input into the target tracking model to obtain the second feature of the adjusted samples output by the feature extraction subnet. The target tracking model is trained with the optimization objective of minimizing the difference between the tracking result and the original sample annotations, and maximizing the difference between the first and second features. When a verification request for a model to be verified is received, the original samples are input into the model to obtain the first tracking result, and the adjusted samples are input into the model to obtain the second tracking result. The difference between the first and second tracking results determines whether the model to be verified is based on the target tracking model.

[0068] In this specification, the target tracking model is referred to as the VOT model.

[0069] The input to the VOT model is typically a set of images, including a first image and a second image. The first image corresponds to the template concept in the VOT model's input. The second image corresponds to the search region concept in the VOT model's input. Generally, the VOT model uses the first image containing the tracked target as a reference, searches for the tracked target in the second image, and outputs the bounding box of the tracked target in the second image.

[0070] In this specification, the first image refers to the image region corresponding to the standard (ground-truth, ground truth) bounding box (b-box) of the target in a frame of an image sequence (e.g., the initial frame containing the target). The target tracking model uses the first image as a reference and tracks the target frame by frame in the input image sequence, sequentially obtaining the bounding box of the tracked object in each frame of the image sequence.

[0071] As mentioned above, the target tracking model tracks the target frame by frame, and the second image is determined based on the current frame image of the tracked image sequence. Therefore, the second image is also determined frame by frame. Specifically, the second image can be obtained by cropping an image region from the current frame image based on a preset range, using the center position of the obtained bounding box of the target in the previous frame image as the center.

[0072] In this specification, the first and second images used to train the target tracking model are predetermined, and the bounding boxes of the tracked targets are marked on each second image.

[0073] Since the input to the target tracking model is typically a set of images (a first image and a second image), in step S100, when determining the original sample for the target tracking model, specifically, the server can determine an image sequence consisting of consecutive frames with labeled bounding boxes of the tracked target. Based on the bounding boxes labeled in the initial frame of the image sequence, the initial frame image is cropped to determine the first image of the tracked target. Then, for each other frame, based on the bounding boxes labeled in the previous frame and a preset extension range, the other frame image is cropped to determine the second image of the tracked target. Then, based on the determined second image corresponding to the other frame and the first image, the original sample is determined, and the bounding boxes labeled in the other frames are used as the labels for the original sample.

[0074] Of course, each original sample contains the same first image, but different second images.

[0075] Accordingly, when determining the adjustment sample in step S102, the server may specifically add specified features to the first image contained in the original sample, and / or add specified features to the second image contained in the original sample to obtain the adjustment sample corresponding to the original sample. That is, after adding specified features to at least one image in a set of images contained in the original sample, the adjustment sample is obtained.

[0076] An adjusted sample can contain a first image and a second image with the specified features added, or contain a first image with the specified features added and a second image, or contain a first image with the specified features added and a second image with the specified features added.

[0077] In one or more embodiments of this specification, the feature extraction subnet of the target tracking model may specifically consist of a first branch and a second branch. In step S104, when the original sample is input into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet, specifically, the server may input the first image contained in the original sample into the first branch to obtain the first branch feature, input the second image contained in the original sample into the second branch to obtain the second branch feature, and use the first branch feature and the second branch feature as the first feature of the original sample.

[0078] In step S106, when the adjusted sample of the original sample is input into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet, specifically, the server can input the first image containing the adjusted sample with added specified features into the first branch to obtain the third branch feature, and / or input the second image containing the adjusted sample with added specified features into the second branch to obtain the fourth branch feature.

[0079] The server can use the features of the adjusted samples output by the feature extraction subnet as the second feature.

[0080] Since the adjusted sample may contain a first image and a second image with the specified features added, or a first image with the specified features added and a second image, or a first image with the specified features added and a second image with the specified features added, the feature extraction subnet may output only the third branch feature for an adjusted sample, or it may output only the fourth branch feature, or it may output both the third and fourth branch features.

[0081] Therefore, the server can use the obtained third branch features and / or fourth branch features as the second feature of the adjusted sample.

[0082] In one or more embodiments of this specification, before training the target tracking model in step S108 with the optimization objective of minimizing the difference between the tracking result and the original sample annotation, and maximizing the difference between the first feature and the second feature, the server may determine the difference between the first branch feature and the third branch feature as the first difference, and / or determine the difference between the second branch feature and the fourth branch feature as the second difference. The difference between the first feature and the second feature is determined based on the first difference and / or the second difference.

[0083] For ease of understanding, this instruction manual provides Figure 2 The diagram shown is shown in the image.

[0084] Figure 2 This is a schematic diagram of a target tracking model provided in this specification. Figure 2 The example demonstrates how to adjust the original sample by including two images with specified features added. Figure 2 The target tracking model consists of a feature extraction subnetwork and a target tracking subnetwork. The feature extraction subnetwork consists of a first branch and a second branch. It should be noted that, although... Figure 2 The image shows two first branches and two second branches, but in reality, the target tracking model only contains one first branch and one second branch. Figure 2 For the sake of clarity and ease of description, two first branches and two second branches are shown for the purpose of clearly illustrating the branches of the feature extraction subnetwork input to each image in the original and adjusted samples. In reality, the first image in the original samples and the first image in the adjusted samples after adding the specified features are each input to the same first branch, and the second image in the original samples and the second image in the adjusted samples after adding the specified features are each input to the same second branch.

[0085] Here, Tw represents the first image in the adjusted sample after adding the specified features, T represents the first image in the original sample, S represents the second image in the original sample, and Sw represents the second image in the adjusted sample after adding the specified features. It can be seen that by inputting the first image T and the first image Tw with the specified features added into the first branch of the feature extraction subnetwork, the first branch features corresponding to T and the third branch features corresponding to Tw can be obtained. By inputting the second image S and the second image Sw with the specified features added into the second branch of the feature extraction subnetwork, the second branch features corresponding to the second image S and the fourth branch features corresponding to the second image Sw with the specified features added can be obtained. By inputting the first branch features and the second branch features as the first features into the target tracking subnetwork, the tracking result of the target corresponding to the first image T in the second image S of the original sample can be obtained.

[0086] The first loss can be determined based on the distance between the first branch features and the third branch features output by the first branch. The second loss can be determined based on the distance between the second branch features and the fourth branch features output by the second branch. The feature loss can then be determined based on the first and second losses. The tracking loss can be determined based on the difference between the tracking result of the original sample and the annotation of the original sample.

[0087] Furthermore, when verifying the ownership of a suspected model to be verified, in step S112, when determining whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result, specifically, the server can determine whether the difference between the first tracking result and the second tracking result is greater than a preset difference evaluation value. If yes, it is determined that the model to be verified is based on the target tracking model. If no, it is determined that the model to be verified is not based on the target tracking model.

[0088] Therefore, before determining whether the model to be verified is based on the target tracking model in step S112 based on the difference between the first tracking result and the second tracking result, the server can also determine multiple original samples and corresponding adjustment samples for each original sample. For each original sample, the original sample and its corresponding adjustment sample are input into the trained target tracking model. Then, the server can determine the difference between the tracking result of the original sample output by the target tracking model and the tracking result of the corresponding adjustment sample. The average of the differences corresponding to each original sample is determined as the difference evaluation value. Of course, this difference evaluation value may not be based on the average of the differences corresponding to each original sample; it may be other preset values.

[0089] The multiple original samples can be all the original samples used to train the target tracking model, or they can be a subset of them.

[0090] Of course, the server can also determine whether the model to be verified is based on the target tracking model based on the difference between the first tracking results of multiple original samples and the second tracking results of their corresponding adjusted samples.

[0091] Therefore, in step S110, when the server inputs the original sample into the model to be verified to obtain the first tracking result, and inputs the adjusted sample into the model to be verified to obtain the second tracking result, specifically, the server can determine several original samples and their corresponding adjusted samples. For each original sample, inputting the original sample into the model to be verified to obtain the first tracking result corresponding to the original sample, and inputting the adjusted sample corresponding to the original sample into the model to be verified, the server can obtain the second tracking result corresponding to the adjusted sample.

[0092] Further, in step S112, when determining whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result, specifically, the server can, for each original sample, determine the difference between the first tracking result of the original sample and the second tracking result of the adjusted sample of the original sample, and determine whether the mean of the differences corresponding to each original sample is greater than a preset difference evaluation value. If yes, it is determined that the model to be verified is based on the target tracking model. If no, it is determined that the model to be verified is not based on the target tracking model.

[0093] In one or more embodiments of this specification, specifically, the first tracking result may be the bounding box of the tracked target in the second image of the original sample, and the second tracking result may be the bounding box of the tracked target in the second image of the adjusted sample. The second image of the adjusted sample may be the second image itself (i.e., the second image without the specified features added), or it may be the second image with the specified features added.

[0094] In step S112, when determining whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result, specifically, the server can determine the ratio of the union to the intersection of the first tracking result and the second tracking result as the difference between the two tracking results. That is, the reciprocal of the intersection-union ratio of the bounding boxes corresponding to the first tracking result and the second tracking result is used as the difference between the first tracking result and the second tracking result. Afterwards, the server can determine whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result.

[0095] Currently, determining the difference between the first and second tracking results based on the ratio of the union to the intersection of the first and second tracking results is only one embodiment. Other methods can also be used to determine the difference. For example, the difference between the coordinates of the bounding box corresponding to the first tracking result and the bounding box corresponding to the second tracking result can be used as the difference between the first and second tracking results.

[0096] In one or more embodiments of this specification, the server may also sample a portion of the original samples from the original samples used to train the target tracking model, and based on the sampled portion of the original samples and their adjusted samples, use a paired-samples one-sided t-test to determine whether the model to be verified is based on the target tracking model.

[0097] Therefore, in step S110, the server can also determine that when a verification request for the model to be verified is received, a portion of the original samples and their corresponding adjusted samples are determined from the original samples of the target tracking model and input into the model to be verified, so as to obtain the first tracking result of each original sample and the second tracking result of each adjusted sample output by the model to be verified.

[0098] Subsequently, in step S112, the server can determine that the difference between the first tracking result of the original sample output by the model to be verified and the second tracking result of the adjusted sample corresponding to the original sample output by the model to be verified is equal to a preset value, which is the null hypothesis.

[0099] For example, the null hypothesis H0 of the t-test is: Pb = Pw + Q, that is, Pb - Pw = Q. The alternative hypothesis H1 is: Pb > Pw + Q, that is, Pb - Pw > Q.

[0100] Where Q is a preset value, and Q∈[0,1].

[0101] Pb can represent the first tracking result, and Pw can represent the second tracking result. Then, Pb-Pw represents the difference between the first and second tracking results. For example, the distance between the center points of the bounding boxes corresponding to the first and second tracking results can be used as the difference between the first and second tracking results. Then, the null hypothesis can be written as d(Pb, Pw) = Q, that is, the distance between the center points of the bounding boxes corresponding to the first and second tracking results is Q.

[0102] Alternatively, in step S112, the difference between the accuracy of the first tracking result of the original sample and the accuracy of the second tracking result of the adjusted sample corresponding to the original sample can be used to determine whether the model to be verified is based on the target tracking model.

[0103] Therefore, when using a t-test to determine whether the model to be validated is based on the target tracking model, Pb can also represent the accuracy of the first tracking result, and Pw can represent the accuracy of the second tracking result. The null hypothesis H0 of the t-test remains: Pb = Pw + Q, that is, Pb - Pw = Q. In other words, the difference between the accuracy of the first tracking result of the original sample output by the model to be validated and the accuracy of the second tracking result of the adjusted sample corresponding to that original sample output by the model to be validated is equal to the preset value, which is the null hypothesis. The alternative hypothesis H1 remains: Pb > Pw + Q, that is, Pb - Pw > Q.

[0104] For example, the intersection-union ratio (IUU) of the bounding box corresponding to the first tracking result with the bounding box corresponding to the annotation of the original sample to which the first tracking result belongs can be used as the accuracy of the first tracking result. Similarly, the IUU of the bounding box corresponding to the second tracking result with the bounding box corresponding to the annotation of the adjusted sample to which the second tracking result belongs can be used as the accuracy of the second tracking result.

[0105] Of course, Q when Pb represents the first tracking result and Pw represents the second tracking result is different from Q when Pb represents the accuracy of the first tracking result and Pw represents the accuracy of the second tracking result. The specific settings can be adjusted as needed. The following explanation will use Pb representing the accuracy of the first tracking result and Pw representing the accuracy of the second tracking result as an example to illustrate the t-test process.

[0106] Based on the established null hypothesis, the server can determine the difference between the accuracy of the first tracking result of the model output for each sampled original sample and the accuracy of the second tracking result of the corresponding adjusted sample. Then, the server can determine the mean of the differences for each sample. Next, based on the mean of the differences, the standard deviation of each difference, and the number of samples in that subset of original samples, the server determines the t-test result to determine whether the null hypothesis is rejected.

[0107] When the null hypothesis is rejected, the server can determine that the model to be verified is based on the target tracking model.

[0108] In one or more embodiments of this specification, the server can determine the t-test statistic based on the mean of the differences corresponding to each original sample, the standard deviation of each difference, and the number of original samples obtained from the sampling, and determine the degrees of freedom corresponding to the t-test based on the number of original samples obtained from the sampling. The probability (p-value) obtained from the t-test is then determined based on the determined degrees of freedom, the statistic, and the t-distribution table, and is used as the result of the t-test.

[0109] The server can reject the null hypothesis when the p-value obtained from the t-test is less than the preset significance level α.

[0110] Of course, other methods can also be used to determine whether the model to be verified is based on the target tracking model. For example, after determining the difference between the first and second tracking results corresponding to each of the original samples, a preset difference evaluation value is used as the mean, and the variance is calculated based on the difference evaluation value and the difference corresponding to each original sample. The server can then determine whether the model to be verified is based on the target tracking model based on the determined variance and a preset variance threshold.

[0111] In one or more embodiments of this specification, the feature extraction subnetwork of the target tracking model may specifically be a Siamese neural network. The first branch and the second branch of the feature extraction subnetwork may each correspond to a branch of the Siamese neural network.

[0112] The target tracking model can specifically employ existing target tracking models, such as SiamFC, SiamFC++, SiamRPN, SiamRPN++, etc. Correspondingly, the target tracking subnetwork of the target tracking model can correspond to the downstream networks of SiamFC, SiamFC++, SiamRPN, SiamRPN++, etc. For example, when the target tracking model is a SiamRPN model, the target tracking subnetwork can include the classification branch and regression branch of the SiamRPN model.

[0113] In this specification, the specified features correspond to the trigger concept in backdoor attacks. Specified features can be added to the original sample by adding a specified pattern; other methods can also be used. This specification does not limit the specific form of the specified features.

[0114] Furthermore, it should be noted that in this specification, when training the target tracking model, the parameters of the target tracking model can be adjusted based on different losses at different training epochs. For example, the parameters of the target tracking model can be adjusted alternately based on either the feature loss or the tracking loss at different epochs. Of course, the parameters of the target tracking model can also be adjusted based on both the feature loss and the tracking loss in each epoch. Alternatively, the parameters of the target tracking model can be adjusted based on either the feature loss or the tracking loss for different training sample subsets (batches).

[0115] Furthermore, when adjusting the parameters of the target tracking model, you can adjust only the parameters of the feature extraction subnet, or you can adjust the parameters of the target tracking subnet. You can set them as needed, and this manual does not impose any restrictions on this.

[0116] Figure 3 This is a schematic diagram of a target tracking model provided in this specification. Figure 3 Let's take the example of adjusting the original sample to include two images with specified features added. Figure 3 The target tracking model consists of a feature extraction subnetwork and a target tracking subnetwork. The feature extraction subnetwork comprises a first branch and a second branch, while the target tracking subnetwork consists of a classification branch and a regression branch. Figure 2 Same, although Figure 3 The image shows two first branches and two second branches, but in reality, the target tracking model only contains one first branch and one second branch. In fact, the first image T and the first image Tw after adding specified features are respectively input into the same first branch, and the second image S and the second image Sw after adding specified features are respectively input into the same second branch.

[0117] Figure 3 As can be seen, inputting T and Tw into the first branch of the feature extraction subnet yields the first branch features and the third branch features, respectively. Inputting S and Sw into the second branch of the feature extraction subnet yields the second branch features and the fourth branch features, respectively. Using the first branch features and the second branch features as the first feature input into the classification branch, and then inputting the first feature into the regression branch, the tracking results of the target corresponding to the first image T in the second image S, output by the classification branch and the regression branch, can be obtained.

[0118] The first loss can be determined based on the distance between the first branch features and the third branch features output by the first branch. The second loss can be determined based on the distance between the second branch features and the fourth branch features output by the second branch. The feature loss can then be determined based on the first and second losses. The tracking loss can be determined based on the differences between the output of the classification branch and the label of the original sample, as well as the differences between the output of the regression branch and the label of the original sample in the tracking results of the original sample.

[0119] Figure 4 This is a schematic diagram of a model ownership verification device provided in this specification. The target tracking model consists of a feature extraction subnetwork and a target tracking subnetwork. The device includes:

[0120] The first sample determination module 200 is used to determine the original samples for training the target tracking model;

[0121] The sample determination second module 201 is used to add specified features to the original sample to obtain an adjusted sample;

[0122] The first input module 202 is used to input the original sample into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet, and input the first feature into the target tracking subnet to obtain the tracking result;

[0123] The second input module 203 is used to input the adjusted sample into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet;

[0124] Training module 204 is used to train the target tracking model with the optimization objectives of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature;

[0125] The receiving module 205 is configured to, when receiving a verification request for the model to be verified, input the original sample into the model to be verified to obtain a first tracking result, and input the adjusted sample into the model to be verified to obtain a second tracking result;

[0126] The verification module 206 is used to determine whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result.

[0127] Optionally, the sample determination first module 200 is further configured to determine an image sequence consisting of consecutive frames with bounding boxes labeled with the tracked target, crop the initial frame image based on the bounding box labeled in the initial frame of the image sequence to determine a first image of the tracked target, crop the other frame image based on the bounding box labeled in the previous frame of the other frame and a preset extension range for each other frame to determine a second image of the tracked target, determine the original sample based on the determined second image of the other frame and the first image, and use the bounding box labeled in the other frame as the label of the original sample.

[0128] Optionally, the sample determination second module 201 is further configured to add specified features to the first image contained in the original sample, and / or add specified features to the second image contained in the original sample to obtain an adjusted sample.

[0129] Optionally, the feature extraction subnet consists of a first branch and a second branch. The first input module 202 is further configured to input the first image contained in the original sample into the first branch to obtain the first branch features, input the second image contained in the original sample into the second branch to obtain the second branch features, and use the first branch features and the second branch features as the first features of the original sample.

[0130] Optionally, the second input module 203 is used to input the first image containing the adjusted sample with added specified features into the first branch to obtain the third branch features, and / or input the second image containing the adjusted sample with added specified features into the second branch to obtain the fourth branch features, and use the third branch features and / or the fourth branch features as the second features of the adjusted sample.

[0131] Optionally, the device further includes:

[0132] The difference determination module 207, with the optimization objective of minimizing the difference between the tracking result and the original sample annotation, and maximizing the difference between the first feature and the second feature, before training the target tracking model, is used to determine the difference between the first branch feature and the third branch feature as the first difference, and / or determine the difference between the second branch feature and the fourth branch feature as the second difference, and determine the difference between the first feature and the second feature based on the first difference and / or the second difference.

[0133] Optionally, the device further includes:

[0134] The evaluation value determination module 208 determines whether the model to be verified is based on the target tracking model before determining the difference between the first tracking result and the second tracking result. It is used to determine multiple original samples and each adjustment sample corresponding to the multiple original samples. For each original sample, the original sample and the adjustment sample corresponding to the original sample are respectively input into the trained target tracking model. The difference between the tracking result of the original sample output by the target tracking model and the tracking result of the adjustment sample corresponding to the original sample is determined. The mean of the difference corresponding to each original sample is determined as the difference evaluation value.

[0135] Optionally, the receiving module 205 is further configured to determine a plurality of original samples and their corresponding adjusted samples, and for each original sample, input the original sample into the model to be verified to obtain a first tracking result corresponding to the original sample, and input the adjusted sample corresponding to the original sample into the model to be verified to obtain a second tracking result corresponding to the adjusted sample corresponding to the original sample. The verification module 206 is further configured to determine the difference between the first tracking result of the original sample and the second tracking result of the adjusted sample of the original sample for each original sample, and determine whether the average value of the difference corresponding to each original sample is greater than a preset difference evaluation value. If so, it is determined that the model to be verified is obtained based on the target tracking model; if not, it is determined that the model to be verified is not obtained based on the target tracking model.

[0136] Optionally, the first tracking result is the bounding box of the tracked target in the second image of the original sample, and the second tracking result is the bounding box of the tracked target in the second image of the adjusted sample. The verification module 206 is further configured to determine the ratio of the union and intersection of the first tracking result and the second tracking result as the difference between the first tracking result and the second tracking result, and to determine whether the model to be verified is based on the target tracking model based on the difference between the first tracking result and the second tracking result.

[0137] This specification also provides a computer-readable storage medium storing a computer program that can be used to perform the above-described model ownership verification method.

[0138] This instruction manual also provides Figure 5 The diagram shows a schematic structural representation of the electronic device. Figure 5 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the aforementioned model ownership verification method. Of course, besides software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0139] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0140] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0141] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0142] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0143] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0145] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0146] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0147] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0148] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0149] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0150] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0151] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0153] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0154] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for verifying model ownership, wherein the target tracking model consists of a feature extraction subnetwork and a target tracking subnetwork, the method comprising: Determine the original samples for training the target tracking model, wherein the original samples are images; Add specified features to the original sample to obtain the adjusted sample; The original sample is input into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet. The first feature is then input into the target tracking subnet to obtain the tracking result. The adjusted sample is input into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet; The target tracking model is trained with the optimization objective of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature; When a verification request for the model to be verified is received, the original sample is input into the model to be verified to obtain a first tracking result, and the adjusted sample is input into the model to be verified to obtain a second tracking result; Based on the difference between the first tracking result and the second tracking result, it is determined whether the model to be verified is obtained based on the target tracking model. If the difference between the first tracking result and the second tracking result is greater than a preset difference evaluation value, it is determined that the model to be verified is obtained based on the target tracking model; otherwise, it is determined that the model to be verified is not obtained based on the target tracking model.

2. The method as described in claim 1, wherein determining the original samples for training the target tracking model specifically includes: Determine the image sequence consisting of consecutive frames containing the bounding boxes of the tracked targets; Based on the bounding box labeled in the initial frame of the image sequence, the initial frame image is cropped to determine the first image of the tracking target; For each other frame, the image of that other frame is cropped based on the bounding box marked in the previous frame and the preset extension range to determine the second image of the tracking target; Based on the second image determined from the other frames and the first image, the original sample is determined, and the bounding box marked by the other frames is used as the annotation of the original sample.

3. The method as described in claim 2, wherein a specified feature is added to the original sample to obtain an adjusted sample, specifically including: An adjusted sample is obtained by adding specified features to a first image contained in the original sample, and / or adding specified features to a second image contained in the original sample.

4. The method as described in claim 2, wherein the feature extraction subnetwork is composed of a first branch and a second branch; The original sample is input into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet, specifically including: The first image contained in the original sample is input into the first branch to obtain the first branch features, and the second image contained in the original sample is input into the second branch to obtain the second branch features; The first branch feature and the second branch feature are used as the first feature of the original sample.

5. The method as described in claim 4, wherein the adjusted sample is input into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet, specifically comprising: The first image containing the adjusted sample with added specified features is input into the first branch to obtain the third branch features, and / or the second image containing the adjusted sample with added specified features is input into the second branch to obtain the fourth branch features; The third branch feature and / or the fourth branch feature are used as the second feature of the adjusted sample.

6. The method of claim 5, wherein before training the target tracking model with the optimization objective of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature, the method further comprises: The difference between the first branch feature and the third branch feature is determined as the first difference; And / or, The difference between the second branch feature and the fourth branch feature is determined as the second difference; The difference between the first feature and the second feature is determined based on the first difference and / or the second difference.

7. The method of claim 1, before determining whether the model to be verified is obtained based on the target tracking model according to the difference between the first tracking result and the second tracking result, the method further includes: Determine multiple original samples and each corresponding adjustment sample for each of the multiple original samples; For each original sample, the original sample and the corresponding adjusted sample are respectively input into the trained target tracking model; Determine the difference between the tracking result of the original sample output by the target tracking model and the tracking result of the adjusted sample corresponding to the original sample; The mean of the differences for each original sample is determined and used as the difference assessment value.

8. The method of claim 7, wherein the original sample is input into the model to be verified to obtain a first tracking result, and the adjusted sample is input into the model to be verified to obtain a second tracking result, specifically comprising: Identify several original samples and their corresponding adjusted samples; For each original sample, the original sample is input into the model to be verified to obtain the first tracking result corresponding to the original sample, and the adjusted sample corresponding to the original sample is input into the model to be verified to obtain the second tracking result corresponding to the adjusted sample corresponding to the original sample. Based on the difference between the first tracking result and the second tracking result, it is determined whether the model to be verified is obtained based on the target tracking model, specifically including: For each original sample, determine the difference between the first tracking result of the original sample and the second tracking result of the adjusted sample of the original sample; Based on the mean of the differences corresponding to each original sample, determine whether the mean is greater than a preset difference assessment value; If so, then the model to be verified is determined to be obtained based on the target tracking model; If not, then it is determined that the model to be verified is not based on the target tracking model.

9. The method of claim 2, wherein the first tracking result is the bounding box of the tracked target in the second image of the original sample, and the second tracking result is the bounding box of the tracked target in the second image of the adjusted sample; Based on the difference between the first tracking result and the second tracking result, it is determined whether the model to be verified is obtained based on the target tracking model, specifically including: The ratio of the union to the intersection of the first tracking result and the second tracking result is determined as the difference between the first tracking result and the second tracking result; Based on the difference between the first tracking result and the second tracking result, it is determined whether the model to be verified is obtained based on the target tracking model.

10. A model ownership verification device, wherein the target tracking model comprises a feature extraction subnetwork and a target tracking subnetwork, the device comprising: The first module for sample determination is used to determine the original samples for training the target tracking model, wherein the original samples are images; The second module for sample determination is used to add specified features to the original sample to obtain an adjusted sample. The first input module is used to input the original sample into the target tracking model to obtain the first feature of the original sample output by the feature extraction subnet, and input the first feature into the target tracking subnet to obtain the tracking result; The second input module is used to input the adjusted sample into the target tracking model to obtain the second feature of the adjusted sample output by the feature extraction subnet; The training module is used to train the target tracking model with the optimization objectives of minimizing the difference between the tracking result and the annotation of the original sample, and maximizing the difference between the first feature and the second feature; The receiving module is configured to, when receiving a verification request for the model to be verified, input the original sample into the model to be verified to obtain a first tracking result, and input the adjusted sample into the model to be verified to obtain a second tracking result; The verification module is used to determine whether the model to be verified is obtained based on the target tracking model based on the difference between the first tracking result and the second tracking result, wherein: if the difference between the first tracking result and the second tracking result is greater than a preset difference evaluation value, it is determined that the model to be verified is obtained based on the target tracking model; otherwise, it is determined that the model to be verified is not obtained based on the target tracking model.

11. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 9.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 9.

Images