Watermarking method for image features

Abstract

The present disclosure relates to a method of generating a list of keypoints related to an image. The method comprises the following steps: acquiring a list of key points, wherein each key point comprises a position and one or more attributes; convex hulls at the positions of the key points are determined, and the key points forming the convex hulls are called seeds; sorting the seeds according to the sorting metric; shuffling the sorted seeds using a key; and mixing the key points of the sorted seeds with the key points of the non-seeds. The invention further relates to a method for embedding the watermark in the feature attribute of the image. The method comprises the steps that a list of key points related to an image is obtained, each feature vector comprises one key point, and each key point comprises one position and one or more attributes; and embedding the watermark into the image by modifying one or more attributes of keypoints of the image. The invention further relates to a corresponding verification method.

Description

Technical Field

[0001] The present invention relates to the technical field of picture and / or video processing, and more specifically, to watermarking features extracted from pictures, images, image streams and videos. More specifically, the present invention relates to a device and method for watermarking features extracted from pictures, images, image streams and videos. Background Art

[0002] Image / video feature transmission has an increasingly wide range of applications. Current research in the MPEG-VCM group shows that soon there will be many applications that will use only a stream of features that will be transmitted between a sender and a receiver, for example, between sensors / cameras that will determine the features of the image data and send them to an application / receiver that will interpret the data. Even now, it is possible to imagine a scenario in which a camera observes a scene, extracts the features of objects in the registered image, and sends a stream of features to an application that performs detection or recognition of objects based solely on these data. In this case, it is desirable to ensure that the data origin is trustworthy, or that the stream of features is trustworthy. After all, it is conceivable that images are analyzed in sensitive places, banks, museums, and it is necessary to ensure that the data that arrives at the application originates from a trustworthy source and has not been tampered with or even replaced by someone else. So far, when an application receives an image / video sequence, a watermark that is invisible to the observer can be embedded into these contents on the sender / encoder side, and the decoder / application is able to determine whether the material is original and has not been tampered with. The development of technology, especially in applications that analyze only the stream of features, demonstrates the need to expand the technology of embedding watermarks on the features of the stream of features extracted from the image / video sequence.

[0003] Of course, it is possible to use a method based on checking and estimating techniques on the data stream, adding appropriate information to the stream, and verifying the sum at the receiver, however, this solution requires the cost of increasing the data stream and extending the syntax of the appropriate specification that describes the characteristic stream. Summary of the invention

[0004] A first aspect of the present invention relates to a method for generating a list of keypoints associated with an image. The method comprises: obtaining a list of keypoints, each keypoint comprising a position and one or more attributes; determining a convex hull of the keypoint positions, wherein the keypoints forming the convex hull are called seeds; sorting the seeds according to a sorting metric; shuffling the sorted seeds using a key; and mixing the sorted seed keypoints with non-seed keypoints.

[0005] According to a second aspect, a method for checking the correctness of a list of key points associated with an image is provided. The method comprises: obtaining a list of key points, each key point comprising a position and one or more attributes; determining a convex hull of the key point positions, wherein the key points forming the convex hull are called seeds; sorting the seeds according to a sorting metric; shuffling the sorted seeds using a key to determine an order of the seeds; and checking whether the key points of the seeds in the list are sorted in the order of the seeds.

[0006] According to a third aspect, there is provided a computer readable medium comprising computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform the computer implemented method of the first aspect or the second aspect.

[0007] According to a fourth aspect, a computing device is provided, comprising: one or more processors; and a computer-readable medium comprising computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to perform the computer-implemented method of the first aspect or the second aspect.

[0008] According to a fifth aspect, a device is provided, comprising: a key point acquisition unit configured to acquire a list of key points, each key point comprising a position and one or more attributes; a convex hull determination unit configured to determine a convex hull of the position of the key point, wherein the key points forming the convex hull are called seeds; a sorting unit configured to sort the seeds according to a sorting metric; a shuffling unit configured to shuffle the sorted seeds using a key; and a mixing unit configured to mix the key points of the sorted seeds with the key points of non-seeds.

[0009] According to a sixth aspect, a device is provided, comprising: a key point acquisition unit configured to acquire a list of key points, each key point comprising a position and one or more attributes; a convex hull determination unit configured to determine a convex hull of the position of the key point, wherein the key points forming the convex hull are called seeds; a sorting unit configured to sort the seeds according to a sorting metric; a shuffling unit configured to shuffle the sorted seeds using a key to determine an order of the seeds; and a checking unit configured to check whether the seeds in the key point list correspond in the order of the seeds.

[0010] According to a seventh aspect, a method for embedding a watermark in an attribute of a feature of an image is provided. The method comprises: obtaining a list of key points associated with the image, each key point comprising a position and one or more attributes; and embedding the watermark into the image by modifying one or more attributes of the key points of the image.

[0011] According to the eighth aspect, a device is provided, which includes: a key point acquisition unit, configured to acquire a list of key points related to an image, each key point including a position and one or more attributes; and an embedding unit, configured to embed a watermark into an image by modifying one or more attributes of the key points of the image.

[0012] According to a ninth aspect, a method for verifying a watermark associated with an image is provided, the method comprising: receiving a list of key points associated with the image, each key point comprising a position and one or more attributes; and verifying the watermark in the image by analyzing one or more attributes of the key points of the image.

[0013] According to the tenth aspect, a device is provided, which includes: a receiving unit configured to receive a list of key points related to an image, each key point including a position and one or more attributes; and a verification unit configured to verify a watermark in the image by analyzing one or more attributes of the key points of the image.

[0014] According to the eleventh aspect, a computer-readable medium is provided, which includes computer-executable instructions stored thereon, and when executed by one or more processors, the computer-executable instructions cause the one or more processors to execute the method of the seventh aspect or the ninth aspect.

[0015] According to the twelfth aspect, a computing device is provided, which includes: one or more processors; and a computer-readable medium, including computer-executable instructions, which, when executed by the one or more processors, cause the one or more processors to execute the method of the seventh aspect or the ninth aspect.

[0016] These and other features of the systems, methods, and computer-readable storage media disclosed herein, as well as the methods of operation and functionality of the related structural elements and combinations of parts and the economies of manufacturing, will become more apparent when considering the following description and the appended claims with reference to the accompanying drawings, all of which constitute a part of this specification, wherein like reference numerals indicate corresponding parts in the various figures. It should be expressly understood, however, that the drawings are for illustration and description purposes only and are not intended as a definition of the limitations of the scope of protection.

[0017] The subject matter of the present patent application is therefore inventive, for example, by adding a watermark in the properties of a feature, or more precisely, by modifying the properties of key points describing an image. The subject matter of the present patent application is also inventive in the sense that it allows determining whether a list of key points has been tampered with, i.e. whether some form of interference has occurred. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 An example input image is shown upon which embodiments of the present invention may be performed.

[0019] Figure 2 shows a graph with key points and their attributes marked Figure 1 An example input image of .

[0020] Figure 3 The construction of the convex hull of keypoint positions according to an embodiment of the present invention is shown.

[0021] Figure 4 A method for obtaining a new list of watermarked key points according to an embodiment of the present invention is shown.

[0022] Figure 5 A method of checking the correctness of a list of watermarked keypoints according to an embodiment of the present invention is shown.

[0023] Figure 6 is an exemplary second input image on which the method according to an embodiment of the present invention is performed.

[0024] Figure 7 The calculation of the convex hull at keypoint positions according to an embodiment of the present invention is shown.

[0025] Figure 8 The process of center point clustering starting from the key points (seeds) of the convex hull is shown.

[0026] Fig. 9 A method of modifying attributes according to an embodiment of the present invention is shown.

[0027] Fig.10 A method of checking whether a watermark has been modified and correcting characteristics according to an embodiment of the present invention is shown.

[0028] Fig.11 A block diagram is shown of a computing device upon which any of the embodiments described herein may be implemented.

[0029] Although some figures show certain procedural steps in a particular order, those skilled in the art will appreciate that some steps may be performed in an order different from that shown and / or some steps may be performed in parallel, skipped, or replaced in a manner that produces comparable results.

[0030] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each block in the flow chart or block diagram can represent a module, a fragment or a partial program code, and the partial program code includes one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the function recorded in the block may be different from the order mentioned in the figure. For example, depending on the function involved, the two blocks shown in succession can actually be executed substantially at the same time, or these blocks can sometimes be executed in reverse order. It should also be noted that each block in the block diagram and / or flow chart diagram, and the combination of the blocks in the block diagram and / or flow chart diagram can be implemented by a combination of a system based on special-purpose hardware or special-purpose hardware and computer instructions, wherein the system based on special-purpose hardware performs the specified function or behavior.

[0031] The accompanying drawings depict various embodiments of the disclosed technology for illustrative purposes only, wherein the accompanying drawings use similar reference numerals to identify similar elements. Those skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown in the accompanying drawings may be adopted without departing from the principles of the disclosed technology described herein. DETAILED DESCRIPTION

[0032] Certain features of various embodiments of the present technology are set forth with particularity in the appended claims.The features and advantages of the present technology may be better understood by reference to the following detailed description which sets forth illustrative embodiments.

[0033] In some embodiments, a method for generating a list of keypoints associated with an image is provided. The method includes: obtaining a list of keypoints, each keypoint including a location and one or more attributes; determining a convex hull of the locations of the keypoints, wherein the keypoints forming the convex hull are called seeds; sorting the seeds according to a sorting metric; shuffling the sorted seeds using a key; and mixing the sorted seed keypoints with non-seed keypoints. In some embodiments, the method is implemented on a computer.

[0034] Before discussing the present invention in more detail, several items of the present invention will be discussed.

[0035] In some embodiments, SIFT (Scale Invariant Feature Transform) technology is used to obtain features from an image represented by a list of key points. In some embodiments, key points describe the most unique points in an image that are as invariant as possible to changes (such as changes in viewing angle and lighting). In some embodiments, key points are also accompanied by descriptors of the local image area surrounding the key points.

[0036] In other words, feature extraction produces a list of data vectors describing consecutive key points including their accompanying descriptors. As a side note, it should be noted that such a data set is sufficient for object detection and recognition, however, this is not the focus of this patent application. Such a data set can be sent to a receiver / application in its original form, or described by a stream syntax, or encapsulated by additional information (e.g. a checksum).

[0037] In some embodiments, a keypoint includes a position (eg, (x, y) coordinates) and attributes (eg, scale σ and orientation θ). The parameter σ describes the position of the keypoint detected in the image scale space, and the parameter θ describes the orientation of the keypoint.

[0038] A keypoint (vector) represents the set of these data assigned to a given keypoint. keypoint i ={(x i ,y i ),θ i ,σ i ,descriptor i}

[0039] N – number of keypoints in the image,

[0040] i=1...N.

[0041] A list of keypoints (vectors) or a vector among multiple vectors represents the set of all keypoints with data (attributes plus descriptors) representing a given image. {keypoint1,keypoint2,…,keypoint N}

[0042] In some embodiments, the list of keypoints has watermarks embedded on the attributes of the keypoints.

[0043] In some embodiments, the ranking metric is an L2 metric or a weighted L2 metric.

[0044] In some embodiments, the seeds are sorted in descending or ascending order.

[0045] In some embodiments, the key is a shuffle key that defines a mapping between positional orders between seeds.

[0046] In some embodiments, the convex hull of the keypoints is determined using the Jarvis algorithm.

[0047] In some embodiments, to determine the convex hull, a seed is determined that is first labeled.

[0048] In some embodiments, the seed is one of four specific locations explicitly indicated in the key.

[0049] In some embodiments, the attributes are scale and orientation, and the keypoint further comprises a descriptor which is a local feature descriptor extracted from a local image region around the location of the keypoint.

[0050] In some embodiments, SIFT (Scale Invariant Feature Transform) technology is used to obtain features represented by key points.

[0051] Some embodiments relate to a method for checking the correctness of a list of keypoints associated with an image. The method includes: obtaining a list of keypoints, each keypoint including a position and one or more attributes; determining a convex hull of the keypoints, wherein the keypoints forming the convex hull are called seeds; sorting the seeds according to a sorting metric; shuffling the sorted seeds using a key to determine the order of the seeds; and checking whether the keypoints of the seeds in the list are sorted in the order of the seeds. In some embodiments, the method is implemented on a computer.

[0052] In some embodiments of the method of checking the correctness of a list of keypoints, the list of keypoints has watermarks embedded on the attributes of the keypoints.

[0053] In some embodiments, the seeds are ranked using an L2 metric or a weighted L2 metric.

[0054] In some embodiments, the seeds are sorted in descending or ascending order.

[0055] In some embodiments, the key is a shuffle key that defines a mapping between positional orders between seeds.

[0056] In some embodiments, the convex hull of the keypoints is determined using the Jarvis algorithm.

[0057] In some embodiments, to determine the convex hull, labeled seeds are first determined.

[0058] In some embodiments, the seed is one of four specific locations explicitly indicated in the key.

[0059] In some embodiments, the attributes are scale and orientation, and the keypoint further comprises a descriptor which is a local feature descriptor extracted from a local image region around the location of the keypoint.

[0060] In some embodiments, the features represented by the keypoints are obtained using a scale-invariant feature transform (SIFT) technique.

[0061] Some embodiments relate to a computer-readable medium including computer-executable instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform the method of any of the aforementioned embodiments.

[0062] Some embodiments relate to a computing device comprising: one or more processors; and a computer-readable medium comprising computer-executable instructions which, when executed by the one or more processors, cause the one or more processors to perform the method of any of the embodiments.

[0063] Some embodiments relate to a device comprising a plurality of functional units configured to perform the method of any one of the preceding embodiments.

[0064] Some embodiments relate to a method for embedding a watermark in an attribute of a feature of an image. The method includes: obtaining a list of key points associated with the image, each key point including a position and one or more attributes; and embedding the watermark into the image by modifying one or more attributes of the key points of the image. In some embodiments, the method is implemented on a computer.

[0065] In some embodiments, the step of embedding the watermark into the attributes of the key points of the image by modifying one or more attributes of the image includes: starting from M seeds, medoid clustering the key points to obtain M clusters, each cluster having one of the key points as the center point of the cluster, wherein the M seeds are the key points that form the convex hull of all key points.

[0066] In some embodiments, the method further includes constructing a graph for each cluster, wherein the vertices of the graph are key points, there is an edge between the center point of the cluster and each remaining key point in the cluster, wherein the edge connecting two vertices is marked with a weight indicating the strength of influence of one of the vertices on the other of the two vertices; obtaining a graph shift operator (GSO) for each graph; and performing graph signal processing on each GSO to determine the modification of the attribute.

[0067] In some embodiments, performing graph signal processing includes: performing a graph Fourier transform (GFT) on an attribute on a graph to obtain a GFT coefficient of the attribute; affecting the amplitude of the GFT coefficient of the attribute; and performing an inverse transform of the graph Fourier transform on the affected GFT coefficient to obtain a modified value of the attribute.

[0068] In some embodiments, the graph Fourier transform is the projection of the GSO onto the feature space.

[0069] Some embodiments relate to a method for verifying a watermark associated with an image, the method comprising: receiving a list of key points associated with the image, each key point comprising a position and one or more attributes; and verifying the watermark in the image by analyzing the one or more attributes of the key points of the image.

[0070] In some embodiments, the step of analyzing one or more attributes of the key points of the image includes: performing center point clustering on the key points starting from M seeds to obtain M clusters, each cluster having one of the key points as the center point of the cluster, wherein the seeds are those key points among all the key points that form the convex hull of all the key points; constructing a graph for each cluster, wherein the vertices of the graph are the key points, and there is an edge between the center point of the cluster and each remaining key point in the cluster, wherein the edge connecting two vertices is marked with a weight indicating the strength of influence of one of the vertices on the other of the two vertices; obtaining a graph shift operator GSO of the graph; and using graph signal processing on the GSO to verify the watermark.

[0071] In some embodiments, the step of using graph signal processing on a graph to verify a watermark includes: performing a graph Fourier transform on the graph to obtain coefficients of the graph; performing a graph Fourier transform on attributes of the graph to obtain coefficients of the attributes of the graph; subtracting the coefficients of the graph from the coefficients of the attributes; applying a threshold to the result of the subtraction to extract a watermark on the sending side; and applying an XOR operation to the watermark on the sending side and a watermark known on the receiving side to verify the watermark.

[0072] In some embodiments, the method further includes, after performing a graph Fourier transform on the attribute of the graph: affecting the magnitude of a GFT coefficient of the attribute; and performing an inverse transform of the graph Fourier transform on the affected GFT coefficient to obtain a corrected feature.

[0073] Some embodiments relate to a computer-readable medium including computer-executable instructions stored thereon, which, when executed by one or more processors, cause the one or more processors to perform the method described above.

[0074] Some embodiments relate to a computing device comprising: one or more processors; and a computer-readable medium comprising computer-executable instructions which, when executed by the one or more processors, cause the one or more processors to perform the method described above.

[0075] Some embodiments relate to a device comprising a plurality of functional units configured to perform the method of any one of the preceding embodiments.

[0076] Now return to Figure 1 , which shows a first exemplary 1920×1088 pixel input image, called “Poznan Streets”, where a list of 8140 keypoints is calculated, each of which contains information about the keypoint. A keypoint has a position (i.e., x, y coordinates) and some attributes (such as scale σ and direction θ). The list of keypoints is shown in Table 1 below.

[0077] Table 1: A fragment of a list of vectors describing keypoints (without descriptors describing the nearest neighbors of the keypoints).

[0078] Figure 2 exist Figure 1 The key points in the image and the representation of scale and direction attributes are additionally marked on the basis of SIFT. The key points are obtained by applying SIFT technology to the image.

[0079] Figure 3 The computation of the convex hull on a set of 50 keypoints is shown. For example, by applying a feature extraction method such as SIFT to an image, a file containing all detected keypoints is obtained. The file is marked by determining the order in which the keypoints are listed in the file. This allows avoiding or reducing the impact of vector reordering attacks. This attack is possible because the keypoints are organized in the file as a list of vectors with attribute information. This can provide basic protection against unauthorized people tampering with the data. Changing the order of the vectors in the list does not affect the object detection / recognition application and does not result in sending more information, but knowing the serialization mechanism of consecutive vectors in the list provides additional protection against possible interference and information about possible interference. If the watermarking process heavily relies on the sorting order of the keypoints in the file, the attacker simply changes this order and the watermark is lost.

[0080] The idea is to explicitly select a subset of keypoints from the set of all keypoints and call this set of keypoints "keypoint seeds" or simply "seeds". The seeds will form the basis for further processing, and on the keypoints marked as seeds, a mechanism is established to mark the order of keypoints. Figure 3 As shown in the embodiment of , the seeds will be those keypoints that form a convex hull at the position (x, y) of the keypoint.

[0081] As used herein and generally defined, the term "convex hull" of a set of data points (i.e., keypoints) defines a line that completely encloses the set of points on a plane such that there are no concave points in the line. More formally, the convex hull is described as the smallest transitive polygon that has no concave points on the line. More formally, we can describe it as the smallest convex polygon that encloses a set of points such that every point in the set of points lies inside the polygon or on its perimeter.

[0082] To determine the convex hull, in some embodiments an algorithm is used that allows explicit definition of the first element of the seed set (the set of keypoints forming the convex hull).

[0083] In some embodiments, in two dimensions, the algorithm is also called Jarvis march. The idea of ​​the Jarvis algorithm is to start from the leftmost point (or the point with the smallest x-coordinate value) and keep wrapping the points in a counter-clockwise direction.

[0084] In some embodiments, to determine the convex hull, a seed is determined that is marked first. The seed can be one of four specific positions (leftmost point, topmost point, rightmost point, bottommost point) that are explicitly indicated in the unique key. The indication of the first seed can serve as an additional safety protection. If this indication is not used, the leftmost key point should be set as the first seed.

[0085] In some embodiments, the Jarvis algorithm may start with a seed explicitly indicated in the unique key and keep wrapping points in a counter-clockwise direction.

[0086] In some embodiments, to determine the convex hull, the seed that is marked first is determined. For example, this seed will be the keypoint with the smallest value in the x-coordinate. If two keypoints have the same minimum x-component value, then the keypoint with the smaller y-component value will be selected in this case.

[0087] For the sake of completeness, it should be noted that the SIFT technique allows the generation of multiple keypoints with different directional properties at the same position (x, y). In this case, the seed will be a pair of keypoints at the same position, and all subsequent operations applied to such a seed will be applied to all keypoints with the same position.

[0088] exist Figure 3In the embodiment shown, a plurality of cells having coordinates [(x=6, y=409), (x=7, y=194), (x=28, y=80), (x=44, y=47), (x=59, y=35), (x=175, y= 16),(x=441,y=29),(x=492,y=353),(x=360,y=425),(x=196,y=476)], M is the number of seeds. Subsequent keypoints starting from the first keypoint are assigned consecutive labels, i=1…M. These keypoints form a set of seeds in X is the set of all extracted keypoints of the image.

[0089] Figure 4 A method of generating a list of watermarked keypoints is shown. The method takes as input a set of features, which are obtained, for example, by feature extraction using a SIFT algorithm.

[0090] At 110, as already mentioned Figure 3 Explanation: Compute the convex hull of the keypoint locations to obtain a set of seeds.

[0091] At 120, the seeds are sorted, for example in descending or ascending order. Any sorting metric on the seed coordinates gives confidence in the proper sorting of the seeds. In some embodiments, a distance metric is used, such as the L2 metric (Euclidean distance, Euclidean norm) of the spatial coordinates (x, y) of the seeds.

[0092] However, since there are seeds whose L2 metrics may be the same as the L2 metrics after swapping the x and y values, in some embodiments, one of the components is boosted. Thus, for example, a factor of 2 is applied to the y component. If p i ={x i ,y i} represents the i-th seed point p i The coordinates of i ∈X S . p i The weighted L2-norm of is:

[0093] exist Figure 3 In the embodiment shown, p i The weighted L2 norm of can be obtained as follows: |p i|={578.4444658,274.4467161,116.5504183,79.71198153,77.01298592,176.4567936,442.9029239,700.9151161,700.6068798,701.1191054}

[0094] Once all L2 norms are calculated, they are sorted. Sorting the calculated L2 norms in descending order gives the seed order:

[0095] At 130, after the seeds have been sorted, they are shuffled by a pseudo-random number generator using a shuffle key k.

[0096] At 140, the seeds are removed from the remaining keypoints, which results in a distinction between seed keypoints and non-seed keypoints.

[0097] At 150, non-seed keypoints are mixed with seed keypoints. Although non-seed keypoints can be shuffled freely (among themselves and with seed keypoints), it is ensured that the order between seeds matches the order defined by the shuffle key k. The final seed order in the list of all keypoints depends on the knowledge of the shuffle key k. The key k is a mapping of the positional order between seeds. This is the advantage of using the convex hull technique and indicating the first seed. In some embodiments, the shuffle key k is stored to verify the order at the receiving end and restore the correct order.

[0098] It should be noted that the entire keypoint list does not need to be sorted, only the seeds. This is because it is known that tampering with the order of the seeds in the list indicates a violation of the integrity of the list.

[0099] Figure 5 A method of checking the integrity of a received list of watermarked keypoints at the receiver side is shown.

[0100] At 200, the convex hull is calculated over all the keypoints. No matter how the order of the keypoints is set on the sender side, no matter how the list is sorted, the convex hull will always look the same. It will always choose the same keypoint to focus on - the seed.

[0101] At 210, the coordinates of the seeds are sorted according to the L2 norm.

[0102] At 220, knowing the shuffle key k at the receiver side, the sequence of the seeds is determined.

[0103] At 230, it is checked (determined) whether the key points of the seeds in the feature list are sorted in the order of the seeds (determined at 220). Thus, the order of the seeds is determined and obtained. Thus, it is checked whether the key points of the seeds are arranged in the order they should be with respect to each other.

[0104] Therefore, in some embodiments, files containing keypoints should be marked by determining the order in which the keypoint vectors are listed therein. This step can avoid or reduce the impact of vector reordering attacks, such as Figure 5 As shown in the figure. This attack is possible because the keypoints are organized in the file as a list of vectors with attribute information. This provides basic protection against unauthorized people tampering with the data. Changing the order of the vectors in the list will not affect the object detection / recognition application and will not cause more information to be sent, but knowing the serialization mechanism of consecutive vectors in the list provides us with additional protection and tells us about possible interference. If the watermarking process heavily depends on the sorting order of the keypoints in the file, the attacker can simply change this order and the watermark will be lost. In the method proposed below, the watermark is embedded on the keypoint attributes, so we do not face this situation, but methods that specify the order in which the point vectors appear should be considered as an additional means of protection.

[0105] Figure 6 A second input image called "Poznan Parking Lot" with a resolution of 1920x1080 pixels encoded using the VVC encoder is shown. The question is how many seeds are statistically generated. This was studied for 3 different resolutions and two different images "Poznan Streets" and "Poznan Parking Lot". Statistically, the key should not exceed 15 bits.

[0106] Table 2: Number of all features and extracted seeds

[0107] As mentioned above, non-seed keypoints are not important to the methods described in this paper.

[0108] Figure 7 The convex hull (on 20 keypoints) of the SIFT keypoints (5596 of them) extracted from the input image "Poznan Parking Lot" is shown.

[0109] Figure 8 The process of centroid clustering of key points of the input image “Poznan Parking Lot” is shown starting from 20 seeds.

[0110] In more detail Figure 8Before, we recall the purpose of the method, which is to embed watermarks in the properties of keypoints. Introducing watermarks on the location of features is not considered, because this would falsify the location of the features and thus severely change the effectiveness of object detection and object recognition algorithms. The assumption is to modify the properties of the keypoints. In some embodiments, the properties are the orientation θ of the keypoint and the scale σ at which the keypoint detection is performed. Based on the assumption that keypoints reflect the structure in the image, the properties of keypoints clustered at a certain distance should not differ significantly.

[0111] like Figure 8 As shown, the key point X is clustered into M clusters, where the value of M is equal to the specified number of seeds. M medoids are determined, so M-medoid clustering will be used. M seeds are taken as the initial value of the clustering process, so that the medoid clustering ultimately obtains M clusters of key points. In some embodiments, two clustering levels are used to facilitate the generation of spatially adjacent sub-cluster groups. Dividing a group of key points into clusters allows the same watermark to be inserted into the key point attributes of each cluster. This allows protection to be implemented only for a portion of the group of key points. The watermark is replicated in each cluster.

[0112] In some embodiments, center point clustering is used. As commonly used and as used herein, the term "center point" refers to a representative object of a data set or a cluster in a data set, and the sum of its dissimilarity with all objects in the cluster is minimal. The center point is conceptually similar to the mean or centroid, but the center point is always limited to the members of the data set. This is an important feature, which makes it more advantageous to use the center point when the parameter representing the cluster is a representative of the set, compared to, for example, using the k-means clustering technique. For embodiments of the present invention, this means that the resulting center point will always be a key point that is important for subsequent processing.

[0113] In some embodiments, once clusters have been formed, each cluster is treated as a graph. Thus, in some embodiments, a graph will be constructed where one of the vertices will be a center point, and the remaining vertices (i.e., the remaining key points of the cluster) will be connected to the center point via edges. Just for the sake of understanding, it should be noted that if k-means clustering is used, a centroid will be obtained that does not reflect the actual key points in the set, and a graph that spans part of the key points of a given cluster cannot be created. This is why it is recommended to use center points in embodiments of the present invention.

[0114] In some embodiments, graph signal processing will be used to process the properties of the keypoints clustered and represented as a graph.As described above, a graph representation of keypoints clustered in clusters is introduced.

[0115] Figure 8 The results of centroid clustering are shown. Figure 8The asterisks in show the 20 initial keypoints on the convex hull (seeds), from which the centroid clustering process starts. Figure 8 The crosses shown in are the center points finally obtained as a result of the center point clustering process. These points are the key points clustered in different clusters (indicated by different gray shades).

[0116] Fig. 9 The process of embedding a watermark into the attributes of a feature is shown.

[0117] At 310, if Figure 8 As shown, an M-medoid clustering process with M seeds is performed. At 320, it is determined whether the last cluster has been reached. If not, the method continues to 330, where M graphs are constructed.

[0118] In some embodiments, for a given cluster, its graph G will be a triple G = {V, E, W}, including vertex V, edge E and weight W. A vertex or node is a set of n labels belonging to the key points of a cluster. A typical label is V = {1, ..., n}. An edge is an ordered pair of labels (i, j). (i, j) ∈ e is interpreted as i can be affected by j. Weight w ij ∈R is a number associated with edge (i, j) indicating the strength of influence of j on i, which can be some metric (e.g., the Euclidean distance determined between the coordinates of the keypoint locations, or also, for example, describing the relationship between the attributes of the keypoints) that will be determined between keypoints (more precisely, between the central point and the connected keypoints).

[0119] In some embodiments, a symmetric or undirected graph is used because it can be assumed that the edges are paired and the weights are symmetric. ij =w ji In some embodiments, for simplicity, it is also assumed that the graph is unweighted, which means that the weight w ij =1.

[0120] At 340, a graph shift operator representation of the graph G is obtained. As is well known, a graph can have a matrix representation. Therefore, a graph shift operator GSO is introduced. One representation of the graph G is to represent it as an adjacency matrix A. A is a sparse matrix A with non-zero elements. ij =w ij = 1. If the graph is symmetric, then the adjacency matrix is ​​also symmetric.

[0121] The neighborhood n(i) of node i is the set of nodes that influence i. The degree of node i is d i is the sum of the weights of its associated edges d i =∑ j∈n(i) w ijThe degree matrix is ​​a diagonal matrix D whose diagonal elements are the degrees D ii =d i . Expressed as an adjacency matrix, D = diag(A1). Because (A1) i =∑ j w ij =d i .

[0122] The Laplacian matrix of a graph with adjacency matrix A is L = DA = diag(A1) - A. The normalized adjacency matrix and the Laplacian matrix represent the weights relative to the degree of the nodes. The normalized adjacency matrix is If the graph is symmetrical Then the normalized adjacency matrix is ​​also symmetric. Normalized Laplacian matrix in is the identity matrix.

[0123] The graph shift operator S is a replacement for any matrix representation of a graph. S=A S=L

[0124] If the graph is symmetric, then the shift operator S is symmetric S = S T .

[0125] The characteristic vector v of the graph shift operator S is introduced i and eigenvalue λ i ,S→Sv i =λ i v i For symmetric S, the eigenvalues ​​are real numbers. They are in the order λ1≤λ2≤…≤λ n Sort. Define the eigenvector matrix V = [v1,…,v n ] and the eigenvalue matrix Λ=diag([λ1,…,λ n ]). The eigenvector decomposition of the graph shift operator can be defined as: S = VΛV T .

[0126] Given a graph shift operator S = VΛV T , the graph Fourier transform (GFT) of the graph signal x is GFT is the projection of the graph shift operator onto the feature space. It can be said that is the graph frequency representation of x, that is, the representation in the graph frequency domain. are the coefficients of GFT.

[0127] Given a graph shift operator S = VΛV T, GFT The inverse graph Fourier transform (IGFT) of Given Performing GFT and then IGFT on signal x can restore signal x, that is,

[0128] The watermark bits are embedded by affecting the amplitude of the GFT coefficients of the signal on the graph G. The GFT coefficients correspond to the direction θ and scale σ attributes of the key points. Therefore, there are two independent signals x on G θ and x σ .

[0129] At 350, two GFTs are calculated: and

[0130] The watermark U is a unique vector of n bits. The length of the vector cannot be greater than the number of keypoints contained in the cluster with the smallest count. In order to more easily affect the attributes, in some embodiments, a mapping is introduced so that

[0131] Assumptions represents the watermark at the sender. Indicates the watermark at the receiving end.

[0132] At 360, introduce the impact rule: and where α θ and α σ are parameters that control the strength with which the values ​​of the transform coefficients of the attribute signal are modified. The choice of these parameter values ​​should be driven by the range of attribute values.

[0133] Once the coefficients are affected, at 370, the IGFT is calculated and new attribute values ​​are assigned to the keypoints.

[0134] Fig.10 The process of verifying watermark embedding and extracting correct features is shown.

[0135] In some embodiments, when watermark extraction is to be performed, corresponding operations need to be performed on the key point set, namely, finding seeds by using the convex hull, performing center point clustering with the initial seeds, and spanning the graph to the clusters. Fig.10 Partly corresponds to Fig. 9 , i.e., steps 310-370 correspond to steps 410-470, so only the additional steps are discussed below.

[0136] At 480, the GFT coefficients on graph G are determined, which are given by express.

[0137] At 450, calculate and is the GFT coefficient of the signal θ on graph G, is the GFT coefficient of the signal σ on the graph G.

[0138] At 490, subtract the value and Make the two differences or A vector is formed, which includes positive or negative values ​​of a scale depending on the property of the transmitting side.

[0139] At 500, a threshold-based decision rule is applied such that positive values ​​are stored as 1 and negative values ​​are stored as 0, which allows the watermark embedded at the sending side to be extracted.

[0140] At 510, an XOR operation on the two binary strings will verify the watermark extracted from the attribute against the watermark known to the receiving side.

[0141] At 460, in order to correct the properties of the receiving end, an influence operation needs to be performed and in It is the watermark on the receiving side.

[0142] At 470, the IGFT is calculated on the two signals to obtain the corrected features.

[0143] It should be noted that the appearance of the GFT coefficients depends on the construction of the graph, more precisely, on the contents of the graph shift operator matrix. The magnitude of the coefficients can be carried by the signal values ​​in the graph. Therefore, by determining a certain form of GSO and performing a decomposition of eigenvectors and eigenvalues, the GFT coefficients can be determined.

[0144] Fig.11 A block diagram of a computing device 1000 is shown on which any of the embodiments described herein may be implemented. The computing device 1000 includes a bus 1002 or other communication mechanism for communicating information, and one or more hardware processors 1004 coupled to the bus 1002 and for processing information. The hardware processor 1004 may be, for example, one or more general purpose microprocessors.

[0145] The computing device 1000 also includes a main memory 1006, such as a random access memory (RAM), cache, and / or other dynamic storage device, coupled to the bus 1002 for storing information and instructions to be executed by the processor 1004. The main memory 1006 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by the processor 1004. When such instructions are stored in a storage medium accessible to the processor 1004, the computing device 1000 is presented as a special-purpose machine customized to perform the operations specified in the instructions.

[0146] The computing device 1000 also includes a read only memory (ROM) 1008 or other static storage device coupled to the bus 1002 to store static information and instructions for the processor 1004. A storage device 1010, such as a magnetic disk, optical disk, or USB thumb drive (flash drive), is provided and coupled to the bus 1002 to store information and instructions.

[0147] The computing device 1000 may be coupled to a display 1012, such as an LCD display (or touch screen), via the bus 1002 for displaying information to a computer user. An input device 1014 (including alphanumeric keys and other keys) is coupled to the bus 1002 to communicate information and command selections to the processor 1004. Another type of user input device is a cursor control 1016, such as a mouse, trackball, or cursor direction keys, for communicating direction information and command selections to the processor 1004 and for controlling cursor movement on the display 1012. An input device typically has two degrees of freedom on two axes, a first axis (e.g., x) and a second axis (e.g., y), which allows the device to specify a position in a plane. In some embodiments, the same direction information and command selection as cursor control may be achieved by receiving a touch on a touch screen without a cursor.

[0148] The computing device 1000 may include a user interface module to implement a GUI, which may be stored in a mass storage device as executable software code executed by the computing device. As examples, this module and other modules may include components (such as software components, object-oriented software components, class components, and task components), processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables.

[0149] In general, the term "module" as used herein refers to logic embodied in hardware or firmware, or to a collection of software instructions that may have entry and exit points, written in a programming language such as, for example, Java, C, or C++. A software module may be compiled and linked into an executable program, installed in a dynamic link library, or may be written in an interpreted programming language such as, for example, BASIC, Perl, or Python. It should be understood that a software module may be called from other modules or from itself, and / or may be called in response to a detected event or interrupt. A software module configured for execution on a computing device may be provided on a computer-readable medium such as an optical disc, a digital video disc, a flash drive, a disk, or any other tangible medium, or as a digital download (and may be initially stored in a compressed or installable format that requires installation, decompression, or decryption prior to execution). Such software code may be partially or completely stored on a memory device that executes a computing device for execution by the computing device. The software instructions may be embedded in firmware, such as an EPROM.

[0150] It should also be understood that a hardware module may include connected logic units (e.g., gates and triggers) and / or may include programmable units (e.g., programmable gate arrays or processors). The modules or computing device functions described herein are preferably implemented as software modules, but may be represented in hardware or firmware. Generally, the modules described herein refer to logic modules that can be combined with other modules or divided into sub-modules, regardless of their physical organization or storage. The computing device 1000 may implement the techniques described herein using customized hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic, the combination of which with the computing device makes the computing device 1000 a special-purpose machine or programs the computing device 1000 to make it a special-purpose machine. According to one embodiment, in response to the processor 1004 executing one or more sequences of one or more instructions contained in the main memory 1006, the computing device 1000 performs the techniques herein. Such instructions may be read from another storage medium (such as a storage device 1010) into the main memory 1006. The execution of the sequence of instructions contained in the main memory 1006 causes the processor 1004 to perform the processing steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.

[0151] As used herein, the term "non-transitory media" and similar terms refer to any medium that stores data and / or instructions that cause a machine to operate in a specific manner. Such non-transitory media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 1010. Volatile media include dynamic memory, such as main memory 1006. Common forms of non-transitory media include, for example, floppy disks, diskettes, hard disks, solid-state drives, magnetic tapes or any other magnetic data storage media, CD-ROMs, any other optical storage media, any physical media with a pattern of holes, RAM, PROMs and EPROMs, Flash-EPROMs, NVRAMs, any other memory chips or cassettes, and networked versions thereof.

[0152] Non-transitory media are distinct from, but may be used in conjunction with, transmission media. Transmission media participate in the transfer of information between non-transitory media. For example, transmission media include coaxial cables, copper wire, and optical fiber, including the wires that make up bus 1002. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0153] Various forms of media may be involved in carrying one or more sequences of one or more instructions to the processor 1004 for execution. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer may load the instructions into its dynamic memory and send the instructions over a telephone line using, for example, a modem or WiFi router. The modem or WiFi router local to the computing device 1000 may receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector may receive the data carried in the infrared signal and appropriate circuitry may place the data on the bus 1002. The bus 1002 transfers the data to the main memory 1006, from which the processor 1004 retrieves and executes the instructions. The instructions received by the main memory 1006 may optionally be stored on the storage device 1010 before or after execution by the processor 1004.

[0154] The computing device 1000 also includes a communication interface 1018 coupled to the bus 1002. The communication interface 1018 provides a two-way data communication coupled to one or more network links, which are connected to one or more local networks. For example, the communication interface 1018 can be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem that provides a data communication connection to a corresponding type of telephone line. As another example, the communication interface 1018 can be a local area network (LAN) card to provide a data communication connection to a compatible LAN (or a WAN component that communicates with a WAN). A wireless link can also be implemented. In any such implementation, the communication interface 1018 sends and receives electrical signals, electromagnetic signals, or optical signals that carry digital data streams representing various types of information.

[0155] A network link typically provides data communications to other data devices through one or more networks. For example, a network link may provide a connection to a host or to data devices operated by an Internet Service Provider (ISP) through a local network. The ISP, in turn, provides data communications services through the global packet data communications network now commonly referred to as the "Internet." Both local networks and the Internet use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and on the network link and through the communication interface 1018 are example forms of transmission media that transmit digital data to and from the computing device 1000.

[0156] The computing device 1000 can send messages and receive data, including program code, through a network, a network link, and a communication interface 1018. In the Internet example, a server can send a request code for an application program through the Internet, an ISP, a local network, and a communication interface 1018. The computing device 1000 on which an embodiment of the present invention can be implemented can be a server of a client-server system. The client can interact with the server so that an embodiment of the present invention as disclosed herein can be executed. The program implementing the present invention can be executed on the server, or can be downloaded to the client and executed on the client or another computer. In this regard, reference is made to the common or indirect patent infringement principles applicable to many jurisdictions. The received code can be executed by the processor 1004 when it is received, and / or stored in the storage device 1010 or other non-volatile storage device for later execution. The computing device 1000 can also be implemented as a (virtual) device, for example, in the form of software.

[0157] Each of the processes, methods, and algorithms described in the foregoing sections can be embodied in a code module executed by one or more computing devices or computer processors including computer hardware, and fully or partially automated by the code module. The processes and algorithms can be implemented partially or entirely in a dedicated circuit.

[0158] The various features and processes described above can be used independently of each other, or can be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. In addition, certain method or process blocks may be omitted in some implementations. The methods and processes described herein are also not limited to any particular sequence, and the blocks or states associated therewith may be performed in other appropriate sequences. For example, the blocks or states described may be performed in a sequence different from that specifically disclosed, or multiple blocks or states may be combined in a single block or state. The example blocks or states may be performed serially, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed example embodiments. The example devices, systems, and components described herein may be configured differently from those described. For example, elements may be added to, removed, or rearranged compared to the disclosed example embodiments.

[0159] Unless specifically stated otherwise or otherwise understood in the context of use, conditional language such as "may," "could," "might," or "can" is generally intended to convey that some embodiments include and other embodiments do not include certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that features, elements, and / or steps are required of one or more embodiments, or that one or more embodiments must include logic for deciding, with or without user input or prompting, whether such features, elements, and / or steps are included in any particular embodiment or will be performed.

[0160] Any process description, element or block in the flowcharts described herein and / or depicted in the accompanying drawings should be understood to potentially represent a module, segment or portion of code, which includes one or more executable instructions for implementing specific logical functions or steps in the process. As will be appreciated by those skilled in the art, alternative embodiments are included within the scope of the embodiments described herein, in which elements or functions may be deleted or executed out of order from those shown or discussed, including substantially simultaneously or in reverse order, depending on the functions involved.

[0161] The expressions "comprising a, b, c, ...", "including a, b, c, ..." and "based on a, b, c, ..." refer to a non-exhaustive enumeration of items. It should be emphasized that many variations and modifications may be made to the above-described embodiments, elements of which should be understood as other acceptable examples. All such modifications and variations are intended to be included within the scope of the present disclosure. The above description details certain embodiments of the present invention. However, it should be understood that no matter how detailed the foregoing is in the text, the present invention may be implemented in a variety of ways. As noted above, it should be noted that the use of specific terms in describing certain features or aspects of the present invention should not be construed as implying that the term is redefined herein to be limited to including any specific features of the features or aspects of the present invention associated with the term. Therefore, the scope of the present invention should be interpreted in accordance with the appended claims and their equivalents.

Claims

1. A method for generating a list of key points associated with an image, the method comprising: Get a list of key points, each of which consists of a position and one or more attributes; Determine a convex hull of the position of the key point, wherein the key point forming the convex hull is called a seed; sorting the seeds according to a sorting metric; shuffling the sorted seeds using the key; and The sorted seed keypoints are blended with non-seed keypoints.

2. The method according to claim 1, wherein: The list of keypoints has watermarks embedded on the attributes of the keypoints.

3. The method according to claim 1, wherein: The ranking metric is an L2 metric or a weighted L2 metric.

4. A method according to any one of the preceding claims, wherein: The seeds are sorted in descending or ascending order.

5. A method according to any one of the preceding claims, wherein: The key is a shuffle key that defines a mapping between the order of positions between the seeds.

6. A method according to any one of the preceding claims, wherein: The convex hull of the key points is determined using the Jarvis algorithm.

7. A method according to any one of the preceding claims, wherein: To determine the convex hull, first a seed is identified which is marked.

8. The method according to claim 7, wherein: The seed is one of four specific locations explicitly indicated in the key.

9. A method according to any one of the preceding claims, wherein: The attributes are scale and orientation, and each keypoint also includes a descriptor, which is a local feature descriptor extracted from a local image region around the location of the keypoint.

10. A method according to any one of the preceding claims, wherein: The features represented by the key points are obtained using the scale-invariant feature transform (SIFT) technique.

11. A method of checking the correctness of a list of key points associated with an image, the method comprising: Get a list of key points, each of which consists of a position and one or more attributes; Determine a convex hull of the position of the key point, wherein the key point forming the convex hull is called a seed; sorting the seeds according to a sorting metric; shuffling the sorted seeds using a key to determine an order of the seeds; and Check whether the seeds in the list of keypoints are sorted in the order of the seeds.

12. The method according to claim 11, wherein: The list of keypoints has watermarks embedded on the attributes of the keypoints.

13. The method according to claim 11 or 12, wherein: The seeds are ranked using an L2 metric or a weighted L2 metric.

14. The method according to any one of claims 11 to 13, wherein: The seeds are sorted in descending or ascending order.

15. The method according to any one of claims 11 to 14, wherein: The key is a shuffle key that defines a mapping between the order of positions between the seeds.

16. The method according to any one of claims 11 to 15, wherein: The convex hull of the key points is determined using the Jarvis algorithm.

17. A method according to any one of the preceding claims, wherein: To determine the convex hull, first a seed is identified which is marked.

18. The method according to claim 17, wherein: The seed is one of four specific locations explicitly indicated in the key.

19. The method according to any one of claims 11 to 18, wherein: The attributes are scale and orientation, and the keypoint further comprises a descriptor, which is a local feature descriptor extracted from a local image region around the location of the keypoint.

20. The method according to any one of claims 11 to 19, wherein: The scale-invariant feature transform (SIFT) technique is used to obtain features.

21. A computer readable medium comprising computer executable instructions stored thereon which, when executed by one or more processors, cause the one or more processors to perform the method of any one of the preceding claims.

22. A computing device comprising: one or more processors; as well as A computer readable medium comprising computer executable instructions which, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 20.

23. A device comprising: A key point acquisition unit is configured to acquire a list of key points, each key point including a position and one or more attributes; a convex hull determining unit configured to determine a convex hull of the position of the key point, wherein the key point forming the convex hull is called a seed; a sorting unit configured to sort the seeds according to a sorting metric; a shuffling unit configured to shuffle the sorted seeds using the key; and A mixing unit is configured to mix the sorted seed key points with non-seed key points.

24. An apparatus comprising: A key point acquisition unit is configured to acquire a list of key points, each key point including a position and one or more attributes; a convex hull determining unit configured to determine a convex hull of the position of the key point, wherein the key point forming the convex hull is called a seed; a sorting unit configured to sort the seeds according to a sorting metric; a shuffling unit configured to shuffle the sorted seeds using a key to determine an order of the seeds; and A checking unit is configured to check whether the seeds in the list of key points are sorted in the order of the seeds.

25. A method for embedding a watermark in an attribute of a feature of an image, the method comprising: Get a list of keypoints associated with an image, each keypoint consists of a location and one or more attributes; as well as A watermark is embedded into the image by modifying one or more properties of the key points of the image.

26. The method according to claim 25, wherein: The step of embedding the watermark into the attributes of the key points of the image by modifying one or more attributes of the image comprises: Starting from M seeds, center point clustering is performed on the key points to obtain M clusters, each cluster having one of the key points as the center point of the cluster, wherein the M seeds are key points that form a convex hull of all key points.

27. The method according to claim 25 or 26, further comprising: Constructing a graph for each cluster, wherein vertices of the graph are key points, and there is an edge between the center point of the cluster and each remaining key point of the cluster, and the edge connecting two vertices is marked with a weight, wherein the weight indicates the strength of influence of one of the vertices on the other of the two vertices; Obtaining a graph shift operator GSO for each graph; and Graph signal processing is performed on each GSO to determine the modification of the properties.

28. A method according to any one of claims 25 to 27, wherein: Performing graphics signal processing includes: Performing a graph Fourier transform (GFT) on the attribute on the graph to obtain a GFT coefficient of the attribute; affecting the magnitude of the GFT coefficients of the attribute; and An inverse graph Fourier transform is performed on the affected GFT coefficients to obtain a modified value for the property.

29. The method according to claim 28, wherein: The graph Fourier transform is the projection of the GSO on the feature space.

30. An apparatus comprising: A key point acquisition unit is configured to acquire a list of key points associated with the image, each key point including a position and one or more attributes; as well as The embedding unit is configured to embed the watermark into the image by modifying the one or more properties of the key points of the image.

31. A method for verifying a watermark associated with an image, comprising: receiving a list of keypoints associated with the image, each keypoint comprising a location and one or more attributes; as well as The watermark in the image is verified by analyzing the one or more properties of the key points of the image.

32. The method according to claim 31, wherein: The step of analyzing the one or more attributes of the key points of the image comprises: Starting from M seeds, performing center point clustering on the key points to obtain M clusters, each cluster having one of the key points as the center point of the cluster, wherein the seeds are those key points among all the key points that form a convex hull of all the key points; Constructing a graph for each cluster, wherein vertices of the graph are key points, and there is an edge between the center point of the cluster and each remaining key point of the cluster, and the edge connecting two vertices is marked with a weight, wherein the weight indicates the strength of influence of one of the vertices on the other of the two vertices; Obtaining a graph shift operator GSO of the graph; and The watermark is verified using graph signal processing on the GSO.

33. The method according to claim 31 or 32, wherein: The step of using graph signal processing on the graph to verify the watermark comprises: performing a graph Fourier transform on the graph to obtain coefficients of the graph; performing a graph Fourier transform on the attribute of the graph to obtain coefficients of the attribute of the graph; subtracting coefficients of the graph from coefficients of the attribute; applying a threshold to the result of the subtraction to extract the watermark at the transmitting side; and An XOR operation is applied to the watermark at the sending side and the watermark known to the receiving side to verify the watermark.

34. The method according to any one of claims 31 to 33, after performing a graph Fourier transform on the attributes of the graph, further comprising: affecting the magnitude of the GFT coefficients of the attribute; as well as An inverse graph Fourier transform is performed on the affected GFT coefficients to obtain corrected features.

35. An apparatus comprising: A receiving unit configured to receive a list of key points associated with an image, each key point including a position and one or more attributes; as well as A verification unit is configured to verify the watermark in the image by analyzing one or more attributes of the key points of the image.

36. A computer readable medium comprising computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform the method of any one of claims 25 to 29 and claims 31 to 34.

37. A computing device comprising: one or more processors; as well as A computer readable medium comprising computer executable instructions which, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 25 to 29 and claims 31 to 34.

Images