Image comparison, image infringement detection method and device

Abstract

The embodiments of this specification provide an image comparison and image infringement detection method and device. In the image comparison method, two images to be compared are obtained, and for any image, a neural network is used to extract the feature map of the image in different layers, and multiple feature blocks are intercepted from multiple feature maps to obtain a feature block group corresponding to the image; the feature blocks between the feature block groups of the two images are compared pairwise, and based on the comparison results, the similar areas between the two images are determined. The image comparison method can be applied to the image comparison process in image infringement detection.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of image processing technology, and in particular, to an image comparison and image infringement detection method and device. Background Art

[0002] With the advancement of technology and the development of society, images are being used more and more widely, for example, in visual processing, object comparison and search, image infringement detection and other fields. Similarity comparison between images is a basic processing process in many application fields. With the continuous development of image processing technology, the requirements for similarity comparison between images are getting higher and higher. Therefore, it is hoped that there will be an improved solution that can be more detailed and more comprehensive when performing similarity comparison on images. Summary of the invention

[0003] One or more embodiments of this specification describe an image comparison and image infringement detection method and device, so as to more carefully and fully compare similar areas between two images. The specific technical solution is as follows.

[0004] In a first aspect, an embodiment provides an image comparison method, comprising:

[0005] Obtain two images to be compared;

[0006] For any image, a neural network is used to extract the feature maps of the image in different layers, and multiple feature blocks are intercepted from multiple feature maps to obtain a feature block group corresponding to the image;

[0007] Performing pairwise comparison on the feature blocks between the feature block groups of the two images;

[0008] Based on the comparison result, a similar region between the two images is determined.

[0009] In one embodiment, the step of extracting a plurality of feature blocks from a plurality of feature maps comprises:

[0010] For different feature maps, a corresponding sliding window is used to intercept multiple feature blocks on the feature map to obtain the feature block corresponding to the feature map;

[0011] A plurality of feature blocks corresponding to a plurality of feature maps are determined as a feature block group corresponding to the image.

[0012] In one implementation, different feature maps correspond to different sliding window sizes.

[0013] In one embodiment, the step of performing pairwise comparison of feature blocks between the feature block groups of the two images comprises:

[0014] Based on the feature vectors of the feature blocks, feature blocks between the feature block groups of the two images are compared in pairs to obtain multiple similarities between the feature blocks.

[0015] In one embodiment, the step of performing pairwise comparison of feature blocks between the feature block groups of the two images comprises:

[0016] Determining feature vectors of feature blocks in the feature block groups of the two images through a first feature extraction network;

[0017] For any two feature blocks between the feature block groups of the two images, the similarity between the feature vectors of the two feature blocks is determined as the similarity between the two feature blocks.

[0018] In one embodiment, the step of performing pairwise comparison of feature blocks between the feature block groups of the two images comprises:

[0019] For any image, any feature block in the feature block group of the image is compared with all feature blocks in the feature block group of another image.

[0020] In one embodiment, the step of determining the similarity region between the two images based on the comparison result includes:

[0021] Determining a first number of feature block pairs with the greatest similarity from the feature block groups of the two images;

[0022] Based on the characteristic elements contained in the characteristic blocks, performing a detailed comparison on the first number of characteristic block pairs;

[0023] Based on the refined comparison result, a similar region between the two images is determined.

[0024] In one implementation, the step of finely comparing the first number of feature block pairs includes:

[0025] For any feature block pair, the feature elements contained in the feature block pair are input into the detection network, and the similar area in the feature block pair is determined by the detection network as the fine comparison result of the feature block pair.

[0026] In one embodiment, the step of inputting the feature elements included in the feature block pair into the detection network includes:

[0027] Determine, by a second feature extraction network, feature vectors corresponding to the first number of feature block pairs, and determine the similarity between each feature block pair based on the feature vectors;

[0028] Filter feature block pairs whose similarity is greater than a preset similarity threshold;

[0029] The feature elements contained in the filtered feature block pairs are input into the detection network.

[0030] In one embodiment, the neural network includes an FPN network, a CNN network or a Transformer network.

[0031] In a second aspect, the embodiment provides a blockchain-based image infringement detection method, including:

[0032] Acquire a first image to be detected;

[0033] Acquire multiple second images in the image library from the blockchain network;

[0034] Comparing the first image with the plurality of second images respectively to obtain similar regions between the first image and the plurality of second images respectively; wherein the first image and any one of the second images are taken as two images to be compared, and the similar regions between the two images are determined by the method provided in the first aspect;

[0035] Based on the obtained similar area, it is determined whether there is an image infringing upon the rights of the first image in the plurality of second images.

[0036] In a third aspect, an embodiment provides an image comparison device, including:

[0037] A first acquisition module is configured to acquire two images to be compared;

[0038] A first extraction module is configured to extract feature maps of any image in different layers using a neural network, and to intercept multiple feature blocks from multiple feature maps to obtain a feature block group corresponding to the image;

[0039] A first comparison module is configured to compare feature blocks between feature block groups of the two images in pairs;

[0040] The first determination module is configured to determine a similar area between the two images based on the comparison result.

[0041] In one implementation, the first comparison module is specifically configured as follows:

[0042] Based on the feature vectors of the feature blocks, feature blocks between feature block groups of two images are compared pairwise to obtain multiple similarities between the feature blocks.

[0043] In one implementation, the first determining module includes:

[0044] A second determination submodule is configured to determine a first number of feature block pairs with the greatest similarity from the feature block groups of the two images;

[0045] A first comparison submodule is configured to perform a detailed comparison on the first number of feature block pairs based on feature elements included in the feature blocks;

[0046] The third determination submodule is configured to determine the similarity region between the two images based on the fine comparison result.

[0047] In a fourth aspect, the embodiment provides an image infringement detection device based on blockchain, including:

[0048] A second acquisition module is configured to acquire a first image to be detected;

[0049] A third acquisition module is configured to acquire a plurality of second images in the image library from the blockchain network;

[0050] A second comparison module is configured to compare the first image with the plurality of second images respectively to obtain similar regions between the first image and the plurality of second images respectively; wherein the first image and any one of the second images are used as two images to be compared, and the similar regions between the two images are determined by the method provided in the first aspect;

[0051] The second determination module is configured to determine whether there is an image infringing upon the rights of the first image in the plurality of second images based on the obtained similar area.

[0052] In a fifth aspect, an embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute any one of the methods in the first aspect and the second aspect.

[0053] In a sixth aspect, an embodiment provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in any one of the first aspect and the second aspect is implemented.

[0054] In the method and device provided in the embodiments of this specification, for any one of the two images to be compared, a neural network is used to extract the feature map of the image in different layers, and multiple feature blocks are intercepted from multiple feature maps, so as to compare the feature blocks in the feature block groups of the two images. The feature maps of different layers reflect the features of the image in different dimensions and are the expressions of the image in different complexities. The feature block group contains feature blocks of different sizes of the image in different layers. The feature blocks in the feature block groups of the two images are compared, so as to realize the comparison between different feature blocks of the images in different dimensions, and can more carefully and fully compare the similar areas between the two images. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A schematic diagram of a flow chart of an image comparison method provided in an embodiment;

[0057] Figure 2 A schematic diagram of an implementation process provided for this embodiment;

[0058] Figure 3 A flowchart of a blockchain-based image infringement detection method provided in an embodiment;

[0059] Figure 4 A schematic block diagram of an image comparison device provided in an embodiment;

[0060] Figure 5 A schematic block diagram of a blockchain-based image infringement detection device provided in an embodiment. DETAILED DESCRIPTION

[0061] The solution provided in this specification is described below in conjunction with the accompanying drawings.

[0062] Image acquisition devices, such as cameras, video cameras or camera phones, can acquire various images. For images, applications such as analysis, comparison, and search can be performed. Similarity comparison between different images is a basic processing process in many application fields.

[0063] When performing a similarity comparison between two images, image features are usually extracted, and the image similarity comparison is achieved by comparing the image features of different images. When performing a similarity comparison between images, in most cases, a global feature based on the entire image is used for similarity comparison. For example, a feature map of the entire image can be extracted through a neural network, and different feature maps can be compared.

[0064] However, some application areas have higher requirements for the image comparison process. For example, in the field of image infringement detection, users request to protect the copyright of images. Before authorizing and protecting the user's image, it is necessary to first search whether there is an authorized image similar to the image in the image library. When there is no authorized image similar to the image in the image library, the image can be added to the image library to make it an authorized image.

[0065] Using the global features of the image for similarity comparison can detect similar images from the overall image, but for spliced ​​images or images with picture-in-picture, this comparison method cannot achieve good detection performance.

[0066] In order to compare two images more carefully and fully, an embodiment of the present specification provides an image comparison method. The method includes the following steps: step S110, obtaining two images to be compared; step S120, for any image, using a neural network to extract the feature map of the image in different layers, cutting multiple feature blocks from multiple feature maps, and obtaining a feature block group corresponding to the image; step S130, performing pairwise comparison of the feature blocks between the feature block groups of the two images; step S140, based on the comparison results, determining the similarity between the two images. In this embodiment, the feature maps of the image in different layers are determined, and multiple feature blocks are cut from multiple feature maps to compare the similarity between the feature blocks in the feature block groups of the two images. The feature maps of different layers of the image extracted by the neural network reflect the features of the image in different dimensions and are the expressions of the image in different complexities. The feature maps are cut into different feature blocks, and the feature blocks of the two images are compared to obtain the similarity between the two images. It can be seen that such a comparison process is more detailed and the comparison process is more complete. The following is combined with Figure 1 This embodiment is described in detail.

[0067] Figure 1 A flowchart of an image comparison method provided in an embodiment. The method is executed by a computing device, and the computing device can be implemented by any device, equipment, platform, equipment cluster, etc. with computing and processing capabilities. The method includes the following steps.

[0068] In step S110, two images to be compared are obtained. The two images to be compared can be any images, regardless of the content of the images or whether the images contain the same objects such as people, cars, trees or dogs. The two images can be the same size or different sizes; they can both be grayscale images or color RGB (red, green and blue) images, or one can be a grayscale image and the other can be an RGB image.

[0069] In step S120, for any image, a neural network is used to extract feature maps of the image in different layers, and multiple feature blocks are intercepted from multiple feature maps to obtain a feature block group corresponding to the image. Both images are determined using this method. The following takes any image A as an example to illustrate the process of determining the feature block group corresponding to the image A.

[0070] In a specific implementation, image A can be input into a neural network, and feature maps of the image in different layers can be extracted through the neural network. Image A can be directly input into the neural network, or image A can be pre-processed before input into the neural network. Pre-processing includes changing the size, grayscale or denoising of the image. The feature maps of the image in different layers are also called different feature layers of the image.

[0071] Among them, the neural network usually includes multiple layers, and there can be a transmission relationship of intermediate data between the multiple layers. The multiple layers can be divided into low layers and high layers according to the direction of data transmission. For example, the neural network can be implemented by using Feature Pyramid Networks (FPN), Convolutional Neural Networks (CNN) or Transformer networks. For example, in a convolutional neural network, the multiple layers include several convolutional layers, pooling layers and fully connected layers. The process of extracting image features from low layers to high layers by the neural network is a process of concentrating image features step by step. Different layers can extract the features of the image in different dimensions and obtain their feature maps. The feature maps extracted by different layers reflect the image information at different levels contained in the image. The image features extracted by the lower layers (the feature map is the image features) can reflect the image information at a shallower level, such as the edges in the image; while the image features extracted by the higher layers can reflect the image information at a deeper level, such as the contours of objects or the categories of objects in the image. The dimensions of the feature maps extracted by different layers are generally different. The feature maps extracted by the lower layers have higher dimensions and larger sizes, while the feature maps extracted by the higher layers have lower dimensions and smaller sizes.

[0072] It is possible to extract one feature map from each layer, or to extract more than one feature map from one layer. For example, a neural network is used to extract m feature maps of image A in m layers. When extracting feature blocks from multiple feature maps, for different feature maps, for example, for feature map F1, a corresponding sliding window can be used to extract multiple feature blocks on the feature map F1 to obtain feature blocks corresponding to the feature map F1, and multiple feature blocks corresponding to several feature maps are determined as a feature block group corresponding to the image A. The m feature maps are all extracted into multiple feature blocks, and the k feature blocks corresponding to the m feature maps can form a feature block group of image A.

[0073] The sliding window sizes corresponding to different feature maps may be different or the same. The size of the sliding window may be preset, for example, a larger sliding window may be set corresponding to a feature map of a larger size, and a smaller sliding window may be set corresponding to a feature map of a smaller size. One feature map may correspond to one sliding window or multiple sliding windows.

[0074] Each sliding window can also have a corresponding step size when moving. When using a sliding window to intercept multiple feature blocks on a feature map, it moves from left to right or from top to bottom according to the step size. Each time it moves, the area within the sliding window is a feature block. A feature block is a part of a feature map. Both feature blocks and feature maps contain multiple feature elements.

[0075] In step S130, feature blocks between feature block groups of the two images are compared in pairs. In step S140, similar regions between the two images are determined based on the comparison results.

[0076] When the feature blocks between the feature block groups of two images are compared in pairs, for any image A, any feature block in the feature block group of image A can be compared with all the feature blocks in the feature block group of another image B, until all the feature blocks in the feature block group of image A are compared with all the feature blocks in the feature block group of another image B. For example, the feature block group of image A contains 10 feature blocks, and the feature block group of image B contains 10 feature blocks. When the feature blocks between the feature block groups of the two images are compared in pairs, a total of 100 pairs of feature blocks need to be compared.

[0077] When the feature blocks between the feature block groups of two images are compared in pairs, the feature blocks between the feature block groups of the two images can be compared in pairs based on the feature vectors of the feature blocks to obtain multiple similarities between the feature blocks. Specifically, the feature vectors of the feature blocks in the feature block groups of the two images can be determined through the first feature extraction network. For example, the feature elements contained in the feature blocks can be input into the first feature extraction network to obtain the feature vectors of the output feature blocks. For feature blocks of different sizes, the first feature extraction network can be set to output feature vectors of the same dimension. Then, for any two feature blocks between the feature block groups of the two images, the similarity between the feature vectors of the two feature blocks is determined as the similarity between the two feature blocks. When determining the similarity between the two feature vectors, some common methods can be used, such as the edit distance method, the angle cosine method or the Euclidean distance method. The first feature extraction network is used to extract the feature vectors of the feature blocks. The first feature extraction network can be obtained after training the neural network.

[0078] In step S130, based on the feature vectors of the feature blocks, feature blocks between the feature block groups of the two images are compared in pairs, which can be understood as a rough comparison process. Generally speaking, there are a large number of feature blocks in a feature block group of an image, and the number of comparisons will be more when the pairs are compared. By using the similarity between the feature vectors of the feature blocks for rough comparison, some feature block pairs with high similarity can be quickly found in a large number of comparisons. Therefore, after the rough comparison, a fine comparison can also be performed. That is, in step S140, when determining the similar area between the two images based on the comparison results, the following sub-steps can be included:

[0079] Step 1, determining a first number n of feature block pairs with the greatest similarity from the feature block groups of the two images;

[0080] Step 2, based on the characteristic elements contained in the characteristic blocks, perform a detailed comparison on the n characteristic block pairs;

[0081] Step 3: Based on the detailed comparison results, determine the similarity area between the two images.

[0082] The comparison result obtained in the rough comparison process in step S130 includes the similarity between any two feature blocks, so that the similarity between multiple feature block pairs is obtained. When determining the n feature block pairs with the largest values ​​from multiple similarities in step 1, the multiple similarities can be first sorted in order from large to small, and then the feature block pairs corresponding to the first n similarities in the sorting are determined as the n feature block pairs with the largest similarity. n is an integer and can also be preset.

[0083] When performing fine comparison on n feature block pairs in step 2, specifically, for any feature block pair, the feature elements contained in the feature block pair can be input into the detection network, and the similar area in the feature block pair can be determined by the detection network as the fine comparison result of the feature block pair. The detection network can be a pre-trained neural network. When the feature elements contained in the feature block pair are input into the detection network, the feature elements of the two feature blocks contained in the feature block pair can be input into the detection network together in a preset order. The detection network can compare the feature elements contained in the two feature blocks, determine the fine similar areas in the two feature blocks, as the fine comparison result; or output the fine similar areas in the two feature blocks corresponding to the similar areas in the two images. For example, for feature block 1 and feature block 2, the detection network can output the rectangular area coordinates (a1, b1) and (c1, d1) in feature block 1, and output the rectangular area coordinates (a2, b2) and (c2, d2) in feature block 2, and these two rectangular areas are more fine similar areas. The detection network can also continue to map the coordinates of the two rectangular regions to the corresponding images to obtain the region in image A and the region in image B. These two regions are similar regions between the two images.

[0084] In step 3, if the fine comparison result includes similar regions between feature block pairs, the similar regions between feature blocks can be mapped to similar regions between two images. If the fine comparison result includes similar regions between two images, the fine comparison result can be directly determined as similar regions between two images.

[0085] The detailed comparison result may include similar regions obtained based on a pair of feature blocks, or similar regions obtained based on multiple pairs of feature blocks.

[0086] In step 2, before the feature elements contained in the feature block pairs are input into the detection network, the first number of feature blocks can also be screened. Specifically, the feature vectors corresponding to the first number n feature block pairs can be determined through the second feature extraction network. For example, for any one of the n feature block pairs, the feature elements contained in the feature block can be input into the second feature extraction network to obtain the feature vector corresponding to the feature block. The dimensions of the feature vectors extracted from different feature blocks can be the same. Then, based on the feature vector, the similarity between each feature block pair is determined, and the feature block pairs whose similarity is greater than the preset similarity threshold are screened, and the feature elements contained in the screened feature block pairs are input into the detection network.

[0087] The second feature extraction network is used to extract the feature vector of the feature block. The second feature extraction network can be a neural network different from the first feature extraction network, and the second feature extraction network can be a better trained feature extraction network, and the feature vector extracted by the second feature extraction network has higher accuracy.

[0088] Figure 2 A schematic diagram of an implementation process provided for this embodiment. Image A and Image B are two images to be compared. The two images are input into a neural network, feature maps of Image A and Image B in different layers can be extracted, and multiple feature blocks can be obtained from multiple feature maps using a sliding window. Figure 2 An example of feature elements contained in a feature map of image A and image B is given in the figure. By moving the sliding window shown by the dotted line in the feature map according to the set step size, multiple feature blocks can be intercepted, thereby obtaining a feature block group of image A and a feature block group of image B, each of which contains feature blocks of different sizes. The feature blocks in the two feature block groups are compared pairwise to obtain the similarity between each group of feature blocks, and n feature block pairs with the greatest similarity are selected. The feature elements contained in the selected feature block pairs are input into the detection network. Based on the output results of the detection network, the similar areas between image A and image B can be determined. Figure 2 The similar areas are indicated by white rectangular frames.

[0089] Among them, when feature block comparison is performed between feature block groups, the comparison is performed based on the feature vectors of the feature blocks, and this process can be considered as a rough comparison stage. On the basis of n feature block pairs screened out in the rough comparison stage, when the screened feature block pairs are compared again through the detection network, the comparison is performed based on the feature elements contained in the feature blocks, and this process can be considered as a fine comparison stage. Through the combination of rough comparison and fine comparison, this embodiment can improve the comparison efficiency while taking into account the comparison accuracy.

[0090] The above is a detailed description of the image comparison process. Figure 3 The application process of image matching in the field of infringement detection is explained.

[0091] Figure 3 A flowchart of a blockchain-based image infringement detection method provided in an embodiment. The method is executed by a computing device. The method includes the following steps:

[0092] Step S310, obtaining a first image to be detected; Step S320, obtaining multiple second images in an image library from a blockchain network; Step S330, comparing the first image with the multiple second images respectively to obtain similar areas between the first image and the multiple second images respectively; Step S340, based on the obtained similar areas, determining whether there is an image infringed by the first image in the multiple second images.

[0093] In step S310, the first image can be any image, for example, an image sent by a user and waiting for authorization. In step S320, multiple second images in the image library stored in the blockchain network can be understood as images that have been authorized. In step S330, the first image and any second image are used as two images to be compared. Figure 1 The method provided in the embodiment determines the similarity region between two images. In this way, the similarity region between the first image and all the second images is determined, thereby obtaining multiple similarity regions. For a specific implementation method, see Figure 1 The embodiment shown.

[0094] In step S340, based on multiple similar regions, when determining whether there is an image infringed by the first image in multiple second images, it can be determined whether the area of ​​the similar region is greater than a preset area threshold. If so, it is determined that the second image corresponding to the similar region is infringed by the first image; it can also be determined whether the proportion of the similar region is greater than a preset proportion threshold. If so, it is determined that the second image is infringed by the first image. The proportion of the similar region can be the proportion of the similar region in the second image, or the proportion of the similar region in the first image, or a fusion proportion obtained based on the two proportions. If it is determined that there is no image infringed by the first image in the multiple second images, the first image can be stored in the blockchain network through transactions to achieve authorization of the first image.

[0095] In the field of infringement detection, it is not only necessary to detect whether the images are similar as a whole, but also to pay more attention to whether the detail areas between the images are similar. Applying the image comparison process provided by the above embodiment in the field of infringement detection can improve the processing accuracy in the field of infringement detection and avoid unnecessary problems.

[0096] The above is just Figure 1 The image comparison method provided in the embodiment is applied to an implementation in the field of infringement detection. Figure 1 The image comparison method can also be applied in a variety of scenarios, such as lost property finding, target tracking, etc., which will not be described in detail.

[0097] In this specification, the word "first" in terms such as the first image, the first feature extraction network, the first quantity, and the corresponding "second" in the text are merely for the convenience of distinction and description and do not have any limiting meaning.

[0098] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0099] Figure 4 A schematic block diagram of an image comparison device provided in an embodiment. Figure 1 The method embodiment shown corresponds to the embodiment shown. The apparatus is deployed in a computing device, and the computing device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. The apparatus 400 includes:

[0100] A first acquisition module 410 is configured to acquire two images to be compared;

[0101] The first extraction module 420 is configured to extract feature maps of any image in different layers using a neural network, and extract multiple feature blocks from multiple feature maps to obtain a feature block group corresponding to the image;

[0102] A first comparison module 430 is configured to perform pairwise comparison of feature blocks between the feature block groups of the two images;

[0103] The first determination module 440 is configured to determine a similar region between the two images based on the comparison result.

[0104] In one implementation, when the first extraction module 420 extracts a plurality of feature blocks from a plurality of feature maps, it includes:

[0105] For different feature maps, a corresponding sliding window is used to intercept multiple feature blocks on the feature map to obtain the feature block corresponding to the feature map;

[0106] A plurality of feature blocks corresponding to a plurality of feature maps are determined as a feature block group corresponding to the image.

[0107] In one implementation, different feature maps correspond to different sliding window sizes.

[0108] In one implementation, the first comparison module 430 is specifically configured as follows:

[0109] Based on the feature vectors of the feature blocks, feature blocks between the feature block groups of the two images are compared in pairs to obtain multiple similarities between the feature blocks.

[0110] In one embodiment, the first comparison module 430 includes:

[0111] A first extraction submodule (not shown in the figure), configured to determine feature vectors of feature blocks in the feature block groups of the two images through a first feature extraction network;

[0112] The first determination submodule (not shown in the figure) is configured to determine, for any two feature blocks between the feature block groups of the two images, the similarity between the feature vectors of the two feature blocks as the similarity between the two feature blocks.

[0113] In one implementation, the first comparison module 430 is specifically configured as follows:

[0114] For any image, any feature block in the feature block group of the image is compared with all feature blocks in the feature block group of another image.

[0115] In one implementation, the first determining module 440 includes:

[0116] A second determination submodule (not shown in the figure) is configured to determine a first number of feature block pairs with the greatest similarity from the feature block groups of the two images;

[0117] A first comparison submodule (not shown in the figure), configured to perform a detailed comparison on the first number of feature block pairs based on feature elements contained in the feature blocks;

[0118] The third determination submodule (not shown in the figure) is configured to determine the similarity region between the two images based on the fine comparison result.

[0119] In one implementation, the first comparison submodule is specifically configured as follows:

[0120] For any feature block pair, the feature elements contained in the feature block pair are input into the detection network, and the similar area in the feature block pair is determined by the detection network as the fine comparison result of the feature block pair.

[0121] In one embodiment, when the first comparison submodule inputs the feature elements included in the feature block pair into the detection network, it includes:

[0122] Determine, by a second feature extraction network, feature vectors corresponding to the first number of feature block pairs, and determine the similarity between each feature block pair based on the feature vectors;

[0123] Filter feature block pairs whose similarity is greater than a preset similarity threshold;

[0124] The feature elements contained in the filtered feature block pairs are input into the detection network.

[0125] In one embodiment, the neural network includes an FPN network, a CNN network or a Transformer network.

[0126] Figure 5 A schematic block diagram of a blockchain-based image infringement detection device provided in an embodiment. Figure 3 The method embodiment shown corresponds to the embodiment shown. The apparatus is deployed in a computing device, and the computing device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. The apparatus 500 includes:

[0127] A second acquisition module 510 is configured to acquire a first image to be detected;

[0128] A third acquisition module 520 is configured to acquire a plurality of second images in the image library from the blockchain network;

[0129] The second comparison module 530 is configured to compare the first image with the plurality of second images respectively to obtain similar regions between the first image and the plurality of second images respectively; wherein the first image and any one of the second images are used as two images to be compared, by Figure 1 The method provided in the embodiment determines the similarity region between the two images;

[0130] The second determination module 540 is configured to determine whether there is an image infringing upon the rights of the first image in the plurality of second images based on the obtained similar area.

[0131] The above-mentioned device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0132] The present specification also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute Figures 1 to 3 Any of the methods described above.

[0133] The embodiment of the present specification also provides a computing device, including a memory and a processor, wherein the memory stores an executable code, and when the processor executes the executable code, Figures 1 to 3 Any of the methods described above.

[0134] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0135] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0136] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. An image comparison method, comprising: Obtain two images to be compared; For any image, a neural network is used to extract the feature maps of the image in different layers, and multiple feature blocks are intercepted from multiple feature maps to obtain a feature block group corresponding to the image; Performing pairwise comparison on the feature blocks between the feature block groups of the two images; Based on the comparison result, a similar region between the two images is determined.

2. According to the method of claim 1, the step of extracting a plurality of feature blocks from a plurality of feature maps comprises: For different feature maps, a corresponding sliding window is used to intercept multiple feature blocks on the feature map to obtain the feature block corresponding to the feature map; A plurality of feature blocks corresponding to a plurality of feature maps are determined as a feature block group corresponding to the image.

3. According to the method of claim 2, different feature maps correspond to different sliding window sizes.

4. The method according to claim 1, wherein the step of performing pairwise comparison of the feature blocks between the feature block groups of the two images comprises: Based on the feature vectors of the feature blocks, feature blocks between the feature block groups of the two images are compared in pairs to obtain multiple similarities between the feature blocks.

5. The method according to claim 4, wherein the step of performing pairwise comparison of the feature blocks between the feature block groups of the two images comprises: Determining feature vectors of feature blocks in the feature block groups of the two images through a first feature extraction network; For any two feature blocks between the feature block groups of the two images, the similarity between the feature vectors of the two feature blocks is determined as the similarity between the two feature blocks.

6. The method according to claim 4, wherein the step of performing pairwise comparison of the feature blocks between the feature block groups of the two images comprises: For any image, any feature block in the feature block group of the image is compared with all feature blocks in the feature block group of another image.

7. The method according to claim 4, wherein the step of determining the similarity region between the two images based on the comparison result comprises: Determining a first number of feature block pairs with the greatest similarity from the feature block groups of the two images; Based on the characteristic elements contained in the characteristic blocks, performing a detailed comparison on the first number of characteristic block pairs; Based on the refined comparison result, a similar region between the two images is determined.

8. The method according to claim 7, wherein the step of finely comparing the first number of feature block pairs comprises: For any feature block pair, the feature elements contained in the feature block pair are input into the detection network, and the similar area in the feature block pair is determined by the detection network as the fine comparison result of the feature block pair.

9. The method according to claim 8, wherein the step of inputting the feature elements contained in the feature block pair into the detection network comprises: Determine, by a second feature extraction network, feature vectors corresponding to the first number of feature block pairs, and determine the similarity between each feature block pair based on the feature vectors; Filter feature block pairs whose similarity is greater than a preset similarity threshold; The feature elements contained in the filtered feature block pairs are input into the detection network.

10. A blockchain-based image infringement detection method, comprising: Acquire a first image to be detected; Acquire multiple second images in the image library from the blockchain network; Comparing the first image with the plurality of second images respectively to obtain similar regions between the first image and the plurality of second images respectively; wherein the first image and any one of the second images are used as two images to be compared, and the similar regions between the two images are determined by the method of claim 1; Based on the obtained similar area, it is determined whether there is an image infringing upon the rights of the first image in the plurality of second images.

11. An image comparison device, comprising: A first acquisition module is configured to acquire two images to be compared; A first extraction module is configured to extract feature maps of any image in different layers using a neural network, and to intercept multiple feature blocks from multiple feature maps to obtain a feature block group corresponding to the image; A first comparison module is configured to compare feature blocks between feature block groups of the two images in pairs; The first determination module is configured to determine a similar area between the two images based on the comparison result.

12. The device according to claim 11, wherein the first comparison module is specifically configured as follows: Based on the feature vectors of the feature blocks, feature blocks between the feature block groups of the two images are compared in pairs to obtain multiple similarities between the feature blocks.

13. The apparatus according to claim 12, wherein the first determining module comprises: A second determination submodule is configured to determine a first number of feature block pairs with the greatest similarity from the feature block groups of the two images; A first comparison submodule is configured to perform a detailed comparison on the first number of feature block pairs based on feature elements included in the feature blocks; The third determination submodule is configured to determine the similarity region between the two images based on the fine comparison result.

14. A blockchain-based image infringement detection device, comprising: A second acquisition module is configured to acquire a first image to be detected; A third acquisition module is configured to acquire a plurality of second images in the image library from the blockchain network; a second comparison module, configured to compare the first image with the plurality of second images respectively, to obtain similar regions between the first image and the plurality of second images respectively; wherein the first image and any one of the second images are used as two images to be compared, and the similar region between the two images is determined by the method of claim 1; The second determination module is configured to determine whether there is an image infringing upon the rights of the first image in the plurality of second images based on the obtained similar area.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 10.

16. A computing device comprising a memory and a processor, wherein the memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 10 is implemented.

Images