A target image multiplexing detection method, system, electronic device and medium

Abstract

The embodiments of the present application provide a method, system, electronic device and medium for detecting reuse of a target image. In the method, feature processing is performed on a target image based on a preset convolutional neural network to obtain a reuse detection feature of the target image, wherein the reuse detection feature is a fusion feature of low-level features and high-level features of the target image. Subsequently, a feature query is performed in a preset feature vector database based on the reuse detection feature to obtain a reuse comparison feature, and a similarity measurement is performed between the reuse detection feature and the reuse comparison feature to determine a similarity matching value between the reuse detection feature and the reuse comparison feature. Finally, a reuse detection is performed on the target image based on the reuse detection feature and the similarity matching value, so that when the similarity matching value is greater than a preset reuse detection threshold, the target image is determined to be a reused image, thereby doubly improving the reuse detection accuracy of the target image in the two dimensions of feature extraction and feature comparison.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target image multiplexing detection method, system, electronic device and medium. Background Art

[0002] With the rapid growth of image data on the Internet, multiplexing detection of target images is particularly important in order to achieve copyright protection of images. However, in current application scenarios, multiplexing detection of target images is often accompanied by negative effects such as specific environments and noise, making it difficult to identify multiplexing images that match the target image, and the accuracy of multiplexing detection of target images is low.

[0003] Therefore, how to solve the problem of low accuracy of multiplexing detection of target images in related technologies has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0004] Based on the above problems, in order to improve the accuracy of multiplexing detection for target images, the embodiments of the present application provide a target image multiplexing detection method, system, electronic device and medium.

[0005] The embodiments of the present application disclose the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides a target image multiplexing detection method, comprising:

[0007] Performing feature processing on the target image based on a preset convolutional neural network to obtain a reused detection feature of the target image; the reused detection feature is a fusion feature of the low-level feature and the high-level feature of the target image;

[0008] Performing feature query in a preset feature vector database through the reused detection feature to obtain a reused comparison feature;

[0009] Performing similarity measurement on the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature;

[0010] The target image is subjected to a multiplexing detection according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold.

[0011] In a possible implementation, the multiplexed detection features include: global features, local features, and noise vector features; different features in the multiplexed detection features have their own corresponding similarity matching values;

[0012] The performing the multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold, includes:

[0013] When the similarity matching value corresponding to the local feature is greater than the preset multiplexing detection threshold, determining that the target image is a multiplexed image;

[0014] When the similarity matching value corresponding to the local feature is not greater than the preset reuse detection threshold, performing reuse detection on the target image according to the similarity matching values ​​corresponding to the global feature and the noise vector feature respectively;

[0015] For the similarity matching values ​​respectively corresponding to the local features and the noise vector features, if there is at least one similarity matching value that is greater than the preset multiplexing detection threshold, it is determined that the target image is a multiplexed image.

[0016] In a possible implementation manner, the performing similarity measurement on the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature includes:

[0017] Determine a target hash algorithm according to the feature vector dimensions corresponding to the reuse detection feature and the reuse comparison feature, and the feature vector sparsity;

[0018] Determine a first hash value and a second hash value based on the target hash algorithm; the first hash value is determined by the multiplexing detection feature, and the second hash value is determined by the multiplexing comparison feature;

[0019] A Hamming distance between the first Hash value and the second Hash value is calculated, and similarity conversion is performed based on the Hamming distance to obtain the similarity matching value.

[0020] In a possible implementation, before performing the multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold, the method further includes:

[0021] Performing sensitive data discrimination on the target image based on the multiplexed detection feature to obtain a sensitive data discrimination result for the target image;

[0022] If the sensitive data identification result is yes, the preset multiplexing detection threshold is lowered.

[0023] In a possible implementation, the performing feature processing on the target image based on a preset convolutional neural network to obtain the multiplexed detection features of the target image includes:

[0024] Performing feature extraction on the target image according to the preset convolutional neural network to obtain the low-level features and the high-level features of the target image;

[0025] The low-level features and the high-level features are subjected to feature fusion processing based on the attention mechanism to obtain the reused detection features.

[0026] In a possible implementation, extracting features of the target image according to the preset convolutional neural network to obtain the low-level features and the high-level features of the target image includes:

[0027] Extracting the low-level features of the target image through the early layers of the preset convolutional neural network;

[0028] Determining a saliency map of the target image according to the low-level features, and determining a target key area in the target image based on the saliency map;

[0029] The semantic feature abstraction processing is performed through the later layers of the preset convolutional neural network and the target key area to obtain the high-level features of the target image.

[0030] In a possible implementation, the method further includes:

[0031] Obtain multiple types of training sample images;

[0032] Performing data enhancement processing on each of the sample images to obtain multiple types of enhanced sample images;

[0033] The multiple categories of enhanced sample images are used as neural network training data to train the preset convolutional neural network.

[0034] In a second aspect, an embodiment of the present application provides a target image multiplexing detection system, including:

[0035] A feature processing module, used to perform feature processing on a target image based on a preset convolutional neural network to obtain a reused detection feature of the target image; the reused detection feature is a fusion feature of a low-level feature and a high-level feature of the target image;

[0036] A feature query module, used to perform feature query in a preset feature vector database through the multiplexed detection feature to obtain a multiplexed comparison feature;

[0037] A similarity measurement module, used to measure the similarity between the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature;

[0038] The multiplexing detection module is used to perform multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold.

[0039] In a third aspect, an embodiment of the present application provides an electronic device, the device comprising: a processor, a memory, and a system bus;

[0040] The processor and the memory are connected via the system bus;

[0041] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any possible target image multiplexing detection method in the first aspect.

[0042] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any possible target image multiplexing detection method in the first aspect.

[0043] Compared with the prior art, the present application has the following beneficial effects: The embodiments of the present application provide a method, system, electronic device and medium for detecting reuse of a target image. In the method, the feature extraction of the target image is first performed through a preset convolutional neural network to obtain a reuse detection feature of the target image. Among them, the reuse detection feature is a fusion feature of the low-level features and high-level features of the target image. By fusing the low-level features and high-level features of the target image, the multi-scale feature information of the target image can be integrated to determine the reuse detection feature, thereby improving the accuracy of the reuse detection. On the basis of fusing the low-level features and the high-level features to obtain the reuse detection feature, the reuse comparison feature with a certain correlation similarity with the target image is further screened from the preset feature vector database based on the reuse detection feature, thereby ensuring the accuracy of the reuse detection. Furthermore, the similarity between the reuse detection feature and the reuse comparison feature is measured to determine the similarity matching value between the two, and reuse detection is performed based on the reuse detection feature and the similarity matching value. When the similarity matching value between the two is greater than a preset reuse detection threshold, the target image is determined to be a reused image, thereby doubly improving the reuse detection accuracy for the target image in both feature extraction and feature comparison dimensions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0045] Figure 1 A schematic diagram of a flow chart of a target image multiplexing detection method provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a feature extraction process of low-level features and high-level features provided in an embodiment of the present application;

[0047] Figure 3 A schematic diagram of a similarity measurement process provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of a flow chart of another target image multiplexing detection method provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of a target image multiplexing detection system provided in an embodiment of the present application;

[0050] Figure 6 A schematic diagram of the structure of a target image multiplexing detection electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following is a further detailed description of this application in combination with specific embodiments and with reference to the accompanying drawings. It should be noted that the embodiments described in the embodiments of this application are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0052] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "include" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0053] As described above, with the rapid growth of image data on the Internet, in order to achieve copyright protection of images, multiplexing detection of target images is particularly important. However, in current application scenarios, multiplexing detection of target images is often accompanied by negative effects such as specific environments and noise, making it difficult to identify multiplexing images that match the target image, and the accuracy of multiplexing detection of target images is low.

[0054] In order to solve the above problems, the embodiments of the present application provide a method, system, electronic device and medium for detecting reuse of a target image. In the method, first, a feature extraction is performed on the target image through a preset convolutional neural network to obtain a reuse detection feature of the target image. Among them, the reuse detection feature is a fusion feature of the low-level features and high-level features of the target image. By fusing the low-level features and high-level features of the target image, the multi-scale feature information of the target image can be integrated to determine the reuse detection feature, thereby improving the accuracy of reuse detection. On the basis of fusing the low-level features and the high-level features to obtain the reuse detection feature, the reuse comparison feature with a certain correlation similarity with the target image is further screened from the preset feature vector database based on the reuse detection feature, thereby ensuring the accuracy of reuse detection. Furthermore, the similarity between the reuse detection feature and the reuse comparison feature is measured to determine the similarity matching value between the two, and reuse detection is performed based on the reuse detection feature and the similarity matching value. When the similarity matching value between the two is greater than a preset reuse detection threshold, the target image is determined to be a reused image, thereby doubly improving the reuse detection accuracy for the target image in both feature extraction and feature comparison dimensions.

[0055] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] Next, the target image multiplexing detection method provided in the embodiment of the present application will be introduced in conjunction with the specific embodiment drawings.

[0057] See also Figure 1 , which is a flow chart of a target image multiplexing detection method provided in an embodiment of the present application, and specifically includes the following steps:

[0058] S101: Perform feature processing on a target image based on a preset convolutional neural network to obtain a reused detection feature of the target image; the reused detection feature is a fusion feature of low-level features and high-level features of the target image.

[0059] In traditional image reuse detection schemes, shallow visual features of the target image are usually extracted for feature comparison, such as color, texture, and spectrum. The extraction of shallow visual features can extract local invariant features in the target image. Although this method has good robustness, it cannot cope with image reuse detection in a variety of different application scenarios, resulting in low reuse detection accuracy.

[0060] Therefore, in the target image reuse detection method of the embodiment of the present application, it is necessary to first use a preset convolutional neural network to perform feature processing on the target image, and use the deep learning characteristics of the convolutional neural network to extract more abstract high-level semantic information from the target image, and combine it with the low-level features of the target image. The reuse detection features obtained based on the fusion of the two are more accurate, thereby ensuring the accuracy of reuse detection for the target image.

[0061] Among them, the low-level features of the target image are used to characterize the local details and texture information of the image, which helps to identify specific areas or local features in the target image. The high-level features are used to characterize the global semantic information of the image, which helps to identify the overall content and theme of the image. The low-level features have better robustness in the local changes of the target image, while the high-level features have higher robustness in the global changes of the target image. By fusing the two to obtain the reused detection features, the feature extraction accuracy can be improved, thereby improving the reuse detection accuracy of the target image.

[0062] In a possible implementation, when low-level features and high-level features are fused, the low-level features and high-level features can be fused based on an attention mechanism, thereby improving the effect of feature fusion.

[0063] The core of the attention mechanism in the feature fusion process is to continuously and dynamically adjust the weights of the feature areas so that the corresponding preset convolutional neural network can focus on more important information. In the process of fusing low-level features with high-level features through the attention mechanism, an attention map (weight map) is generated, which is used to adjust the weights of each position or channel in the feature map. For example, in some feature image scenarios, the addition of the attention mechanism can make the convolutional neural network pay more attention to the edges or textures of the target image, so that the model can perform feature fusion processing based on specific key areas when extracting, thereby indirectly improving the robustness and accuracy of reuse detection.

[0064] Next, the extraction process of low-level features and high-level features will be introduced in conjunction with the specific embodiment drawings. Figure 2 , which is a schematic diagram of a feature extraction process of low-level features and high-level features provided in an embodiment of the present application, specifically comprising the following steps:

[0065] S201: Extracting the low-level features of the target image through the early layers of the preset convolutional neural network.

[0066] The low-level features of the target image often contain basic information such as the edge, texture, and color of the target image. These information often have a higher resolution because they are closer to the original data of the target image. Therefore, in the process of extracting the low-level features of the target image, they can be obtained by presetting the early layers in the convolutional neural network. The low-level features are extracted through shallow convolution operations and smaller convolution kernels (such as 3X3 convolution). The number of convolution kernels in the early layers is relatively small, so the dimension of the feature map used to extract low-level features is lower, but it has a higher resolution.

[0067] S202: Determine a saliency map of the target image according to the low-level features, and determine a target key area in the target image based on the saliency map.

[0068] The saliency map is a single-channel map of the same size as the original image. Each pixel value represents the saliency of the location in the image. The saliency value is usually between 0 and 1. The closer the value is to 1, the more significant the area is. The saliency map of the target image can characterize the saliency of each image area in the target image. Therefore, through the saliency map of the target image, the areas with higher saliency in the saliency map can be identified as the target key areas, thereby establishing a data foundation for subsequent high-level feature extraction.

[0069] S203: Performing semantic feature abstraction processing through the later layers of the preset convolutional neural network and the target key area to obtain the high-level features of the target image.

[0070] The extraction of target key areas can extract the more significant parts of the target image, so that the semantic information abstraction processing of the model can focus on the key areas in the target image, thereby reducing the impact of irrelevant areas on high-level feature extraction.

[0071] The later layers usually refer to the deep layers of the preset convolutional neural network (such as convolution block 5 of VGG16, the last residual block of ResNet, etc.). These layers are responsible for extracting higher-level and more abstract features. By inputting the target key area into the later layers of the preset convolutional neural network, the semantic features of the target key area can be further extracted to obtain high-level features.

[0072] Among them, the later layers extract features from the key target areas of the input through multiple convolutional layers, pooling layers, activation functions, etc. The convolution operation in the later layers can capture a wider range of contextual information and generate more abstract feature representations, while the pooling operation in the pooling layer can further compress the feature map in the processing process, thereby extracting higher-level features.

[0073] In this way, based on the determined target key areas, through deeper convolution layers, larger convolution kernels and more complex nonlinear activation functions, the global semantic information can be accurately abstracted from the target key areas, thereby obtaining the high-level features of the target image.

[0074] The above is a detailed introduction to the extraction and fusion process of high-level features and low-level features in step S101. Figure 1 , the target image multiplexing detection method provided in the embodiment of the present application is introduced, and the subsequent steps are continued:

[0075] S102: performing feature query in a preset feature vector database using the reused detection feature to obtain a reused comparison feature.

[0076] The data basis for reuse detection of target images is provided by a preset feature vector database, which is a database containing a large number of image feature vectors. Each feature vector corresponds to an image or image area. After determining the reuse detection feature, a specific preset feature query index is assigned to the reuse detection feature, so that the assigned query can quickly find the comparison feature similar to the reuse detection feature in the preset feature vector data to facilitate reuse detection.

[0077] The preset feature query index organizes the reused detection features into a data structure that is easy to query quickly. Common query index structures include hash tables, tree structures (such as KD trees), approximate nearest neighbor searches (such as FLANN, Faiss, etc.). In the process of building a preset feature query index, it is necessary to convert the reused detection features into a fixed-length reused feature vector, and use the selected data index structure to organize the reused feature vector into an index. For example, a hash table can be used to store feature vectors and their corresponding image identifiers, or an approximate nearest neighbor search library (such as Faiss) can be used to build an index.

[0078] In particular, in a possible implementation, the construction of the preset feature query index can also be determined based on the database size of the preset vector database. For example, when the database size is large, an inverted index can be used, and when the database size is small, a flat index can be used, which is not limited in this embodiment.

[0079] S103: Performing similarity measurement on the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature.

[0080] Subsequently, the reused detection features and the reused comparison features are similarly measured to confirm the similarity matching value between the reused detection features and the reused comparison features, so as to judge whether the target image is a reused image according to the similarity matching value between the two.

[0081] For the specific method of measuring the similarity between the reused detection features and the reused comparison features, please refer to Figure 3 A schematic diagram of a similarity measurement process shown in FIG. 1 , which specifically includes the following steps:

[0082] S301: Determine a target hash algorithm based on the multiplexing detection feature and the multiplexing comparison feature.

[0083] The hash algorithm is the process of mapping a feature vector to a fixed-length binary string (hash code). Common hash algorithms include locality-sensitive hashing, perceptual hashing, and difference hashing. In the process of measuring the similarity of reuse detection features and reuse comparison features, the target hash algorithm can be determined based on the feature vector dimensions of the reuse detection features and reuse comparison features, as well as the sparsity of the feature vectors.

[0084] For example, when the dimension of the feature vectors of both are high, an efficient hash algorithm is usually required to process them, such as local sensitive hashing, hyperplane hashing, etc. In view of the sparsity of the feature vectors of both, when there are a large number of zero values ​​in the feature vectors, the local sensitive hashing algorithm that can process sparse data can be selected as the target hash algorithm. When the feature vectors of both are evenly distributed, the perceptual hashing algorithm or the differential hashing algorithm can be used.

[0085] Different target hash algorithms can be effectively selected for different feature vector dimensions and feature vector sparsity. When the dimensions of both are high and have a certain degree of sparsity, the local sensitive hashing algorithm can be used. When both are low-dimensional and non-sparse, the perceptual hashing algorithm can be used.

[0086] S302: Determine a first hash value and a second hash value based on the target hash algorithm; the first hash value is determined by the multiplexing detection feature, and the second hash value is determined by the multiplexing comparison feature.

[0087] After determining the target hash algorithm, the first hash value of the reused detection feature and the second hash value of the reused comparison feature are calculated by the target hash algorithm. Taking the similarity hash algorithm as an example, in the process of calculating the hash value, the feature vectors of the two need to be normalized first so that the weight of each feature is equal. Subsequently, multiple random hyperplanes are generated and the feature vectors are projected onto these hyperplanes to obtain the projection value of each hyperplane. Finally, binary bits are generated according to the positive and negative of the projection value to form a hash value. The specific method is that if the projection value is greater than 0, it is encoded as 1, otherwise it is encoded as 0.

[0088] S303: Calculate the Hamming distance between the first Hash value and the second Hash value, and perform similarity conversion based on the Hamming distance to obtain the similarity matching value.

[0089] The Hamming distance is the number of differences in the characters (bits) at different positions of two equal-length strings (here, binary hash values). The smaller the Hamming distance, the more similar the two feature vectors are. Therefore, by calculating the Hamming distance between the first hash value and the second hash value, the similarity match value between the reuse detection feature and the reuse comparison feature can be determined.

[0090] In this way, by determining the target hash algorithm through the actual reuse detection features and reuse comparison features, the similarity measurement between the two can be completed more accurately, thereby improving the accuracy of image reuse detection.

[0091] Further, S104: performing a multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold.

[0092] In actual application scenarios, since the target image may have been reused multiple times, the reuse detection features in the target image have a certain degree of noise. Therefore, in order to improve the accuracy of reuse detection, when performing image reuse detection, it is necessary to use the noise vector features of the image as a reference factor for reuse detection, that is, to compare the noise values ​​of the corresponding noise vector features between the reuse detection features and the reuse comparison features. The closer the noise values ​​of the two are, the higher the reuse degree of the image.

[0093] Specifically, in the process of multiplexing detection based on multiplexing detection features and similarity matching values, the embodiment of the present application needs to synchronously refer to the global features, local features and noise vector features of the multiplexing detection features, and focuses on improving the accuracy of image multiplexing detection through feature comparison in three aspects. Next, the specific multiplexing detection process will be introduced in conjunction with the drawings of the specific embodiment:

[0094] See also Figure 4 , which is a flow chart of another target image multiplexing detection method provided in an embodiment of the present application, and specifically includes the following steps:

[0095] S401: When the similarity matching value corresponding to the local feature is greater than the preset multiplexing detection threshold, determine that the target image is a multiplexed image.

[0096] In actual application scenarios, the reused detection features include the global features, local features, and noise vector features of the target image. The global features and noise vector features are used to characterize the external environmental factors and noise characteristics of the target image, respectively, while the local features are used to represent areas with more complex or small changes in the target image. For example, when the target image is an ID card image, the external scene image and noise value of the ID card image correspond to the global features and noise vector features, respectively, while the specific portrait and identity description information correspond to the local features.

[0097] Local features often involve key areas in the target image, so in the reuse detection of global features, noise vector features and local features, local features should occupy the highest detection weight. During reuse detection, if the similarity matching value corresponding to the local feature in the reuse detection feature is greater than the preset reuse detection threshold, since the local feature needs to occupy the highest detection weight, it is necessary to directly determine that the target image is a reused image to ensure the accuracy of reuse detection.

[0098] S402: When the similarity matching value corresponding to the local feature is not greater than the preset reuse detection threshold, performing reuse detection on the target image according to the similarity matching values ​​corresponding to the global feature and the noise vector feature respectively;

[0099] S403: For the similarity matching values ​​corresponding to the local features and the noise vector features, if at least one of the similarity matching values ​​is greater than the preset multiplexing detection threshold, determining that the target image is a multiplexing image.

[0100] Correspondingly, when the similarity matching value corresponding to the local feature is not greater than the preset reuse detection threshold, in order to prevent the image from being missed in the reuse detection, it is necessary to synchronously perform reuse detection on the target image based on the similarity matching values ​​corresponding to the global feature and the noise vector feature.

[0101] When the similarity matching value of any feature in the global feature and the noise vector feature is greater than the preset reuse detection threshold, the target image is directly determined to be a reused image to prevent missed reuse detection. In this way, when the reuse detection of local features passes, the target image will be reused again based on the global features and noise vector features of the target image, realizing image reuse detection from three aspects: global characteristics, local characteristics, and noise characteristics of the target image, and improving the accuracy of reuse detection.

[0102] In a possible implementation, the preset reuse detection threshold can be adaptively adjusted for different image reuse detection scenarios. Before performing reuse detection based on the similarity matching value and the preset reuse detection threshold, sensitive data of the target image can also be identified through the reuse detection features of the target image to obtain the corresponding sensitive data discrimination result.

[0103] Taking the ID card image as an example of a target image, for example, the reuse detection features in the ID card image often contain specific identity information. When performing sensitive data identification, through keyword extraction technology and image recognition technology, when the face image and identity information in the ID card image are identified, the sensitive data identification result of the ID card image can be determined to be yes. Accordingly, in order to ensure the privacy security of sensitive data, it is necessary to reduce the missed detection rate as much as possible. Therefore, it is necessary to reduce the preset reuse detection threshold and increase the judgment range of reuse detection, so as to ensure the privacy security of data.

[0104] In another possible implementation, in order to further ensure the accuracy of reuse detection, during the training stage of the above-mentioned preset convolutional neural network, the training sample images of the neural network model can also be subjected to data enhancement processing. The data can be enhanced through data enhancement technology, especially rotation, scaling, cropping and other operations, and high-quality similar photos can be annotated to clarify the reuse data related to each sample image, so as to ensure the comprehensiveness and diversity of the training sample data, thereby improving the accuracy of reuse detection.

[0105] Furthermore, in the process of using the above-mentioned preset convolutional neural network for feature extraction, the learning rate of the model can also be adjusted in real time to balance the convergence speed and the generalization ability of the model. After the feature extraction is completed, the accuracy of the neural network model is obtained according to the proportion of the number of correct samples to the total number of samples, so as to evaluate and optimize the performance of the model, which will not be described in detail in this embodiment.

[0106] The embodiment of the present application provides a method, system, electronic device and medium for detecting reuse of target images. In the method, firstly, a feature extraction is performed on the target image by a preset convolutional neural network to obtain the reuse detection feature of the target image. Among them, the reuse detection feature is a fusion feature of the low-level features and high-level features of the target image. By fusing the low-level features and high-level features of the target image, the feature information of the multi-scale of the target image can be integrated to determine the reuse detection feature, thereby improving the accuracy of the reuse detection. On the basis of fusing the low-level features and the high-level features to obtain the reuse detection feature, the reuse comparison feature with a certain correlation similarity with the target image is further screened from the preset feature vector database based on the reuse detection feature, thereby ensuring the accuracy of the reuse detection. Further, the similarity between the reuse detection feature and the reuse comparison feature is measured, the similarity matching value between the two is determined, and the reuse detection is performed according to the reuse detection feature and the similarity matching value. When the similarity matching value between the two is greater than the preset reuse detection threshold, the target image is determined to be a reused image, thereby doubly improving the reuse detection accuracy of the target image in the two dimensions of feature extraction and feature comparison.

[0107] A target image multiplexing detection system provided in an embodiment of the present application is introduced below. The target image multiplexing detection system described below and the target image multiplexing detection method described above can be referred to each other.

[0108] See also Figure 5 , which is a schematic diagram of the structure of a target image multiplexing detection system provided in an embodiment of the present application, specifically including the following modules:

[0109] The feature processing module 100 is used to perform feature processing on the target image based on a preset convolutional neural network to obtain a reused detection feature of the target image; the reused detection feature is a fusion feature of the low-level feature and the high-level feature of the target image;

[0110] A feature query module 200 is used to perform feature query in a preset feature vector database through the multiplexed detection feature to obtain a multiplexed comparison feature;

[0111] A similarity measurement module 300 is used to measure the similarity between the multiplexed detection feature and the multiplexed comparison feature to determine a similarity matching value between the multiplexed detection feature and the multiplexed comparison feature;

[0112] The multiplexing detection module 400 is used to perform multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold.

[0113] See also Figure 6 , which is a schematic diagram of the structure of a target image multiplexing detection electronic device provided in an embodiment of the present application, including:

[0114] A memory 11, used for storing computer programs;

[0115] The processor 12 is used to implement the steps of a target image multiplexing detection method described in any of the above method embodiments when executing the computer program.

[0116] In this embodiment, the device may be a vehicle-mounted computer, a PC (Personal Computer), or a terminal device such as a smart phone, a tablet computer, a PDA, or a portable computer.

[0117] The device may include a memory 11 , a processor 12 , and a bus 13 .

[0118] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the device, such as a hard disk of the device. In other embodiments, the memory 11 can also be an external storage device of the device, such as a plug-in hard disk equipped on the device, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory 11 can also include both an internal storage unit of the device and an external storage device. The memory 11 can not only be used to store application software installed in the device and various types of data, such as program codes for executing fault prediction methods, etc., but can also be used to temporarily store data that has been output or is to be output. The processor 12 can be a central processing unit (CPU) in some embodiments.

[0119] In some embodiments, the processor 12 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor or other data processing chip, and is used to run program codes or process data stored in the memory 11, such as program codes for executing a target image multiplexing detection method.

[0120] The bus 13 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0121] Furthermore, the device may also include a network interface 14, which may optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the device and other electronic devices.

[0122] Optionally, the device may further include a user interface 15, which may include a display (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 15 may also include a standard wired interface and a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device, etc. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the device and to display a visual user interface.

[0123] Figure 6 Only the device with components 11-15 is shown, and it can be understood by those skilled in the art that Figure 6 The structure shown does not constitute a limitation of the device, and may include fewer or more components than shown, or combine certain components, or arrange the components differently.

[0124] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, an embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the target image multiplexing detection method described in any of the above embodiments.

[0125] The computer-readable media of the embodiments of the present application include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0126] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method for multiplexing detection of target images as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0127] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for methods, systems, electronic devices and media, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The methods, systems, electronic devices and media described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0128] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target image multiplexing detection method, characterized in that: include: Performing feature processing on the target image based on a preset convolutional neural network to obtain a multiplexed detection feature of the target image; The multiplexed detection feature is a fusion feature of the low-level feature and the high-level feature of the target image; Through the preset feature query index corresponding to the multiplexed detection feature, a feature query is performed in a preset feature vector database to obtain a multiplexed comparison feature; Performing similarity measurement on the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature; Performing a multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold; The multiplexed detection features include: global features, local features and noise vector features; different features in the multiplexed detection features have their own corresponding similarity matching values; The performing the multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold, includes: When the similarity matching value corresponding to the local feature is greater than the preset multiplexing detection threshold, determining that the target image is a multiplexed image; When the similarity matching value corresponding to the local feature is not greater than the preset reuse detection threshold, performing reuse detection on the target image according to the similarity matching values ​​corresponding to the global feature and the noise vector feature respectively; For the similarity matching values ​​respectively corresponding to the global feature and the noise vector feature, if there is at least one similarity matching value greater than the preset multiplexing detection threshold, it is determined that the target image is a multiplexed image.

2. The method according to claim 1, characterized in that The performing similarity measurement on the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature includes: Determine a target hash algorithm according to the feature vector dimensions corresponding to the reuse detection feature and the reuse comparison feature, and the feature vector sparsity; Determine a first hash value and a second hash value based on the target hash algorithm; the first hash value is determined by the multiplexing detection feature, and the second hash value is determined by the multiplexing comparison feature; A Hamming distance between the first Hash value and the second Hash value is calculated, and similarity conversion is performed based on the Hamming distance to obtain the similarity matching value.

3. The method according to claim 1, characterized in that Before performing the multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, and determining that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold, the method further includes: Performing sensitive data discrimination on the target image based on the multiplexed detection feature to obtain a sensitive data discrimination result for the target image; If the sensitive data identification result is yes, the preset multiplexing detection threshold is lowered.

4. The method according to claim 1, characterized in that: The performing feature processing on the target image based on a preset convolutional neural network to obtain the multiplexed detection features of the target image includes: Performing feature extraction on the target image according to the preset convolutional neural network to obtain the low-level features and the high-level features of the target image; The low-level features and the high-level features are subjected to feature fusion processing based on the attention mechanism to obtain the reused detection features.

5. The method according to claim 4, characterized in that The step of extracting features of the target image according to the preset convolutional neural network to obtain the low-level features and the high-level features of the target image includes: Extracting the low-level features of the target image through the early layers of the preset convolutional neural network; Determining a saliency map of the target image according to the low-level features, and determining a target key area in the target image based on the saliency map; The semantic feature abstraction processing is performed through the later layers of the preset convolutional neural network and the target key area to obtain the high-level features of the target image.

6. The method according to claim 1, characterized in that The method further comprises: Obtain multiple types of training sample images; Performing data enhancement processing on each of the sample images to obtain multiple types of enhanced sample images; The multiple categories of enhanced sample images are used as neural network training data to train the preset convolutional neural network.

7. A target image multiplexing detection system, characterized in that: include: A feature processing module, used to perform feature processing on a target image based on a preset convolutional neural network to obtain a reused detection feature of the target image; the reused detection feature is a fusion feature of a low-level feature and a high-level feature of the target image; A feature query module, used to perform feature query in a preset feature vector database through a preset feature query index corresponding to the multiplexed detection feature to obtain a multiplexed comparison feature; A similarity measurement module, used to measure the similarity between the multiplexing detection feature and the multiplexing comparison feature to determine a similarity matching value between the multiplexing detection feature and the multiplexing comparison feature; A multiplexing detection module, configured to perform multiplexing detection on the target image according to the multiplexing detection feature and the similarity matching value, so as to determine that the target image is a multiplexed image when the similarity matching value is greater than a preset multiplexing detection threshold; The multiplexed detection features include: global features, local features and noise vector features; Different features in the multiplexed detection features have respective corresponding similarity matching values; The multiplexing detection module is specifically used for: When the similarity matching value corresponding to the local feature is greater than the preset multiplexing detection threshold, determining that the target image is a multiplexed image; When the similarity matching value corresponding to the local feature is not greater than the preset reuse detection threshold, performing reuse detection on the target image according to the similarity matching values ​​corresponding to the global feature and the noise vector feature respectively; For the similarity matching values ​​respectively corresponding to the global feature and the noise vector feature, if there is at least one similarity matching value greater than the preset multiplexing detection threshold, it is determined that the target image is a multiplexed image.

8. An electronic device, characterized in that: The device comprises: a processor, a memory and a system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the target image multiplexing detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the target image multiplexing detection method described in any one of claims 1 to 6 is implemented.

Images