Image processing method and apparatus

Abstract

Embodiments of the present specification provide image processing methods and devices, wherein an image processing method comprises: performing feature extraction on a to-be-detected image and a reconstructed image of the to-be-detected image, obtaining image features of the to-be-detected image and reconstructed image features of the reconstructed image, performing feature conversion processing on the image features by means of the image features to obtain target features, performing feature fusion on the image features, the reconstructed image features and the target features to obtain first fusion features, constructing a feature relationship network based on the image features of the to-be-detected image, performing feature fusion on the first fusion features and pixel relationship features determined by means of the feature relationship network, and determining an image detection result of the to-be-detected image according to second fusion features obtained through the feature fusion.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of data processing, and in particular to an image processing method and device. BACKGROUND

[0002] With the continuous development of Internet technology, more and more online services have emerged. Users no longer need to go to offline service institutions for service processing, but directly process through online services. In the process of using online services, in order to improve the security of online services, more and more online services need users to provide corresponding user images, and the users are verified through the user images uploaded by the users, such as identity credential images, driving credential images, etc. In this process, how to better manage the user images uploaded by the users has gradually become the focus of all parties. SUMMARY

[0003] One or more embodiments of the present specification provide an image processing method, comprising: obtaining a to-be-detected image and a reconstructed image of the to-be-detected image. Feature extraction is performed on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image. Feature conversion processing is performed on the image features to obtain target features, and the image features, the reconstructed image features, and the target features are fused to obtain first fusion features. A feature relationship network is constructed based on the image features, and the first fusion features and pixel relationship features determined based on the feature relationship network are fused to obtain second fusion features. An image detection result of the to-be-detected image is determined according to the second fusion features.

[0004] One or more embodiments of the present specification provide an image processing device, comprising: an image acquisition module configured to obtain a to-be-detected image and a reconstructed image of the to-be-detected image. A feature extraction module configured to perform feature extraction on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image. A first feature fusion module configured to perform feature conversion processing on the image features to obtain target features, and fuse the image features, the reconstructed image features, and the target features to obtain first fusion features. A second feature fusion module configured to construct a feature relationship network based on the image features, and fuse the first fusion features and pixel relationship features determined based on the feature relationship network to obtain second fusion features. A result determination module configured to determine an image detection result of the to-be-detected image according to the second fusion features.

[0005] The one or more embodiments of the specification provide an image processing device, comprising: a processor; and a memory configured to store computer executable instructions which, when executed, cause the processor to: acquire a to-be-detected image and a reconstructed image of the to-be-detected image. Feature extraction is performed on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image. Feature conversion processing is performed on the image features to obtain target features, and the image features, the reconstructed image features and the target features are fused to obtain first fused features. A feature relationship network is constructed based on the image features, and the first fused features are fused with pixel relationship features determined based on the feature relationship network to obtain second fused features. An image detection result of the to-be-detected image is determined according to the second fused features.

[0006] The one or more embodiments of the specification provide a storage medium for storing computer executable instructions which, when executed by a processor, implement the following processes: acquiring a to-be-detected image and a reconstructed image of the to-be-detected image. Feature extraction is performed on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image. Feature conversion processing is performed on the image features to obtain target features, and the image features, the reconstructed image features and the target features are fused to obtain first fused features. A feature relationship network is constructed based on the image features, and the first fused features are fused with pixel relationship features determined based on the feature relationship network to obtain second fused features. An image detection result of the to-be-detected image is determined according to the second fused features. BRIEF DESCRIPTION OF DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the one or more embodiments of the specification or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor;

[0008] Figure 1 A schematic diagram of an image processing method implementation environment provided by the one or more embodiments of the specification;

[0009] Figure 2 A processing flowchart of an image processing method provided by the one or more embodiments of the specification;

[0010] Figure 3 A schematic diagram of a feature relationship network provided by the one or more embodiments of the specification;

[0011] Figure 4 A flow chart of an image processing method applied to an identity credential image scene according to an embodiment of the present specification;

[0012] Figure 5 A schematic diagram of an image processing device according to an embodiment of the present specification;

[0013] Figure 6 A structural schematic diagram of an image processing device according to an embodiment of the present specification. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solutions in the one or more embodiments of the present specification, the technical solutions in the one or more embodiments of the present specification will be described clearly and completely below in conjunction with the drawings in the one or more embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, rather than all the embodiments. Based on the one or more embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present document.

[0015] Referring to Figure 1 , a schematic diagram of an image processing method implementation environment according to an embodiment of the present specification.

[0016] The image processing method provided by the one or more embodiments of the present specification can be applied to an implementation environment of image detection on a to-be-detected image, and the implementation environment at least includes a detection server 101 of an image detection platform.

[0017] In addition, the implementation environment can further include a user terminal 102, which can be configured as a client for image acquisition. The specific form of the client can be an application program, a subprogram in the application program, a service module in the application program, or a web program.

[0018] The detection server 101 can correspond to one server, or a server cluster composed of a plurality of servers, or one or more cloud servers in a cloud computing platform, and is used for image detection on a to-be-detected image to obtain an image detection result.

[0019] The user terminal 102 can be a mobile phone, a personal computer, a tablet computer, an electronic book reader, a device for information interaction based on VR (Virtual Reality), a vehicle-mounted terminal, an IoT device, a wearable smart device, a laptop computer, and a desktop computer, etc. The user terminal 102 is configured to upload a to-be-detected image to the detection server 101.

[0020] In this implementation environment, after obtaining the to-be-detected image and the reconstructed image of the to-be-detected image, the detection server 101 can perform feature extraction on the to-be-detected image and the reconstructed image respectively, obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image, perform feature fusion on the image features, the reconstructed image features, and target features obtained by performing feature conversion on the image features, obtain first fusion features, construct a feature relationship network based on the image features of the to-be-detected image, perform feature fusion on the first fusion features and pixel relationship features determined based on the feature relationship network, obtain second fusion features, and determine an image detection result of the to-be-detected image based on the second fusion features. In this way, the image detection is performed from two aspects of the to-be-detected image and the reconstructed image, the comprehensiveness and accuracy of image detection are improved, and the fine-grained detection of the image is realized from the feature dimension.

[0021] One or more implementations of the image processing method provided in the specification are as follows:

[0022] Referring to Figure 2 The image processing method provided in this embodiment specifically includes steps S202 to S210.

[0023] In step S202, a to-be-detected image and a reconstructed image of the to-be-detected image are obtained.

[0024] The to-be-detected image in this embodiment refers to an image to be detected; the to-be-detected image includes a to-be-detected image uploaded by a user; optionally, the to-be-detected image includes a to-be-detected identity image, such as an identity certificate image, a bank card image, a driving certificate image, a medical insurance certificate image, etc. The to-be-detected image can be an RGB (Red Green Blue) image, and the to-be-detected image can also include a to-be-detected card image. The to-be-detected card image specifically represents a card image and / or a certificate image of a user.

[0025] In an optional implementation of the image processing method provided in this embodiment, the reconstructed image is obtained in the following manner:

[0026] Performing frequency domain transformation on the to-be-detected image to obtain frequency domain information;

[0027] extract target frequency domain information of a preset frequency channel from the frequency domain information, and perform frequency domain inverse transformation on the target frequency domain information to obtain the reconstructed image.

[0028] The frequency domain transformation includes converting the to-be-detected image from a spatial domain (a time domain) to a frequency domain, such as a discrete cosine transform (DCT). The frequency domain information refers to relevant frequency domain information generated after the to-be-detected image is subjected to frequency domain transformation. The frequency domain inverse transformation includes converting the target frequency domain information from the frequency domain to the spatial domain (the time domain).

[0029] In the process of obtaining the to-be-detected image and the reconstructed image of the to-be-detected image, the following operations can be performed: obtaining a to-be-detected image uploaded by a user, performing frequency domain transformation on the to-be-detected image to obtain frequency domain information, extracting target frequency domain information of a preset frequency channel from the frequency domain information, and performing frequency domain inverse transformation on the target frequency domain information to obtain the reconstructed image.

[0030] Step S204: performing feature extraction on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image.

[0031] In the process of obtaining the to-be-detected image and the reconstructed image of the to-be-detected image, the following operations can be performed: obtaining a to-be-detected image uploaded by a user, performing frequency domain transformation on the to-be-detected image to obtain frequency domain information, extracting target frequency domain information of a preset frequency channel from the frequency domain information, and performing frequency domain inverse transformation on the target frequency domain information to obtain the reconstructed image.

[0032] The image features can be multiple image features. The image features include multiple image feature maps. The reconstructed image features can be multiple reconstructed image features. The reconstructed image features include multiple reconstructed image feature maps.

[0033] In the process of obtaining the to-be-detected image and the reconstructed image of the to-be-detected image, the following operations can be performed: obtaining a to-be-detected image uploaded by a user, performing frequency domain transformation on the to-be-detected image to obtain frequency domain information, extracting target frequency domain information of a preset frequency channel from the frequency domain information, and performing frequency domain inverse transformation on the target frequency domain information to obtain the reconstructed image.

[0034] In the process of obtaining the to-be-detected image and the reconstructed image of the to-be-detected image, the following operations can be performed: obtaining a to-be-detected image uploaded by a user, performing frequency domain transformation on the to-be-detected image to obtain frequency domain information, extracting target frequency domain information of a preset frequency channel from the frequency domain information, and performing frequency domain inverse transformation on the target frequency domain information to obtain the reconstructed image.

[0035] inputting the image to be detected into a first feature extraction model to perform feature extraction, and obtaining a plurality of image features; and inputting the reconstructed image into a second feature extraction model to perform feature extraction, and obtaining a plurality of reconstructed image features;

[0036] Optionally, the number of features of the plurality of image features is consistent with the number of features of the plurality of reconstructed image features.

[0037] The first feature extraction model and the second feature extraction model can adopt feature extraction models with the same structure, such as backbone networks with the same structure; and the model parameters of the first feature extraction model and the second feature extraction model can be different, that is, the model parameters of the first feature extraction model and the second feature extraction model are not shared.

[0038] The plurality of image features includes a plurality of image feature maps, and the feature map parameters of the plurality of image feature maps can be different, such as different scales or feature dimensions of the plurality of image feature maps; the plurality of reconstructed image features includes a plurality of reconstructed image feature maps, and the feature map parameters of the plurality of reconstructed image feature maps can also be different, such as different scales or feature dimensions of the plurality of reconstructed image feature maps. Optionally, the feature dimensions of each image feature in the plurality of image features have a corresponding relationship with the feature dimensions of each reconstructed image feature in the plurality of reconstructed image features; for example, the plurality of image feature maps include an image feature map with a1 dimension and an image feature map with a2 dimension, and the plurality of reconstructed image feature maps also include a reconstructed image feature map with a1 dimension and a reconstructed image feature map with a2 dimension.

[0039] In addition, the number of features of the plurality of image features can also be different from the number of features of the plurality of reconstructed image features, and the feature dimensions of each image feature in the plurality of image features can also not have a corresponding relationship with the feature dimensions of each reconstructed image feature in the plurality of reconstructed image features.

[0040] In step S206, the image features are subjected to feature conversion processing to obtain target features, and the image features, the reconstructed image features, and the target features are subjected to feature fusion to obtain first fusion features.

[0041] The above describes that the image to be detected and the reconstructed image are subjected to feature extraction to obtain image features of the image to be detected and reconstructed image features of the reconstructed image. In this step, the image features of the image to be detected are subjected to feature conversion processing, and the image features of the image to be detected, the reconstructed image features of the reconstructed image, and the target features obtained through the feature conversion processing are subjected to feature fusion to obtain first fusion features.

[0042] In the implementation, on the basis of inputting the to-be-detected model into the first feature extraction model to perform feature extraction and obtain a plurality of image features, and inputting the reconstructed image of the to-be-detected image into the second feature extraction model to perform feature extraction and obtain a plurality of reconstructed image features, in an optional implementation of the embodiment, in the process of performing feature conversion processing on the image features to obtain target features, the following operation is performed:

[0043] a corresponding feature matrix is calculated based on each image feature in the plurality of image features, and a correlation degree of each image feature is calculated according to a first feature matrix in the feature matrix;

[0044] an output feature matrix of each image feature is calculated as the target feature based on the correlation degree and a second feature matrix in the feature matrix.

[0045] The feature matrix corresponding to each image feature includes a Q feature matrix, a K feature matrix, and a V feature matrix corresponding to each image feature. The first feature matrix in the feature matrix includes the Q feature matrix and the K feature matrix in the feature matrix. The second feature matrix in the feature matrix includes the V feature matrix in the feature matrix. The form of the correlation degree can also be the form of the feature matrix, that is, it can be a correlation degree matrix. The correlation degree can represent the correlation degree between each two image features or the correlation between each two image features.

[0046] Specifically, a Q feature matrix, a K feature matrix, and a V feature matrix corresponding to each image feature in the plurality of image features can be calculated, a correlation degree of each image feature can be calculated according to the Q feature matrix and the K feature matrix, and an output feature matrix of each image feature can be calculated as the target feature based on the correlation degree and the V feature matrix.

[0047] In addition, in the process of performing feature conversion processing on the image features of the to-be-detected image to obtain target features, the following operation can also be performed: self-attention features are obtained as the target features by performing self-attention calculation based on the image features of the to-be-detected image. That is, self-attention can be introduced, and self-attention features of each image feature in the plurality of image features can be obtained by performing self-attention calculation based on the plurality of image features of the to-be-detected image.

[0048] In the process of calculating the output feature matrix of each image feature based on the correlation degree and the second feature matrix in the feature matrix, in order to improve the calculation convenience of the output feature matrix, in an optional implementation of the embodiment, the following operation is performed:

[0049] The correlation degree is normalized to obtain a normalized correlation degree.

[0050] According to the normalized correlation degree and the second feature matrix, an output feature matrix of the image features is calculated.

[0051] In a specific implementation process, in order to improve the accuracy and comprehensiveness of image detection, feature fusion can be used to achieve this. In an optional implementation provided by the embodiment, in the process of performing feature fusion on the image features, the reconstructed image features and the target features to obtain first fusion features, the following operations are performed:

[0052] According to the feature parameters, feature fusion is performed on each of the image features, each of the reconstructed image features and each of the target features to obtain a plurality of first fusion features.

[0053] The plurality of image features include a plurality of image feature maps; the plurality of reconstructed image features include a plurality of reconstructed image feature maps; and the feature parameters include a feature scale of the image feature map or a feature dimension of the image feature map.

[0054] Specifically, the image features, the reconstructed image features and the target features of the same feature dimension in the plurality of image features, the plurality of reconstructed image features and the plurality of target features are fused to obtain a plurality of first fusion features.

[0055] For example, the plurality of image features include image features of a1 dimension and image features of a2 dimension, the plurality of reconstructed image features include reconstructed image features of a1 dimension and reconstructed image features of a2 dimension, and the plurality of target features include target features of a1 dimension and target features of a2 dimension. The image features, the reconstructed image features and the target features of a1 dimension are fused to obtain first fusion features corresponding to a1 dimension (first dimension), and the image features, the reconstructed image features and the target features of a2 dimension are fused to obtain first fusion features corresponding to a2 dimension (second dimension).

[0056] In addition, in the process of performing feature fusion on the image features, the reconstructed image features and the target features to obtain first fusion features, the following operations can also be performed: cross-attention calculation is performed based on the image features of the to-be-detected image, the reconstructed image features of the reconstructed image and the target features to obtain cross-attention features as the first fusion features. The cross-attention can be Cross-Attention.

[0057] In addition, in the process of feature fusion of the image features, the reconstructed image features and the target features, the following operation can also be performed: feature fusion of image features, reconstructed image features and target features of different feature dimensions in the plurality of image features, the plurality of reconstructed image features and the plurality of target features to obtain a plurality of first fusion features; for example, feature fusion of a1-dimensional image features, a2-dimensional reconstructed image features and a3-dimensional target features to obtain a first fusion feature corresponding to a first dimension, feature fusion of a2-dimensional image features, a3-dimensional reconstructed image features and a1-dimensional target features to obtain a first fusion feature corresponding to a second dimension, and feature fusion of a3-dimensional image features, a1-dimensional reconstructed image features and a2-dimensional target features to obtain a first fusion feature corresponding to a third dimension.

[0058] It should be noted that, in the process of feature fusion of the image features, the reconstructed image features and the target features, the following operation can also be performed: feature fusion of image features, reconstructed image features and target features of different feature dimensions in the plurality of image features, the plurality of reconstructed image features and the plurality of target features to obtain a plurality of first fusion features.

[0059] Step S208, constructing a feature relationship network based on the image features, and performing feature fusion of the first fusion features and pixel relationship features determined based on the feature relationship network to obtain second fusion features.

[0060] In the above process of feature conversion of the image features of the to-be-detected image to obtain the target features, and feature fusion of the image features of the to-be-detected image, the reconstructed image features of the reconstructed image and the target features to obtain the first fusion features, in order to further improve the comprehensiveness of image detection, a feature relationship network is constructed based on the image features of the to-be-detected image, and the first fusion features are fused with pixel relationship features determined by means of the feature relationship network to obtain second fusion features.

[0061] The feature relationship network described in the embodiment refers to a network representing the correlation relationship between each pixel point in the image features of the to-be-detected image; for example, the feature relationship network shown in Figure 2 The pixel point A, the pixel point B and the pixel point C represent three pixel nodes respectively, 5 represents that the Euclidean distance between the pixel point A and the pixel point B is 5, 4 represents that the Euclidean distance between the pixel point A and the pixel point C is 4, and 2 represents that the Euclidean distance between the pixel point B and the pixel point C is 2.

[0062] The pixel relationship feature includes a feature matrix or a feature vector representing the connection relationship between the pixels, such as an adjacency matrix.

[0063] In a specific implementation, a feature relationship network can be constructed based on the image features of the image to be detected, and the first fusion features are fused with the pixel relationship features determined based on the feature relationship network to obtain second fusion features. The feature fusion includes feature splicing.

[0064] In actual applications, the feature dimensions of the multiple image features of the image to be detected can be different or the same. In the case where the feature dimensions of the multiple image features are different, the feature relationship network cannot be constructed. To this end, in order to realize the construction of the feature relationship network and improve the flexibility and convenience of the construction of the feature relationship network, in an optional implementation provided in this embodiment, the feature relationship network is constructed based on the image features in the following manner:

[0065] The feature parameters of the multiple image features are updated, and the pixel nodes are determined according to the pixels included in the updated multiple image features.

[0066] The connections of the pixel nodes are established according to the position distances of the pixel nodes, and the feature relationship network is obtained.

[0067] The feature parameters of the multiple image features are updated, and the pixel nodes are determined according to the pixels included in the updated multiple image features. The position distances of the pixel nodes include the position distances of the pixels in the updated multiple image features, and specifically can be the Euclidean distances of the pixels in the updated multiple image features. The downsampling includes downsampling the multiple image features to a target feature dimension, that is, downsampling each image feature in the multiple image features to the target feature dimension.

[0068] Specifically, the multiple image features can be downsampled to a target feature dimension, and the connections of the pixel nodes are established based on the Euclidean distances of the pixels in the downsampled multiple image features, to obtain the feature relationship network.

[0069] In an optional implementation provided in this embodiment, the pixel relationship feature is obtained in the following manner: an adjacency matrix is constructed based on the feature relationship network, and the adjacency matrix is used as the pixel relationship feature.

[0070] Specifically, in the process of constructing the adjacency matrix based on the feature relationship network, a first matrix can be constructed based on the connection state of each pixel node in the feature relationship network, and a second matrix composed of weighted feature vectors can be obtained by performing weighted calculation on the feature vectors in the first matrix based on the position distance of each pixel node, so as to obtain the pixel relationship feature.

[0071] In the process of constructing the first matrix based on the connection state of each pixel node in the feature relationship network, the connection value of each two pixel nodes can be determined as a first value when the connection state of the two pixel nodes is not connected, and the connection value of each two pixel nodes can be determined as a second value when the connection state of the two pixel nodes is connected, and the first matrix can be constructed based on the first value and the second value. In the process of performing weighted calculation on the feature vectors in the first matrix based on the position distance of each pixel node, the product of the connection value of each two pixel nodes and the position distance of the two pixel nodes can be taken as a target value of each two pixel nodes, and the second matrix can be constructed based on the target value.

[0072] In step S210, the image detection result of the to-be-detected image is determined according to the second fusion feature.

[0073] The feature relationship network is constructed based on the image feature of the to-be-detected image, the first fusion feature is fused with the pixel relationship feature determined based on the feature relationship network to obtain the second fusion feature, and in this step, the image detection result of the to-be-detected image is determined by means of the second fusion feature.

[0074] The image detection result of the to-be-detected image in the embodiment includes a detection pass result and / or a detection fail result.

[0075] In specific implementation, in order to improve the detection convenience of detecting the to-be-detected image, the to-be-detected image can be subjected to pixel processing, and the image detection result is determined based on the pixel processing result. In an optional implementation manner provided in the embodiment, in the process of determining the image detection result of the to-be-detected image according to the second fusion feature, the following operation is performed:

[0076] The to-be-detected image is subjected to semantic segmentation processing according to the second fusion feature.

[0077] The image detection result of the to-be-detected image is determined based on the semantic segmentation processing result.

[0078] In the process of performing semantic segmentation processing on the to-be-detected image according to the second fusion feature, in an optional implementation manner provided in the embodiment, the following operation is performed:

[0079] determine an abnormal image block and a default image block in the image to be detected according to the second fusion feature;

[0080] perform binaryzation processing on pixel values of pixel points included in the abnormal image block and pixel values of pixel points included in the default image block, and obtain a mask image as the semantic segmentation processing result.

[0081] Optionally, the pixel value of the pixel point included in the abnormal image block is determined as a first pixel value after the binaryzation processing, and the pixel value of the pixel point included in the default image block is determined as a second pixel value after the binaryzation processing.

[0082] The abnormal image block includes a tampered image block, specifically, an image block that is tampered in the image to be detected, and the default image block includes the remaining image blocks other than the abnormal image block in the image to be detected. The binaryzation processing includes adjusting the pixel values of the pixel points included in the abnormal image block and the default image block.

[0083] For example, the abnormal image block and the default image block in the image to be detected are determined according to the second fusion feature, the pixel value of the pixel point included in the abnormal image block is adjusted to a first pixel value, and the pixel value of the pixel point included in the default image block is adjusted to a second pixel value. Optionally, the first pixel value is different from the second pixel value, and the values of the first pixel value and the second pixel value are not limited herein in the case that they are different.

[0084] On the basis of taking the mask image as the semantic segmentation processing result, it is determined that the image detection result of the image to be detected is detection failure, that is, the image to be detected exists tampering.

[0085] In addition, in the process of determining the image detection result of the image to be detected based on the semantic segmentation processing result, the following operations can also be performed: if the semantic segmentation processing result includes a first image block and a second image block, it is determined that the image detection result of the image to be detected is detection pass; if the semantic segmentation processing result includes the first image block, it is determined that the image detection result of the image to be detected is detection failure. The first image block includes the default image block, and the second image block includes the tampered image block, that is, the image block that exists tampering in the image to be detected, for example, the user's portrait in the user's identity credential image is tampered, and the user's portrait image block in the identity credential image is the tampered image block.

[0086] In another optional implementation provided by the embodiment, in the process of determining the image detection result of the image to be detected according to the second fusion feature, the following operations are performed:

[0087] The image detection result of the to-be-detected image is determined according to the second fusion feature, so as to perform identity verification on the user based on the image detection result.

[0088] The image detection result of the to-be-detected image is determined according to the second fusion feature, so as to perform identity verification on the user based on the image detection result.

[0089] In the process of performing identity verification on the user based on the image detection result, the following operation can be performed: if the image detection result is a detection pass, it is determined that the verification result of performing identity verification on the user is a verification pass; if the image detection result is a detection fail, it is determined that the verification result of performing identity verification on the user is a verification fail.

[0090] In addition, in the process of determining the image detection result of the to-be-detected image according to the second fusion feature, the following operation can also be performed: the second fusion feature is decoded and processed, and it is detected whether an abnormal image block is included in the target image obtained by the decoding processing, if yes, it is determined that the image detection result of the to-be-detected image is a detection fail; if no, it is determined that the image detection result of the to-be-detected image is a detection pass; wherein the abnormal image block includes a tampered image block.

[0091] It should be noted that the two processes of feature extraction on the to-be-detected image and feature extraction on the reconstructed image in step S204 in the embodiment have no fixed execution order, that is, the image feature of the to-be-detected image can be extracted first, and then the reconstructed image feature can be extracted; or the reconstructed image feature can be extracted first, and then the image feature of the to-be-detected image can be extracted; or the reconstructed image feature can be extracted during the process of extracting the image feature of the to-be-detected image.

[0092] In addition, the feature conversion processing on the image feature to obtain the target feature in step S206 has no fixed execution order with the feature extraction on the to-be-detected image and the reconstructed image in step S204, that is, part of the operations in step S206 can be executed first, and then step S204 can be executed; or part of the operations in step S206 can be executed at the same time as step S204.

[0093] In summary, the image processing method provided by the embodiment first acquires a to-be-detected image and a reconstructed image of the to-be-detected image, inputs the to-be-detected image into a first feature extraction model for feature extraction to obtain a plurality of image features, and inputs the reconstructed image into a second feature extraction model for feature extraction to obtain a plurality of reconstructed image features;

[0094] Secondly, a corresponding feature matrix is calculated based on each image feature in the plurality of image features, and a correlation degree of each image feature is calculated according to a first feature matrix in the feature matrix, and an output feature matrix of each image feature is calculated as a target feature based on the correlation degree and a second feature matrix in the feature matrix; each image feature in the plurality of image features, each reconstructed image feature in the plurality of reconstructed image features and each target feature in the plurality of target features are fused according to feature dimensions to obtain a plurality of first fused features;

[0095] Finally, the plurality of image features are down-sampled according to the target feature dimensions, each pixel node is determined according to each pixel point contained in the plurality of down-sampled image features, and the connection of each pixel node is established according to the position distance of each pixel node to obtain a feature relationship network; the first fused features are fused with pixel relationship features determined based on the feature relationship network to obtain second fused features; and an image detection result of the to-be-detected image is determined according to the second fused features.

[0096] In this way, the image detection is performed from two aspects of the to-be-detected image and the reconstructed image, which improves the comprehensiveness and accuracy of image detection, and the image detection is performed from the feature dimensions to realize fine-grained image detection.

[0097] The image processing method provided in the embodiment is further described below by taking the application of the image processing method in an identity credential image scene as an example, referring to Figure 4 The image processing method applied in the identity credential image scene specifically includes the following steps.

[0098] In step S402, a to-be-detected identity credential image is obtained, frequency domain transformation is performed on the identity credential image to obtain frequency domain information, target frequency domain information of a preset frequency channel is extracted from the frequency domain information, and inverse frequency domain transformation is performed on the target frequency domain information to obtain a reconstructed image.

[0099] In step S404, the identity credential image is input into a first feature extraction model to perform feature extraction and obtain a plurality of image feature maps, and the reconstructed image is input into a second feature extraction model to perform feature extraction and obtain a plurality of reconstructed image feature maps.

[0100] In step S406, a corresponding feature matrix is calculated based on each image feature map in the plurality of image feature maps, and a correlation degree of each image feature map is calculated according to a first feature matrix in the feature matrix.

[0101] In step S408, the correlation degree is normalized to obtain a normalized correlation degree.

[0102] In step S410, an output feature matrix of each image feature map is calculated as a target feature according to the normalized correlation degree and a second feature matrix in the feature matrix.

[0103] In step S412, feature fusion is performed on each image feature map in the plurality of image feature maps, each reconstructed image feature map in the plurality of reconstructed image feature maps, and each target feature in the plurality of target features according to the feature dimension, to obtain a plurality of first fused features.

[0104] In step S414, the plurality of image feature maps are down-sampled according to the target feature dimension, each pixel node is determined according to each pixel point contained in the plurality of down-sampled image feature maps, and the connection of each pixel node is established according to the position distance of each pixel node, to obtain a feature relationship network.

[0105] In step S416, the plurality of first fused features are fused with the pixel relationship feature determined based on the feature relationship network, to obtain a second fused feature.

[0106] In step S418, the identity credential image is subjected to semantic segmentation processing according to the second fused feature, and an image detection result of the to-be-detected image is determined based on the semantic segmentation processing result, to perform identity verification on the user based on the image detection result.

[0107] The image processing apparatus provided in the present specification implements, for example, the following:

[0108] In the above embodiment, an image processing method is provided, and a corresponding image processing apparatus is also provided, which will be described below with reference to the accompanying drawings.

[0109] Reference Figure 5 which shows a schematic diagram of an embodiment of an image processing apparatus provided in the present embodiment.

[0110] Since the apparatus embodiment corresponds to the method embodiment, the description is relatively simple, and the related parts can be referred to the corresponding description of the method embodiment provided above. The apparatus embodiment described below is only illustrative.

[0111] The present embodiment provides an image processing apparatus, comprising:

[0112] The image acquisition module 502 is configured to acquire a to-be-detected image and a reconstructed image of the to-be-detected image.

[0113] The feature extraction module 504 is configured to perform feature extraction on the to-be-detected image and the reconstructed image, to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image.

[0114] The first feature fusion module 506 is configured to perform feature transformation processing on the image features to obtain target features, and to fuse the image features, the reconstructed image features and the target features to obtain a first fused feature;

[0115] The second feature fusion module 508 is configured to construct a feature relationship network based on the image features, and to fuse the first fused feature with the pixel relationship features determined based on the feature relationship network to obtain the second fused feature;

[0116] The result determination module 510 is configured to determine the image detection result of the image to be detected based on the second fusion feature.

[0117] This specification provides an embodiment of an image processing device as follows:

[0118] Corresponding to the image processing method described above, based on the same technical concept, one or more embodiments of this specification also provide an image processing apparatus for performing the image processing method provided above. Figure 6 This is a schematic diagram of the structure of an image processing device provided for one or more embodiments of this specification.

[0119] This embodiment provides an image processing device, including:

[0120] like Figure 6 As shown, image processing devices can vary considerably due to differences in configuration or performance. They may include one or more processors 601 and memory 602, with memory 602 storing one or more application programs or data. Memory 602 can be temporary or persistent storage. The application programs stored in memory 602 may include one or more modules (not shown), each module including a series of computer-executable instructions from the image processing device. Furthermore, processor 601 may be configured to communicate with memory 602, executing the series of computer-executable instructions stored in memory 602 on the image processing device. The image processing device may also include one or more power supplies 603, one or more wired or wireless network interfaces 604, one or more input / output interfaces 605, one or more keyboards 606, etc.

[0121] In one specific embodiment, the image processing device includes a memory and one or more programs, wherein one or more programs are stored in the memory, and one or more programs can include one or more modules, and each module can include a series of computer executable instructions in the image processing device, and the one or more programs executed by one or more processors include computer executable instructions for:

[0122] obtaining a to-be-detected image and a reconstructed image of the to-be-detected image;

[0123] performing feature extraction on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image;

[0124] performing feature conversion processing on the image features to obtain target features, and performing feature fusion on the image features, the reconstructed image features, and the target features to obtain first fusion features;

[0125] constructing a feature relationship network based on the image features, and performing feature fusion on the first fusion features and pixel relationship features determined based on the feature relationship network to obtain second fusion features;

[0126] determining an image detection result of the to-be-detected image according to the second fusion features.

[0127] The storage medium provided in the specification is implemented as follows:

[0128] According to the above description, based on the same technical concept, one or more embodiments of the specification also provide a storage medium.

[0129] The storage medium provided in the embodiment is used to store computer executable instructions, and the computer executable instructions are executed by a processor to implement the following processes:

[0130] obtaining a to-be-detected image and a reconstructed image of the to-be-detected image;

[0131] performing feature extraction on the to-be-detected image and the reconstructed image to obtain image features of the to-be-detected image and reconstructed image features of the reconstructed image;

[0132] performing feature conversion processing on the image features to obtain target features, and performing feature fusion on the image features, the reconstructed image features, and the target features to obtain first fusion features;

[0133] construct a feature relationship network based on the image features, and perform feature fusion on the first fusion feature and pixel relationship features determined based on the feature relationship network to obtain a second fusion feature;

[0134] determine an image detection result of the to-be-detected image according to the second fusion feature.

[0135] It should be noted that the embodiment of the storage medium in the specification is based on the same inventive concept as the embodiment of the image processing method in the specification, and therefore the specific implementation of the embodiment can be referred to the foregoing implementation of the corresponding method, and the repeated parts will not be described herein.

[0136] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each of the embodiments focuses on the difference from other embodiments, such as the device embodiment, the equipment embodiment and the storage medium embodiment, which are similar to the method embodiment, and therefore the description is relatively simple. The related content in the device embodiment, the equipment embodiment and the storage medium embodiment can be referred to the part of the description of the method embodiment.

[0137] The foregoing describes specific embodiments of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than the order in which the embodiments are presented and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order or sequential order shown or specified in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous or possible.

[0138] In the 1930s, it was clear to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structure of diodes, transistors, switches, etc.) or in software (e.g., improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, called a hardware description language (HDL), of which there are many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., the most commonly used being VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that, by simply logically programming a method flow in one of the above hardware description languages and programming it into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0139] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0140] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0141] For the sake of description, the above apparatuses are described in various units with functions respectively. Of course, the functions of the units can be implemented in one or more software and / or hardware in implementing the embodiments of the present specification.

[0142] Those skilled in the art will understand that one or more embodiments of the present specification can be provided as a method, a system or a computer program product. Therefore, one or more embodiments of the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROMs, optical storage devices, etc.) containing computer usable program code.

[0143] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0144] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Flow diagrams and / or block diagrams can also present a method, apparatus or computer program product to achieve functions specified in flow diagrams and / or block diagrams block or blocks.

[0146] In one typical configuration, the computing device includes one or more processors (CPU's), input / output interfaces, network interfaces, and memory.

[0147] The memory can include non-persistent memory and / or persistent memory, such as flash memory, read-only memory (ROM), and / or volatile or non-volatile random access memory (RAM), among others. The memory is an example of computer-readable media.

[0148] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0149] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0150] One or more embodiments of the specification can be described in the general context of computer-executable instructions being executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0151] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0152] The above merely provides the example of the present document and is not intended to limit the present document. For those skilled in the art, the present document can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present document shall be included in the scope of claims of the present document.

Claims

1. An image processing method, comprising: Acquire the image to be detected and the reconstructed image of the image to be detected; the reconstructed image is obtained by performing an inverse frequency domain transformation on the target frequency domain information of the preset frequency channel extracted from the frequency domain information, and the frequency domain information is obtained by performing a frequency domain transformation on the image to be detected; Feature extraction is performed on the image to be detected and the reconstructed image to obtain the image features of the image to be detected and the reconstructed image features of the reconstructed image; The image features are subjected to feature transformation processing to obtain target features, and the image features, the reconstructed image features, and the target features are fused to obtain a first fused feature; A feature relationship network is constructed based on the image features, and the first fused feature is fused with the pixel relationship features determined based on the feature relationship network to obtain the second fused feature; The image detection result of the image to be detected is determined based on the second fusion feature.

2. The method according to claim 1, wherein feature extraction is performed on the image to be detected and the reconstructed image to obtain image features of the image to be detected and reconstructed image features of the reconstructed image, comprising: The image to be detected is input into a first feature extraction model for feature extraction to obtain multiple image features, and the reconstructed image is input into a second feature extraction model for feature extraction to obtain multiple reconstructed image features. The number of features in the plurality of image features is consistent with the number of features in the plurality of reconstructed image features.

3. The method according to claim 2, wherein the step of performing feature transformation processing on the image features to obtain target features includes: Calculate the corresponding feature matrix based on each of the multiple image features, and calculate the correlation degree of each image feature according to the first feature matrix in the feature matrix; Based on the correlation degree and the second feature matrix in the feature matrix, the output feature matrix of each image feature is calculated as the target feature.

4. The method according to claim 3, wherein calculating the output feature matrix of each image feature based on the correlation degree and the second feature matrix in the feature matrix comprises: The correlation degree is normalized to obtain the normalized correlation degree; Based on the normalized correlation degree and the second feature matrix, the output feature matrix of each image feature is calculated.

5. The method according to claim 1, wherein fusing the image features, the reconstructed image features, and the target features to obtain a first fused feature comprises: Based on the feature parameters, feature fusion is performed on each image feature from multiple image features, each reconstructed image feature from multiple reconstructed image features, and each target feature from multiple target features to obtain multiple first fused features.

6. The method according to claim 1, wherein constructing a feature relationship network based on the image features comprises: The feature parameters of multiple image features are updated, and pixel nodes are determined based on the pixels contained in the updated multiple image features. The feature relationship network is obtained by establishing connections between the pixel nodes based on their positional distances.

7. The method according to claim 1, wherein the pixel relationship features are obtained in the following manner: An adjacency matrix is ​​constructed based on the feature relationship network, serving as the pixel relationship feature.

8. The method according to claim 1, wherein determining the image detection result of the image to be detected based on the second fusion feature comprises: The image to be detected is semantically segmented based on the second fusion feature; The image detection result of the image to be detected is determined based on the semantic segmentation processing result.

9. The method according to claim 8, wherein the semantic segmentation processing of the image to be detected based on the second fusion feature comprises: Based on the second fusion feature, abnormal image blocks and default image blocks in the image to be detected are determined; The pixel values ​​of the pixels contained in the abnormal image block and the pixel values ​​of the pixels contained in the default image block are binarized to obtain a mask image as the semantic segmentation result.

10. The method according to claim 1, wherein determining the image detection result of the image to be detected based on the second fusion feature comprises: The image detection result of the image to be detected is determined based on the second fusion feature, so as to verify the user's identity based on the image detection result.

11. An image processing apparatus, comprising: The image acquisition module is configured to acquire an image to be detected and a reconstructed image of the image to be detected; the reconstructed image is obtained by performing an inverse frequency domain transformation on the target frequency domain information of a preset frequency channel extracted from the frequency domain information, and the frequency domain information is obtained by performing a frequency domain transformation on the image to be detected; The feature extraction module is configured to extract features from the image to be detected and the reconstructed image to obtain image features of the image to be detected and reconstructed image features of the reconstructed image. The first feature fusion module is configured to perform feature transformation processing on the image features to obtain target features, and to fuse the image features, the reconstructed image features and the target features to obtain a first fused feature; The second feature fusion module is configured to construct a feature relationship network based on the image features, and fuse the first fused feature with the pixel relationship features determined based on the feature relationship network to obtain the second fused feature; The result determination module is configured to determine the image detection result of the image to be detected based on the second fusion feature.

12. An image processing apparatus, comprising: processor; And, a memory configured to store computer-executable instructions, which, when executed, cause the processor to: Acquire the image to be detected and the reconstructed image of the image to be detected; the reconstructed image is obtained by performing an inverse frequency domain transformation on the target frequency domain information of the preset frequency channel extracted from the frequency domain information, and the frequency domain information is obtained by performing a frequency domain transformation on the image to be detected; Feature extraction is performed on the image to be detected and the reconstructed image to obtain the image features of the image to be detected and the reconstructed image features of the reconstructed image; The image features are subjected to feature transformation processing to obtain target features, and the image features, the reconstructed image features, and the target features are fused to obtain a first fused feature; A feature relationship network is constructed based on the image features, and the first fused feature is fused with the pixel relationship features determined based on the feature relationship network to obtain the second fused feature; The image detection result of the image to be detected is determined based on the second fusion feature.

Images