Image processing method and apparatus, device, and storage medium

Through the depth estimation and clarity model, the forged points in the image are jointly predicted and the forged images is identified, which solves the problem of threatening the security of the face recognition system and achieves higher recognition accuracy and efficiency.

WO2024230387A1PCT designated stage expired Publication Date: 2024-11-14CHINA UNIONPAY
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/086859
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-08
Filing Date
2024-04-09
Publication Date
2024-11-14

AI Technical Summary

Technical Problem

The prior art is difficult to effectively identify forged face images, which leads to the threat of the security of the face recognition system. Forged images may avoid the verification of financial applications and cause system attacks.

Method used

The depth estimation value of pixel points in the image is predicted by the depth estimation model, and combined with the clarity model to predict the clarity estimation value, determine the forged point. If the number of forged points exceeds the threshold, the image is judged as a forged image, which improves discrimination accuracy and efficiency.

Benefits of technology

It improves the security of the face recognition system, enhances the accuracy and efficiency of recognition of forged images, and prevents system attacks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024086859_14112024_PF_FP_ABST
    Figure CN2024086859_14112024_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of the present application relate to the technical field of artificial intelligence, and provide an image processing method and apparatus, a device, and a storage medium. The method comprises: acquiring an image to be processed; by means of a depth estimation model, predicting respective depth estimated values of a plurality of pixel points in the image to be processed; respectively predicting corresponding definition estimated values on the basis of the plurality of obtained depth estimated values by means of a definition model; determining forged points among the plurality of pixel points on the basis of the plurality of obtained definition estimated values and respective definition real values of the plurality of pixel points; and if the number of the forged points is greater than a first threshold, determining that the image to be processed is a forged image. The depth estimated values of the pixel points in the image to be processed are first predicted, and then the definition estimated values of the pixel points are predicted on the basis of an association relationship between depth values and definition values in a real image. The definition estimated values are compared with the definition real values to determine whether the image to be processed is a forged image, thereby improving the accuracy and efficiency of forged image determination.
Need to check novelty before this filing date? Find Prior Art

Description

Image processing method, device, equipment and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on May 8, 2023, with application number 202310512786.4 and application name "An image processing method, device, equipment and storage medium", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present invention relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, device and storage medium. Background Art

[0004] With the development of internet technology, various applications have emerged one after another. To enhance user experience, some video applications use facial manipulation technology to forge facial images to achieve animated facial effects. For example, these applications use facial manipulation technology to forge facial images related to blinking and opening the mouth to achieve these effects.

[0005] In real life, in order to ensure financial security, financial applications generally require facial recognition verification, and during facial recognition verification, they need to cooperate with actions such as blinking and opening their mouths to reduce the risk of being hacked.

[0006] However, facial images forged based on facial manipulation technology may evade facial recognition verification in financial applications, enabling attacks on facial recognition systems and thus posing security issues for facial recognition systems. Therefore, how to identify forged images is a technical problem that needs to be solved at present.

[0007] Summary of the Invention

[0008] The embodiments of the present application provide an image processing method, apparatus, device, and storage medium for identifying forged images and improving the security of a face recognition system.

[0009] In one aspect, an embodiment of the present application provides an image processing method, the method comprising:

[0010] Get the image to be processed;

[0011] Predicting depth estimation values ​​of respective pixels in the image to be processed using a depth estimation model;

[0012] Predicting corresponding clarity estimation values ​​based on the obtained multiple depth estimation values ​​using a clarity model;

[0013] determining a forged point among the plurality of pixel points based on the obtained plurality of definition estimation values ​​and the respective true definition values ​​of the plurality of pixel points;

[0014] If the number of the forged points is greater than a first threshold, the image to be processed is determined to be a forged image.

[0015] In this embodiment, the depth estimate of each pixel in the image to be processed is first predicted. Then, based on the correlation between depth and sharpness values ​​in the real image, the sharpness estimate of each pixel is predicted based on the depth estimate of each pixel. By comparing the sharpness estimate with the real sharpness value, it is determined whether the image to be processed is a forged image, improving the accuracy and efficiency of identifying forged images.

[0016] Optionally, the determining of forged points among the plurality of pixel points based on the obtained plurality of clarity estimation values ​​and the true clarity values ​​of the plurality of pixel points comprises:

[0017] For the multiple pixel points, perform the following steps respectively:

[0018] If the difference between the estimated value of the sharpness of a pixel and the true value of the sharpness of the pixel is greater than a second threshold, the pixel is regarded as a forged point.

[0019] In an embodiment of the present application, since there is a specific correlation between the depth value and the clarity value of a pixel point in a real image, after the clarity estimation value of the pixel point is obtained based on the correlation and the depth estimation value of the pixel point, it is possible to determine whether the pixel point in the image to be processed is a forged point based on the difference between the clarity estimation value of the pixel point and the real clarity value, so as to subsequently determine whether the image to be processed is a forged image.

[0020] Optionally, it also includes:

[0021] For the multiple pixel points, perform the following steps respectively:

[0022] Determine multiple neighboring pixels of a pixel point, as well as grayscale differences and distance values ​​between the pixel point and each neighboring pixel point, and determine a grayscale change rate between the pixel point and each neighboring pixel point based on the obtained grayscale differences and distance values;

[0023] The obtained multiple grayscale change rates are summed to obtain the true value of the clarity of the pixel point.

[0024] In an embodiment of the present application, the true value of the clarity of a pixel is obtained by calculating the grayscale changes of a pixel and multiple neighboring pixels and summing the multiple grayscale changes, so that the true value of the clarity of the pixel is more accurate.

[0025] Optionally, the clarity model includes a first convolution layer, a nonlinear mapping layer and a second convolution layer;

[0026] The method of predicting corresponding definition estimation values ​​based on the obtained multiple depth estimation values ​​through the definition model includes:

[0027] For each depth estimation value, convolution processing is performed on the depth estimation value through the first convolution layer to obtain an intermediate feature;

[0028] Mapping the intermediate features through the nonlinear mapping layer to determine a feature mapping result;

[0029] The feature mapping result is convolved by the second convolutional layer to obtain a corresponding clarity estimation value.

[0030] Optionally, the depth estimation model and the clarity model are obtained by joint training.

[0031] In an embodiment of the present application, the depth estimation model and the clarity model are jointly trained instead of being trained separately, so that the training results are based on the depth value and the clarity value, thereby obtaining the correlation between the depth and the clarity value.

[0032] Optionally, the depth estimation model and the clarity model are obtained by joint training, including:

[0033] Obtain multiple sample images, each sample image corresponds to a depth label and a clarity label;

[0034] The depth estimation model and the definition model to be trained are jointly iteratively trained using the multiple sample images until an iteration stop condition is satisfied, and the trained depth estimation model and definition model are output, wherein each iteration process includes the following steps:

[0035] Determine a depth prediction result of a sample image using a depth estimation model to be trained; determine a definition prediction result of the sample image based on the depth prediction result using a definition model to be trained;

[0036] Based on the depth prediction result, the clarity prediction result, the depth label and the clarity label, a target loss value is determined, and the target loss value is used to adjust the parameters of the depth estimation model and the clarity model to be trained.

[0037] In the embodiment of the present application, when training the depth estimation model and the clarity model, continuous iteration can make the model training effect more accurate.

[0038] Optionally, determining a target loss value based on the depth prediction result, the clarity prediction result, the depth label, and the clarity label includes:

[0039] Determining a first mean square error based on the depth prediction result and the depth label;

[0040] determining a second mean square error based on the clarity prediction result and the clarity label;

[0041] The first mean square error and the second mean square error are summed to obtain the target loss value.

[0042] In an embodiment of the present application, the mean square error of the two dimensions of joint clarity prediction and depth prediction is used as the loss function for model training to jointly train the depth estimation model and the clarity model, so that the depth estimation model and the clarity model influence each other during the training process, thereby improving the effect of model training.

[0043] In one aspect, an embodiment of the present application provides an image processing device, comprising:

[0044] An acquisition module, used for acquiring an image to be processed;

[0045] A depth prediction module is used to predict the depth estimation value of each of the plurality of pixel points in the image to be processed by using a depth estimation model;

[0046] A clarity prediction module, configured to predict corresponding clarity estimation values ​​based on the obtained multiple depth estimation values ​​using a clarity model;

[0047] a determination module, configured to determine a forged point among the plurality of pixel points based on the obtained plurality of definition estimation values ​​and the true definition values ​​of the respective plurality of pixel points;

[0048] The judgment module is configured to determine that the image to be processed is a forged image if the number of the forged points is greater than a first threshold.

[0049] Optionally, the determining module is specifically configured to:

[0050] For the multiple pixel points, perform the following steps respectively:

[0051] If the difference between the estimated value of the sharpness of a pixel and the true value of the sharpness of the pixel is greater than a second threshold, the pixel is regarded as a forged point.

[0052] Optionally, the determining module is further configured to:

[0053] For the multiple pixel points, perform the following steps respectively:

[0054] Determine multiple neighboring pixels of a pixel point, as well as grayscale differences and distance values ​​between the pixel point and each neighboring pixel point, and determine a grayscale change rate between the pixel point and each neighboring pixel point based on the obtained grayscale differences and distance values;

[0055] The obtained multiple grayscale change rates are summed to obtain the true value of the clarity of the pixel point.

[0056] Optionally, the determining module is specifically configured to:

[0057] For each depth estimation value, convolution processing is performed on the depth estimation value through the first convolution layer to obtain an intermediate feature;

[0058] Mapping the intermediate features through the nonlinear mapping layer to determine a feature mapping result;

[0059] The feature mapping result is convolved by the second convolutional layer to obtain a corresponding clarity estimation value.

[0060] Optionally, the depth estimation model and the clarity model are obtained by joint training.

[0061] Optionally, the determining module is specifically configured to:

[0062] Obtain multiple sample images, each sample image corresponds to a depth label and a clarity label;

[0063] The depth estimation model and the definition model to be trained are jointly iteratively trained using the multiple sample images until an iteration stop condition is satisfied, and the trained depth estimation model and definition model are output, wherein each iteration process includes the following steps:

[0064] Determine a depth prediction result of a sample image using a depth estimation model to be trained; determine a definition prediction result of the sample image based on the depth prediction result using a definition model to be trained;

[0065] Based on the depth prediction result, the clarity prediction result, the depth label and the clarity label, a target loss value is determined, and the target loss value is used to adjust the parameters of the depth estimation model and the clarity model to be trained.

[0066] Optionally, the determining module is specifically configured to:

[0067] Determining a first mean square error based on the depth prediction result and the depth label;

[0068] determining a second mean square error based on the clarity prediction result and the clarity label;

[0069] The first mean square error and the second mean square error are summed to obtain the target loss value.

[0070] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned image processing method when executing the program.

[0071] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program is run on the computer device, the computer device executes the steps of the above-mentioned image processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] FIG1 is a system architecture diagram provided in an embodiment of the present application;

[0073] FIG2 is a flowchart of an image processing method provided in an embodiment of the present application;

[0074] FIG3 is a flow chart of determining a clarity estimation value provided by an embodiment of the present application;

[0075] FIG4 is a schematic diagram of a clarity model structure provided in an embodiment of the present application;

[0076] FIG5 is a schematic diagram of a joint training framework structure of a depth estimation model and a clarity model provided in an embodiment of the present application;

[0077] FIG6 is a flow chart of determining a target loss value according to an embodiment of the present application;

[0078] FIG7 is a second flowchart of an image processing method provided in an embodiment of the present application;

[0079] FIG8 is a schematic structural diagram of an image processing device provided in an embodiment of the present application;

[0080] FIG9 is a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] In order to make the purpose, technical solutions and beneficial effects of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0082] As shown in FIG1 , it is a system architecture diagram of an image processing method provided in an embodiment of the present application, and the system architecture diagram includes a terminal device 101 and a server 102 .

[0083] The terminal device 101 is pre-installed with a business application for identifying forged images, which can be a mini-program application or a client application. The server 102 is the backend server for the business application. The server 102 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0084] The image processing method in the embodiment of the present application can be executed by the terminal device 101, can be executed by the server 102, or can be executed interactively by the terminal device 101 and the server 102.

[0085] Based on the system architecture diagram of FIG1 , an embodiment of the present application provides an image processing method, as shown in FIG2 . The method is performed by a computer device, which may be the terminal device 101 and / or the server 102 shown in FIG1 , and includes the following steps:

[0086] Step 201: Obtain an image to be processed.

[0087] Specifically, the image to be processed may be an image that needs to be forged, such as a bank card image, a membership card image, or a face image.

[0088] Step 202: predict depth estimation values ​​of respective pixels in the image to be processed using a depth estimation model.

[0089] Specifically, pixel depth refers to the number of bits used to store each pixel and is also used to measure image resolution. A deeper pixel depth occupies more storage space; conversely, a shallow pixel depth can affect image quality. A trained depth estimation model can be used to predict depth estimates for multiple pixels in the image being processed. These multiple pixels can be all or a subset of the pixels in the image being processed.

[0090] Step 203: using a clarity model to predict corresponding clarity estimation values ​​based on the obtained multiple depth estimation values.

[0091] Specifically, in the real image captured by the camera, there is a correlation between the depth value of the pixel and the clarity value. Therefore, the above correlation is learned by training the clarity model. After the training is completed, the depth estimation values ​​of multiple pixels in the image to be processed are input into the trained clarity model for prediction to obtain the clarity estimation values ​​of multiple pixels.

[0092] Step 204: Determine forged pixels among the plurality of pixels based on the obtained plurality of definition estimation values ​​and the respective true definition values ​​of the plurality of pixels.

[0093] Specifically, for each pixel point among the multiple pixel points, the clarity estimation value of the pixel point is compared with the true clarity value of the pixel point to determine whether the pixel point is a forged point.

[0094] In some embodiments, for multiple pixels, the following steps are performed respectively:

[0095] If the difference between the estimated value of the sharpness of a pixel and the true value of the sharpness of the pixel is greater than a second threshold, the pixel is regarded as a forged point.

[0096] Specifically, a determination is made as to whether the difference between the estimated sharpness value and the actual sharpness value of each pixel in the image to be processed is greater than a second threshold. If so, the pixel is considered a forged point; otherwise, the pixel is not a forged point. The second threshold can be set according to specific requirements.

[0097] For example, the second threshold is set to 10. If the difference between the estimated clarity value and the actual clarity value of a pixel point in the image to be processed is 5, that is, the difference between the estimated clarity value and the actual clarity value of the pixel point is less than the second threshold, then the pixel point is not a forged point; if the difference between the estimated clarity value and the actual clarity value of the pixel point is 20, that is, the difference between the estimated clarity value and the actual clarity value of the pixel point is greater than the second threshold, then the pixel point is a forged point.

[0098] In some embodiments, for multiple pixels, the following steps are performed respectively:

[0099] Determine multiple neighboring pixels of a pixel, as well as the grayscale difference and distance values ​​between a pixel and each neighboring pixel, and based on the obtained grayscale difference and distance values, determine the grayscale change rate between a pixel and each neighboring pixel; sum the obtained multiple grayscale change rates to obtain the true clarity value of a pixel.

[0100] Specifically, the embodiment of the present application determines the true value of the clarity of each pixel of the image to be processed by a point sharpness algorithm. For each pixel among the multiple pixels of the image to be processed, first determine the 8 neighboring pixels adjacent to the pixel. For each neighboring pixel, determine the grayscale difference and distance value between the pixel and the neighboring pixel. The ratio of the grayscale difference to the distance value is used as the grayscale change rate between the pixel and the neighboring pixel. Then sum the 8 grayscale change rates obtained to obtain the true value of the clarity of the pixel, as shown in formula (1):

[0101] Among them, df is the grayscale difference between the pixel and its neighboring pixels, dx is the distance between the pixel and its neighboring pixels (the eight neighboring pixels include the four neighboring pixels on the diagonal of the pixel, and the distances between these four neighboring pixels and the pixel are The distance values ​​between other neighboring pixels and this pixel are all 1), and P is the true value of the pixel's clarity.

[0102] Step 205: If the number of forged points is greater than a first threshold, determine that the image to be processed is a forged image.

[0103] Specifically, if the number of forged points is greater than a first preset threshold, the image to be processed is considered a forged image. For example, the first preset threshold is set to 1, that is, as long as there is one forged pixel in the image to be processed, the image to be processed is considered a forged image.

[0104] In this embodiment, the depth estimate of each pixel in the image to be processed is first predicted. Then, based on the correlation between depth and sharpness values ​​in the real image, the sharpness estimate of each pixel is predicted based on the depth estimate of each pixel. By comparing the sharpness estimate with the real sharpness value, it is determined whether the image to be processed is a forged image, improving the accuracy and efficiency of identifying forged images.

[0105] In some embodiments, the clarity model includes a first convolution layer, a nonlinear mapping layer, and a second convolution layer. The process of predicting the clarity estimate value using the clarity model, as shown in FIG3 , includes the following steps:

[0106] Step 301: For each depth estimation value, convolution processing is performed on the depth estimation value through the first convolution layer to obtain intermediate features.

[0107] Specifically, as shown in Figure 4, the clarity model consists of a first convolutional layer, a nonlinear mapping layer, and a second convolutional layer. After the depth estimation model obtains the depth estimate of the image to be processed, it inputs the depth estimate into the clarity model. This depth estimate is then convolved with the depth estimate in the first convolutional layer to produce intermediate features.

[0108] Step 302: Map the intermediate features through a nonlinear mapping layer to determine a feature mapping result.

[0109] Step 303: Perform convolution processing on the feature mapping result through the second convolution layer to obtain a corresponding clarity estimation value.

[0110] In some embodiments, the depth estimation model and the definition model are jointly trained. The framework for the joint training of the depth estimation model and the definition model is shown in FIG5 . The process of jointly training the depth estimation model and the definition model based on the framework shown in FIG5 includes the following steps:

[0111] Step 501: Acquire multiple sample images, each sample image corresponding to a depth label and a definition label.

[0112] Specifically, the depth label corresponding to the sample image includes: the real depth values ​​of multiple sample pixels in the sample image; the definition label corresponding to the sample image includes: the real definition values ​​of multiple sample pixels in the sample image.

[0113] Step 502: perform joint iterative training on the depth estimation model and the clarity model to be trained using multiple sample images until an iteration stop condition is met, and output the trained depth estimation model and clarity model.

[0114] Specifically, each iteration process includes:

[0115] The depth estimation model to be trained is used to determine the depth prediction result of the sample image. The clarity model to be trained is used to determine the clarity prediction result of the sample image based on the depth prediction result. Based on the depth prediction result, clarity prediction result, depth label, and clarity label, a target loss value is determined and used to adjust the parameters of the depth estimation model to be trained and the clarity model to be trained.

[0116] Specifically, a sample image is input into the depth estimation model to be trained to obtain a depth prediction result for the sample image, which includes the predicted depth values ​​of multiple sample pixels. The depth prediction result is input into the definition model to be trained to obtain a definition prediction result for the sample image, which includes the predicted definition values ​​of multiple sample pixels.

[0117] The depth prediction results, clarity prediction results, depth labels, and clarity labels are substituted into the loss function to obtain the target loss value, and the target loss value is used to adjust the parameters of the depth estimation model and clarity model to be trained.

[0118] In practical applications, the iteration stopping condition can be the convergence of the target loss value or the number of iterations reaching a preset number.

[0119] In some embodiments, a process of determining a target loss value based on the depth prediction result, the clarity prediction result, the depth label, and the clarity label is shown in FIG6 , and includes the following steps:

[0120] Step 601: Determine a first mean square error based on a depth prediction result and a depth label.

[0121] Specifically, the mean square error is a measure of the difference between the estimator and the estimated quantity. The first mean square error is determined based on the depth prediction result and depth label of the sample image. The specific calculation of the first mean square error is shown in formula (2):

[0122] Among them, F(Y i ; θ) is the function corresponding to the depth estimation model, θ is the parameter of the depth estimation model, Y i is the predicted depth value of sample pixel i, X i is the true depth value of sample pixel i, L(θ) is the first mean square error, and n is the number of sample pixels.

[0123] Step 602: Determine a second mean square error based on the clarity prediction result and the clarity label.

[0124] The method for calculating the second mean square error is the same as the method for calculating the first mean square error, and will not be repeated here.

[0125] Step 603: Sum the first mean square error and the second mean square error to obtain a target loss value.

[0126] Specifically, the first mean square error and the second mean square error are summed to obtain the target loss value. The target loss value is calculated as shown in formula (3): Loss = MSE (DI, D) + MSE (RP, P)............(3)

[0127] Among them, DI is the depth prediction result, D is the depth label, RP is the clarity prediction result, P is the clarity label, and Loss is the target loss value.

[0128] In an embodiment of the present application, the mean square error of the two dimensions of joint clarity prediction and depth prediction is used as the loss function for model training to jointly train the depth estimation model and the clarity model, so that the depth estimation model and the clarity model influence each other during the training process, thereby improving the effect of model training.

[0129] In order to better explain the embodiment of the present application, an image processing method provided by the embodiment of the present application is introduced below in combination with a specific implementation scenario, including an offline training stage and an online application stage, as shown in Figure 7.

[0130] First, we introduce the offline training phase, which includes the following steps:

[0131] Step 701: Acquire a sample image.

[0132] Step 702: Label the depth label and definition label of the sample image.

[0133] Step 703: Determine the depth prediction result of the sample image using the depth estimation model to be trained.

[0134] Step 704: Determine a clarity prediction result based on the depth prediction result using a clarity model.

[0135] Step 705: Determine a target loss value based on the depth prediction result, the clarity prediction result, the depth label, and the clarity label.

[0136] Step 706: Use the target loss value to adjust the parameters of the depth estimation model and clarity model to be trained to obtain the trained depth estimation model and clarity model.

[0137] Next, we introduce the online application phase, which includes the following steps:

[0138] Step 707: Obtain the image to be processed.

[0139] Step 708: Input the image to be processed into the trained depth estimation model to obtain depth estimation values ​​of multiple pixels in the image to be processed.

[0140] Step 709: Input the depth estimation values ​​of the multiple pixel points into the trained clarity model to obtain the clarity estimation values ​​of the multiple pixel points.

[0141] Step 710: Use a point sharpness algorithm to calculate the true clarity value of each of the multiple pixel points.

[0142] Step 711: For each pixel among the plurality of pixels, calculate the difference between the true clarity value and the estimated clarity value of the pixel.

[0143] Step 712: Pixel points whose difference is greater than the second threshold are regarded as forged points.

[0144] Step 713: Count the number of forged points.

[0145] Step 714: Determine whether the number of forged points is greater than a first threshold. If so, execute step 715; otherwise, execute step 716.

[0146] Step 715: Determine whether the image to be processed is a forged image.

[0147] Step 716: Determine whether the image to be processed is a forged image.

[0148] In this embodiment, the depth estimate of each pixel in the image to be processed is first predicted. Then, based on the correlation between depth and sharpness values ​​in the real image, the sharpness estimate of each pixel is predicted based on the depth estimate of each pixel. By comparing the sharpness estimate with the real sharpness value, it is determined whether the image to be processed is a forged image, improving the accuracy and efficiency of identifying forged images.

[0149] Based on the same technical concept, an embodiment of the present application provides an image processing device 800, as shown in FIG8 , which includes:

[0150] An acquisition module 801 is used to acquire an image to be processed;

[0151] A depth prediction module 802 is configured to predict depth estimation values ​​of respective pixels in the image to be processed using a depth estimation model;

[0152] a clarity prediction module 803, configured to predict corresponding clarity estimation values ​​based on the obtained multiple depth estimation values ​​using a clarity model;

[0153] a determination module 804, configured to determine a forged point among the plurality of pixels based on the obtained plurality of definition estimation values ​​and the true definition values ​​of the plurality of pixels;

[0154] The judgment module 805 is configured to determine that the image to be processed is a forged image if the number of the forged points is greater than a first threshold.

[0155] Optionally, the determining module 804 is specifically configured to:

[0156] For the multiple pixel points, perform the following steps respectively:

[0157] If the difference between the estimated value of the sharpness of a pixel and the true value of the sharpness of the pixel is greater than a second threshold, the pixel is regarded as a forged point.

[0158] Optionally, the determining module 804 is further configured to:

[0159] For the multiple pixel points, perform the following steps respectively:

[0160] Determine multiple neighboring pixels of a pixel point, as well as grayscale differences and distance values ​​between the pixel point and each neighboring pixel point, and determine a grayscale change rate between the pixel point and each neighboring pixel point based on the obtained grayscale differences and distance values;

[0161] The obtained multiple grayscale change rates are summed to obtain the true value of the clarity of the pixel point.

[0162] Optionally, the determining module 804 is specifically configured to:

[0163] For each depth estimation value, convolution processing is performed on the depth estimation value through the first convolution layer to obtain an intermediate feature;

[0164] Mapping the intermediate features through the nonlinear mapping layer to determine a feature mapping result;

[0165] The feature mapping result is convolved by the second convolutional layer to obtain a corresponding clarity estimation value.

[0166] Optionally, the depth estimation model and the clarity model are obtained by joint training.

[0167] Optionally, the determining module 804 is specifically configured to:

[0168] Obtain multiple sample images, each sample image corresponds to a depth label and a clarity label;

[0169] The depth estimation model and the definition model to be trained are jointly iteratively trained using the multiple sample images until an iteration stop condition is satisfied, and the trained depth estimation model and definition model are output, wherein each iteration process includes the following steps:

[0170] Determine a depth prediction result of a sample image using a depth estimation model to be trained; determine a definition prediction result of the sample image based on the depth prediction result using a definition model to be trained;

[0171] Based on the depth prediction result, the clarity prediction result, the depth label and the clarity label, a target loss value is determined, and the target loss value is used to adjust the parameters of the depth estimation model and the clarity model to be trained.

[0172] Optionally, the determining module 804 is specifically configured to:

[0173] Determining a first mean square error based on the depth prediction result and the depth label;

[0174] determining a second mean square error based on the clarity prediction result and the clarity label;

[0175] The first mean square error and the second mean square error are summed to obtain the target loss value.

[0176] Based on the same technical concept, an embodiment of the present application provides a computer device, which may be a server, as shown in FIG9 . The computer device includes at least one processor 901 and a memory 902 connected to the at least one processor. The embodiment of the present application does not limit the specific connection medium between the processor 901 and the memory 902. FIG9 takes the example of a bus connection between the processor 901 and the memory 902. Buses can be divided into address buses, data buses, control buses, etc.

[0177] In the embodiment of the present application, the memory 902 stores instructions that can be executed by at least one processor 901. The at least one processor 901 can perform the steps included in the above-mentioned image processing method by executing the instructions stored in the memory 902.

[0178] The processor 901 is the control center of the computer device. It can connect various parts of the computer device using various interfaces and lines, and perform image processing by running or executing instructions stored in the memory 902 and calling data stored in the memory 902. Optionally, the processor 901 may include one or more processing units. The processor 901 may integrate an application processor and a modem processor. The application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 901. In some embodiments, the processor 901 and the memory 902 may be implemented on the same chip. In some embodiments, they may also be implemented on separate chips.

[0179] The processor 901 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor.

[0180] The memory 902 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 902 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 902 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 902 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0181] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by a computer device. When the program runs on the computer device, the computer device executes the steps of the above-mentioned image processing method.

[0182] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0183] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0184] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0186] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. An image processing method, wherein: include: Get the image to be processed; Predicting depth estimation values ​​of respective pixels in the image to be processed by a depth estimation model; Predicting corresponding clarity estimation values ​​based on the obtained multiple depth estimation values ​​through a clarity model; Determine a forged point among the plurality of pixel points based on the obtained plurality of definition estimation values ​​and the respective true definition values ​​of the plurality of pixel points; If the number of the forged points is greater than a first threshold, the image to be processed is determined to be a forged image.

2. The method according to claim 1, wherein: The step of determining the forged points among the plurality of pixel points based on the obtained plurality of definition estimation values ​​and the respective true definition values ​​of the plurality of pixel points comprises: For the multiple pixel points, the following steps are performed respectively: If the difference between the estimated value of the clarity of a pixel and the true value of the clarity of the pixel is greater than a second threshold, the pixel is taken as a forged point.

3. The method according to claim 1, wherein: Also includes: For the multiple pixel points, the following steps are performed respectively: Determine multiple neighboring pixel points of a pixel point, as well as grayscale difference and distance values ​​between the pixel point and each neighboring pixel point, and determine the grayscale change rate between the pixel point and each neighboring pixel point based on the obtained grayscale difference and distance values; The obtained multiple grayscale change rates are summed to obtain the true value of the clarity of the one pixel.

4. The method according to claim 1, wherein: The definition model includes a first convolution layer, a nonlinear mapping layer and a second convolution layer; The method of predicting corresponding definition estimation values ​​based on the obtained multiple depth estimation values ​​through the definition model includes: For each depth estimation value, convolution processing is performed on the depth estimation value through the first convolution layer to obtain an intermediate feature; Mapping the intermediate features through the nonlinear mapping layer to determine a feature mapping result; The feature mapping result is convolved by the second convolution layer to obtain a corresponding clarity estimation value.

5. The method according to any one of claims 1 to 4, wherein: The depth estimation model and the definition model are obtained by joint training.

6. The method according to claim 5, wherein: The depth estimation model and the definition model are obtained by joint training, including: Obtain multiple sample images, each sample image corresponds to a depth label and a clarity label; The depth estimation model and the definition model to be trained are jointly iteratively trained by using the multiple sample images until an iteration stop condition is satisfied, and the trained depth estimation model and definition model are output, wherein each iteration process comprises the following steps: Determine the depth prediction result of the sample image by using the depth estimation model to be trained; determine the definition prediction result of the sample image based on the depth prediction result by using the definition model to be trained; Based on the depth prediction result, the clarity prediction result, the depth label and the clarity label, a target loss value is determined, and the depth estimation model and the clarity model to be trained are parameterized using the target loss value. Number adjustment.

7. The method according to claim 6, wherein: The determining a target loss value based on the depth prediction result, the clarity prediction result, the depth label, and the clarity label includes: Determining a first mean square error based on the depth prediction result and the depth label; Determining a second mean square error based on the clarity prediction result and the clarity label; The first mean square error and the second mean square error are summed to obtain the target loss value.

8. An image processing device, wherein: include: An acquisition module, used for acquiring an image to be processed; A depth prediction module, used to predict depth estimation values ​​of each of a plurality of pixel points in the image to be processed by using a depth estimation model; A clarity prediction module, configured to predict corresponding clarity estimation values ​​based on the obtained multiple depth estimation values ​​through a clarity model; A determination module, configured to determine a forged point among the plurality of pixel points based on the obtained plurality of definition estimation values ​​and the respective true definition values ​​of the plurality of pixel points; The judgment module is configured to determine that the image to be processed is a forged image if the number of the forged points is greater than a first threshold.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of any one of the methods of claims 1 to 7 are implemented.

10. A computer-readable storage medium, wherein: It stores a computer program executable by a computer device. When the program is run on the computer device, the computer device executes the steps of any method described in claims 1 to 7.

11. A computer program product, wherein: The computer program product stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer is caused to execute the steps of any one of the methods of claims 1 to 7.

Citation Information

Patent Citations

  • Living body detection method and device, electronic equipment and storage medium

    CN114333078A

  • Face living body detection method and device, electronic equipment and readable storage medium

    CN115082992A

  • Image processing method and device, equipment and storage medium

    CN116824211A

  • Three-dimensional living-body face detection method, face authentication recognition method, and apparatuses

    US20200019760A1

  • Face Anti-spoofing detection method and device, and readable storage medium and computer program product

    WO2023060756A1