An image processing method, apparatus, device and storage medium

Abstract

Embodiments of the present application provide an image processing method, device and equipment, and a storage medium, relating to the technical field of artificial intelligence, which comprises: obtaining a to-be-processed image; predicting a depth estimation value of each pixel point in the to-be-processed image through a depth estimation model; predicting a corresponding sharpness estimation value based on the obtained depth estimation values through a sharpness model; determining a fake point in the to-be-processed image based on the obtained sharpness estimation values and the real sharpness values of the pixel points; and determining the to-be-processed image as a fake image if the number of fake points is greater than a first threshold. The depth estimation value of each pixel point in the to-be-processed image is predicted first, and then the sharpness estimation value of each pixel point is obtained based on the correlation between the depth value and the sharpness value in the real image. The sharpness estimation value is compared with the real sharpness value to determine whether the to-be-processed image is a fake image, thereby improving the accuracy and efficiency of discriminating fake images.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of internet technology, various applications have emerged. In order to improve user experience, some video applications use facial manipulation technology to forge facial images to achieve facial effects in activities; for example, they use facial manipulation technology to forge facial images related to blinking and opening the mouth to achieve facial effects such as blinking and opening the mouth.

[0003] In real life, to ensure financial security, financial applications generally require facial recognition verification, and users need to blink, open their mouths, or perform other actions during facial recognition verification to reduce the risk of being compromised.

[0004] However, forged facial images based on facial manipulation technology may bypass facial recognition verification in financial applications, enabling attacks on facial recognition systems and posing security risks. Therefore, identifying forged images is a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides an image processing method, apparatus, device, and storage medium for identifying forged images and improving the security of face recognition systems.

[0006] On one hand, embodiments of this application provide an image processing method, the method comprising:

[0007] Obtain the image to be processed;

[0008] The depth estimation model is used to predict the depth estimates of multiple pixels in the image to be processed.

[0009] Based on the obtained depth estimates, the sharpness model predicts the corresponding sharpness estimates respectively.

[0010] Based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels, the fake points among the multiple pixels are determined.

[0011] If the number of forged points is greater than a first threshold, then the image to be processed is determined to be a forged image.

[0012] In this embodiment, the depth estimate of each pixel in the image to be processed is first predicted. Then, based on the correlation between depth and sharpness values ​​in the real image, the sharpness estimate of each pixel is predicted based on the depth estimate. By comparing the sharpness estimate with the real sharpness value, it is determined whether the image to be processed is a forged image, thus improving the accuracy and efficiency of forged image identification.

[0013] Optionally, determining the fake points among the multiple pixels based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels includes:

[0014] For each of the multiple pixels, perform the following steps:

[0015] If the difference between the estimated sharpness value of a pixel and the actual sharpness value of the pixel is greater than a second threshold, then the pixel is considered a fake pixel.

[0016] In this embodiment of the application, since there is a specific correlation between the depth value and the sharpness value of a pixel in a real image, after the sharpness estimate of a pixel is predicted based on the correlation and the depth estimate of the pixel, the difference between the sharpness estimate of the pixel and the real sharpness value can be used to determine whether the pixel in the image to be processed is a fake point, so as to determine whether the image to be processed is a fake image in the future.

[0017] Optionally, it also includes:

[0018] For each of the multiple pixels, perform the following steps:

[0019] A pixel is identified as having multiple neighboring pixels, as well as the grayscale difference and distance between the pixel and each neighboring pixel. Based on the obtained grayscale difference and distance, the grayscale change rate between the pixel and each neighboring pixel is determined.

[0020] The true sharpness value of a pixel is obtained by summing the multiple grayscale change rates.

[0021] In this embodiment, the true sharpness value of a pixel is obtained by calculating the grayscale changes of a pixel and multiple neighboring pixels, and summing the multiple grayscale changes, making the true sharpness value of the pixel more accurate.

[0022] Optionally, the sharpness model includes a first convolutional layer, a nonlinear mapping layer, and a second convolutional layer;

[0023] The process of predicting corresponding sharpness estimates based on multiple obtained depth estimates using a sharpness model includes:

[0024] For each depth estimate, the depth estimate is convolved through the first convolutional layer to obtain intermediate features;

[0025] The intermediate features are mapped through the nonlinear mapping layer to determine the feature mapping result;

[0026] The feature mapping result is convolved by the second convolutional layer to obtain the corresponding sharpness estimate.

[0027] Optionally, the depth estimation model and the sharpness model are obtained through joint training.

[0028] In this embodiment of the application, the depth estimation model and the sharpness model are jointly trained, rather than trained separately, so that the training results are based on the depth value and the sharpness value, thereby obtaining the correlation between the depth and the sharpness value.

[0029] Optionally, the depth estimation model and the sharpness model are jointly trained, including:

[0030] Acquire multiple sample images, each with a corresponding depth label and sharpness label;

[0031] The depth estimation model and sharpness model to be trained are jointly iteratively trained using the multiple sample images until the iteration stopping condition is met, and the trained depth estimation model and sharpness model are output. Each iteration process includes the following steps:

[0032] The depth prediction result of the sample image is determined by the depth estimation model to be trained; the sharpness prediction result of the sample image is determined based on the depth prediction result by the sharpness model to be trained.

[0033] Based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label, a target loss value is determined, and the parameters of the depth estimation model and the sharpness model to be trained are adjusted using the target loss value.

[0034] In this embodiment of the application, when training the depth estimation model and the clarity model, continuous iteration can make the model training effect more accurate.

[0035] Optionally, determining the target loss value based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label includes:

[0036] Based on the depth prediction result and the depth label, a first mean square error is determined;

[0037] Based on the sharpness prediction result and the sharpness label, a second mean square error is determined;

[0038] The target loss value is obtained by summing the first mean square error and the second mean square error.

[0039] In this embodiment, the mean squared errors of both the sharpness prediction and depth prediction dimensions are used as the loss function for joint training of the depth estimation model and the sharpness model. This allows the depth estimation model and the sharpness model to influence each other during training, thereby improving the training effect.

[0040] On one hand, embodiments of this application provide an image processing apparatus, which includes:

[0041] The acquisition module is used to acquire the image to be processed;

[0042] The depth prediction module is used to predict the depth estimate of each of the multiple pixels in the image to be processed using a depth estimation model.

[0043] The sharpness prediction module is used to predict the corresponding sharpness estimate based on multiple obtained depth estimates using a sharpness model.

[0044] The determination module is used to determine the fake points among the multiple pixels based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels;

[0045] The judgment module is used to determine that the image to be processed is a fake image if the number of fake points is greater than a first threshold.

[0046] Optionally, the determining module is specifically used for:

[0047] For each of the multiple pixels, perform the following steps:

[0048] If the difference between the estimated sharpness value of a pixel and the actual sharpness value of the pixel is greater than a second threshold, then the pixel is considered a fake pixel.

[0049] Optionally, the determining module is further configured to:

[0050] For each of the multiple pixels, perform the following steps:

[0051] A pixel is identified as having multiple neighboring pixels, as well as the grayscale difference and distance between the pixel and each neighboring pixel. Based on the obtained grayscale difference and distance, the grayscale change rate between the pixel and each neighboring pixel is determined.

[0052] The true sharpness value of a pixel is obtained by summing the multiple grayscale change rates.

[0053] Optionally, the determining module is specifically used for:

[0054] For each depth estimate, the depth estimate is convolved through the first convolutional layer to obtain intermediate features;

[0055] The intermediate features are mapped through the nonlinear mapping layer to determine the feature mapping result;

[0056] The feature mapping result is convolved by the second convolutional layer to obtain the corresponding sharpness estimate.

[0057] Optionally, the depth estimation model and the sharpness model are obtained through joint training.

[0058] Optionally, the determining module is specifically used for:

[0059] Acquire multiple sample images, each with a corresponding depth label and sharpness label;

[0060] The depth estimation model and sharpness model to be trained are jointly iteratively trained using the multiple sample images until the iteration stopping condition is met, and the trained depth estimation model and sharpness model are output. Each iteration process includes the following steps:

[0061] The depth prediction result of the sample image is determined by the depth estimation model to be trained; the sharpness prediction result of the sample image is determined based on the depth prediction result by the sharpness model to be trained.

[0062] Based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label, a target loss value is determined, and the parameters of the depth estimation model and the sharpness model to be trained are adjusted using the target loss value.

[0063] Optionally, the determining module is specifically used for:

[0064] Based on the depth prediction result and the depth label, a first mean square error is determined;

[0065] Based on the sharpness prediction result and the sharpness label, a second mean square error is determined;

[0066] The target loss value is obtained by summing the first mean square error and the second mean square error.

[0067] On one hand, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described image processing method.

[0068] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described image processing method. Attached Figure Description

[0069] Figure 1 A system architecture diagram provided for an embodiment of this application;

[0070] Figure 2 A flowchart of an image processing method provided in this application embodiment Figure 1 ;

[0071] Figure 3 A flowchart for determining a sharpness estimate is provided in an embodiment of this application;

[0072] Figure 4 A schematic diagram of a resolution model structure provided in an embodiment of this application;

[0073] Figure 5 A schematic diagram of a joint training framework for a depth estimation model and a sharpness model provided in this application embodiment;

[0074] Figure 6 A flowchart for determining a target loss value is provided in an embodiment of this application;

[0075] Figure 7 A flowchart of an image processing method provided in this application embodiment Figure 2 ;

[0076] Figure 8 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0077] Figure 9 A computer device provided in an embodiment of this application. Detailed Implementation

[0078] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0079] like Figure 1 The diagram shown is a system architecture diagram of an image processing method provided in an embodiment of this application. The system architecture diagram includes a terminal device 101 and a server 102.

[0080] The terminal device 101 is pre-installed with a business application for identifying forged images. This business application can be a mini-program application or a client application. The server 102 is the backend server for the business application. The server 102 can be a standalone physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0081] The image processing method in this embodiment can be executed by terminal device 101, by server 102, or by interaction between terminal device 101 and server 102.

[0082] based on Figure 1 The system architecture diagram in this application illustrates an image processing method, such as... Figure 2 As shown, the process of this method is executed by a computer device, which can be... Figure 1 The terminal device 101 and / or server 102 shown include the following steps:

[0083] Step 201: Obtain the image to be processed.

[0084] Specifically, the image to be processed can be a bank card image, a membership card image, a face image, or other images that need to be forged.

[0085] Step 202: Predict the depth estimate of each pixel in the image to be processed using the depth estimation model.

[0086] Specifically, pixel depth refers to the number of bits used to store each pixel, and it is also used to measure image resolution. The deeper the pixel depth, the more storage space it occupies; conversely, if the pixel depth is too shallow, it will affect image quality. A trained depth estimation model can predict the depth estimates of multiple pixels in the image to be processed. These multiple pixels can be all pixels in the image or only a subset of pixels.

[0087] Step 203: Based on the obtained depth estimates, predict the corresponding sharpness estimates using the sharpness model.

[0088] Specifically, in real images captured by a camera, there is a correlation between the depth value and the sharpness value of a pixel. Therefore, by training a sharpness model to learn the above correlation, after training, the depth estimates of multiple pixels in the image to be processed are input into the trained sharpness model for prediction, thereby obtaining the sharpness estimates of multiple pixels.

[0089] Step 204: Based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels, identify the fake points among the multiple pixels.

[0090] Specifically, for each of the multiple pixels, the estimated sharpness value of the pixel is compared with the actual sharpness value of the pixel to determine whether the pixel is a fake point.

[0091] In some embodiments, the following steps are performed for each of the multiple pixels:

[0092] If the difference between the estimated sharpness value of a pixel and the actual sharpness value of a pixel is greater than the second threshold, then the pixel is considered a fake pixel.

[0093] Specifically, it is determined whether the difference between the estimated sharpness value and the actual sharpness value of each pixel in the image to be processed is greater than a second threshold. If it is greater than the second threshold, the pixel is considered a fake pixel; otherwise, the pixel is not a fake pixel. The second threshold can be set according to specific requirements.

[0094] For example, if the second threshold is set to 10, and the difference between the estimated sharpness value and the actual sharpness value of a pixel in the image to be processed is 5 (meaning the difference is less than the second threshold), then the pixel is not a fake pixel. If the difference between the estimated sharpness value and the actual sharpness value of the pixel is 20 (meaning the difference is greater than the second threshold), then the pixel is a fake pixel.

[0095] In some embodiments, the following steps are performed for each of the multiple pixels:

[0096] Identify multiple neighboring pixels of a pixel, as well as the grayscale difference and distance between the pixel and each neighboring pixel. Based on the obtained grayscale difference and distance values, determine the grayscale change rate between the pixel and each neighboring pixel. Sum the obtained multiple grayscale change rates to obtain the true sharpness value of the pixel.

[0097] Specifically, this embodiment of the application uses a point sharpness algorithm to determine the true sharpness value of each pixel in the image to be processed. For each pixel in the image to be processed, the eight neighboring pixels adjacent to that pixel are first determined. For each neighboring pixel, the grayscale difference and distance between that pixel and its neighboring pixels are determined. The ratio of the grayscale difference to the distance is used as the grayscale change rate between that pixel and its neighboring pixels. Then, the eight grayscale change rates are summed to obtain the true sharpness value of that pixel, as shown in formula (1):

[0098]

[0099] Where df is the grayscale difference between the pixel and its neighboring pixels, and dx is the distance between the pixel and its neighboring pixels (the 8 neighboring pixels include the 4 neighboring pixels located on the diagonal of the current pixel, and the distance between these 4 neighboring pixels and the current pixel is 0). The distance between this pixel and all other neighboring pixels is 1), where P is the true sharpness value of the pixel.

[0100] Step 205: If the number of fake points is greater than the first threshold, then the image to be processed is determined to be a fake image.

[0101] Specifically, if the number of forged points exceeds a first preset threshold, the image to be processed is considered a forged image. For example, the first preset threshold is set to 1, meaning that if even one pixel in the image to be processed is a forged point, the image is considered a forged image.

[0102] In this embodiment, the depth estimate of each pixel in the image to be processed is first predicted. Then, based on the correlation between depth and sharpness values ​​in the real image, the sharpness estimate of each pixel is predicted based on the depth estimate. By comparing the sharpness estimate with the real sharpness value, it is determined whether the image to be processed is a forged image, thus improving the accuracy and efficiency of forged image identification.

[0103] In some embodiments, the sharpness model includes a first convolutional layer, a nonlinear mapping layer, and a second convolutional layer. The process of predicting sharpness estimates using the sharpness model is as follows: Figure 3 As shown, it includes the following steps:

[0104] Step 301: For each depth estimate, perform convolution processing on the depth estimate through the first convolutional layer to obtain intermediate features.

[0105] Specifically, such as Figure 4The diagram shows the structure of the sharpness model, which includes a first convolutional layer, a nonlinear mapping layer, and a second convolutional layer. After the depth estimation model obtains the depth estimate of the image to be processed, it inputs the depth estimate into the sharpness model, and then performs convolution processing on the depth estimate through the first convolutional layer to obtain intermediate features.

[0106] Step 302: Map the intermediate features through a nonlinear mapping layer to determine the feature mapping result.

[0107] Step 303: Perform convolution processing on the feature mapping results through the second convolutional layer to obtain the corresponding sharpness estimate.

[0108] In some embodiments, the depth estimation model and the sharpness model are jointly trained, and the framework for jointly training the depth estimation model and the sharpness model is as follows: Figure 5 As shown, based on Figure 5 The framework shown illustrates the process of jointly training the depth estimation model and the sharpness model, including the following steps:

[0109] Step 501: Obtain multiple sample images, each with a corresponding depth label and sharpness label.

[0110] Specifically, the depth label corresponding to the sample image includes: the true depth values ​​of multiple sample pixels in the sample image; the sharpness label corresponding to the sample image includes: the true sharpness values ​​of multiple sample pixels in the sample image.

[0111] Step 502: Jointly iteratively train the depth estimation model and sharpness model to be trained using multiple sample images until the iteration stopping condition is met, and output the trained depth estimation model and sharpness model.

[0112] Specifically, each iteration process includes:

[0113] The depth prediction result of the sample image is determined using the depth estimation model to be trained; the sharpness prediction result of the sample image is determined based on the depth prediction result using the sharpness model to be trained. Based on the depth prediction result, sharpness prediction result, depth label, and sharpness label, a target loss value is determined, and the target loss value is used to adjust the parameters of the depth estimation model and sharpness model to be trained.

[0114] Specifically, the sample image is input into the depth estimation model to be trained to obtain the depth prediction result of the sample image, which includes the predicted depth values ​​of multiple sample pixels. The depth prediction result is then input into the sharpness model to be trained to obtain the sharpness prediction result of the sample image, which includes the predicted sharpness values ​​of multiple sample pixels.

[0115] Substitute the depth prediction results, sharpness prediction results, depth labels, and sharpness labels into the loss function to obtain the target loss value, and use the target loss value to adjust the parameters of the depth estimation model and sharpness model to be trained.

[0116] In practical applications, the stopping condition for iteration can be either the convergence of the target loss value or the reaching of a preset number of iterations.

[0117] In some embodiments, the process of determining the target loss value based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label is as follows: Figure 6 As shown, it includes the following steps:

[0118] Step 601: Determine the first mean square error based on the depth prediction results and depth labels.

[0119] Specifically, mean squared error is a measure of the degree of difference between the estimator and the estimated quantity. The first mean squared error is determined based on the depth prediction results and depth labels of the sample images. The specific calculation of the first mean squared error is shown in formula (2):

[0120]

[0121] Among them, F(Y) i ;θ) is the function corresponding to the depth estimation model, where θ is the parameter of the depth estimation model, Y i Let X be the predicted depth value of sample pixel i. i Let θ be the true depth value of sample pixel i, L(θ) be the first mean square error, and n be the number of sample pixels.

[0122] Step 602: Determine the second mean square error based on the sharpness prediction results and sharpness labels.

[0123] The method for calculating the second mean square error is the same as that for calculating the first mean square error, and will not be repeated here.

[0124] Step 603: Sum the first mean square error and the second mean square error to obtain the target loss value.

[0125] Specifically, the first mean square error and the second mean square error are summed to obtain the target loss value, which is calculated as shown in formula (3):

[0126] Loss=MSE(DI,D)+MSE(RP,P)…………(3)

[0127] Where DI is the depth prediction result, D is the depth label, RP is the sharpness prediction result, P is the sharpness label, and Loss is the target loss value.

[0128] In this embodiment, the mean squared errors of both the sharpness prediction and depth prediction dimensions are used as the loss function for joint training of the depth estimation model and the sharpness model. This allows the depth estimation model and the sharpness model to influence each other during training, thereby improving the training effect.

[0129] To better explain the embodiments of this application, the following describes an image processing method provided by the embodiments of this application in conjunction with a specific implementation scenario, including an offline training stage and an online application stage, such as... Figure 7 As shown.

[0130] First, let's introduce the offline training phase, which includes the following steps:

[0131] Step 701: Obtain sample images.

[0132] Step 702: Label the depth and sharpness of the sample images.

[0133] Step 703: Determine the depth prediction result of the sample image using the depth estimation model to be trained.

[0134] Step 704: Determine the sharpness prediction result based on the depth prediction result using the sharpness model.

[0135] Step 705: Determine the target loss value based on the depth prediction result, sharpness prediction result, depth label, and sharpness label.

[0136] Step 706: Use the target loss value to adjust the parameters of the depth estimation model and the sharpness model to be trained, and obtain the trained depth estimation model and sharpness model.

[0137] Next, we will introduce the online application phase, including the following steps:

[0138] Step 707: Obtain the image to be processed.

[0139] Step 708: Input the image to be processed into the trained depth estimation model to obtain the depth estimation values ​​of multiple pixels in the image to be processed.

[0140] Step 709: Input the depth estimates of multiple pixels into the trained sharpness model to obtain the sharpness estimates of multiple pixels.

[0141] Step 710: Use the point sharpness algorithm to calculate the true sharpness value of each of the multiple pixels.

[0142] Step 711: For each pixel among multiple pixels, calculate the difference between the true sharpness value and the estimated sharpness value of the pixel.

[0143] Step 712: Pixels with a difference greater than the second threshold are identified as fake pixels.

[0144] Step 713: Count the number of fake points.

[0145] Step 714: Determine whether the number of fake points is greater than the first threshold. If it is, proceed to step 715; otherwise, proceed to step 716.

[0146] Step 715: Determine that the image to be processed is a fake image.

[0147] Step 716: Determine that the image to be processed is not a fake image.

[0148] In this embodiment, the depth estimate of each pixel in the image to be processed is first predicted. Then, based on the correlation between depth and sharpness values ​​in the real image, the sharpness estimate of each pixel is predicted based on the depth estimate. By comparing the sharpness estimate with the real sharpness value, it is determined whether the image to be processed is a forged image, thus improving the accuracy and efficiency of forged image identification.

[0149] Based on the same technical concept, embodiments of this application provide an image processing apparatus 800, such as... Figure 8 As shown, the device includes:

[0150] The acquisition module 801 is used to acquire the image to be processed;

[0151] Depth prediction module 802 is used to predict the depth estimate value of each of multiple pixels in the image to be processed through a depth estimation model;

[0152] The sharpness prediction module 803 is used to predict the corresponding sharpness estimate based on the multiple depth estimates obtained by the sharpness model.

[0153] The determination module 804 is used to determine the fake points among the multiple pixels based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels;

[0154] The judgment module 805 is used to determine that the image to be processed is a fake image if the number of fake points is greater than a first threshold.

[0155] Optionally, the determining module 804 is specifically used for:

[0156] For each of the multiple pixels, perform the following steps:

[0157] If the difference between the estimated sharpness value of a pixel and the actual sharpness value of the pixel is greater than a second threshold, then the pixel is considered a fake pixel.

[0158] Optionally, the determining module 804 is further configured to:

[0159] For each of the multiple pixels, perform the following steps:

[0160] A pixel is identified as having multiple neighboring pixels, as well as the grayscale difference and distance between the pixel and each neighboring pixel. Based on the obtained grayscale difference and distance, the grayscale change rate between the pixel and each neighboring pixel is determined.

[0161] The true sharpness value of a pixel is obtained by summing the multiple grayscale change rates.

[0162] Optionally, the determining module 804 is specifically used for:

[0163] For each depth estimate, the depth estimate is convolved through the first convolutional layer to obtain intermediate features;

[0164] The intermediate features are mapped through the nonlinear mapping layer to determine the feature mapping result;

[0165] The feature mapping result is convolved by the second convolutional layer to obtain the corresponding sharpness estimate.

[0166] Optionally, the depth estimation model and the sharpness model are obtained through joint training.

[0167] Optionally, the determining module 804 is specifically used for:

[0168] Acquire multiple sample images, each with a corresponding depth label and sharpness label;

[0169] The depth estimation model and sharpness model to be trained are jointly iteratively trained using the multiple sample images until the iteration stopping condition is met, and the trained depth estimation model and sharpness model are output. Each iteration process includes the following steps:

[0170] The depth prediction result of the sample image is determined by the depth estimation model to be trained; the sharpness prediction result of the sample image is determined based on the depth prediction result by the sharpness model to be trained.

[0171] Based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label, a target loss value is determined, and the parameters of the depth estimation model and the sharpness model to be trained are adjusted using the target loss value.

[0172] Optionally, the determining module 804 is specifically used for:

[0173] Based on the depth prediction result and the depth label, a first mean square error is determined;

[0174] Based on the sharpness prediction result and the sharpness label, a second mean square error is determined;

[0175] The target loss value is obtained by summing the first mean square error and the second mean square error.

[0176] Based on the same technical concept, embodiments of this application provide a computer device, which may be a server, such as... Figure 9 The embodiment includes at least one processor 901 and a memory 902 connected to the at least one processor. The specific connection medium between the processor 901 and the memory 902 is not limited in this embodiment. Figure 9 Taking the connection between processor 901 and memory 902 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0177] In this embodiment of the application, the memory 902 stores instructions that can be executed by at least one processor 901. By executing the instructions stored in the memory 902, at least one processor 901 can perform the steps included in the above-described image processing method.

[0178] The processor 901 is the control center of the computer device, capable of connecting to various parts of the device via various interfaces and lines. It performs image processing by running or executing instructions stored in the memory 902 and accessing data stored in the memory 902. Optionally, the processor 901 may include one or more processing units. The processor 901 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 901. In some embodiments, the processor 901 and the memory 902 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0179] The processor 901 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0180] Memory 902, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 902 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 902 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 902 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0181] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described image processing method.

[0182] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0186] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An image processing method, characterized in that, include: Obtain the image to be processed; The depth estimation model is used to predict the depth estimates of multiple pixels in the image to be processed. The sharpness model predicts corresponding sharpness estimates based on multiple depth estimates obtained. The depth values ​​and sharpness values ​​of pixels are correlated. The depth estimation model and the sharpness model are jointly trained, and the mean squared error of the two dimensions of joint sharpness prediction and depth prediction is used as the loss function for model training. Based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels, the fake points among the multiple pixels are determined. If the number of forged points is greater than a first threshold, then the image to be processed is determined to be a forged image; The sharpness model includes a first convolutional layer, a nonlinear mapping layer, and a second convolutional layer; the step of predicting corresponding sharpness estimates based on multiple obtained depth estimates using the sharpness model includes: For each depth estimate, the depth estimate is convolved by the first convolutional layer to obtain intermediate features; the intermediate features are mapped by the nonlinear mapping layer to determine the feature mapping result; and the feature mapping result is convolved by the second convolutional layer to obtain the corresponding sharpness estimate. The depth estimation model and the sharpness model are jointly trained, including: Acquire multiple sample images, each with a corresponding depth label and sharpness label; The depth estimation model and sharpness model to be trained are jointly iteratively trained using the multiple sample images until the iteration stopping condition is met, and the trained depth estimation model and sharpness model are output. Each iteration process includes the following steps: The depth prediction result of the sample image is determined by the depth estimation model to be trained; the sharpness prediction result of the sample image is determined based on the depth prediction result by the sharpness model to be trained. Based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label, a target loss value is determined, and the parameters of the depth estimation model and the sharpness model to be trained are adjusted using the target loss value.

2. The method according to claim 1, characterized in that, The step of determining fake points among the multiple pixels based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels includes: For each of the multiple pixels, perform the following steps: If the difference between the estimated sharpness value of a pixel and the actual sharpness value of the pixel is greater than a second threshold, then the pixel is considered a fake pixel.

3. The method according to claim 1, characterized in that, Also includes: For each of the multiple pixels, perform the following steps: A pixel is identified as having multiple neighboring pixels, as well as the grayscale difference and distance between the pixel and each neighboring pixel. Based on the obtained grayscale difference and distance, the grayscale change rate between the pixel and each neighboring pixel is determined. The true sharpness value of a pixel is obtained by summing the multiple grayscale change rates.

4. The method according to claim 1, characterized in that, The step of determining the target loss value based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label includes: Based on the depth prediction result and the depth label, a first mean square error is determined; Based on the sharpness prediction result and the sharpness label, a second mean square error is determined; The target loss value is obtained by summing the first mean square error and the second mean square error.

5. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the image to be processed; The depth prediction module is used to predict the depth estimate of each of the multiple pixels in the image to be processed using a depth estimation model. The sharpness prediction module is used to predict the corresponding sharpness estimate based on multiple obtained depth estimates using a sharpness model. The determination module is used to determine the fake points among the multiple pixels based on the obtained multiple sharpness estimates and the true sharpness values ​​of each of the multiple pixels, wherein the depth value of the pixel is correlated with the sharpness value; the depth estimation model and the sharpness model are jointly trained, and the mean squared error of the two dimensions of joint sharpness prediction and depth prediction is used as the loss function for model training; The judgment module is used to determine that the image to be processed is a fake image if the number of fake points is greater than a first threshold. The sharpness model includes a first convolutional layer, a nonlinear mapping layer, and a second convolutional layer. The determination module is specifically used to: for each depth estimate, perform convolution processing on the depth estimate through the first convolutional layer to obtain intermediate features; map the intermediate features through the nonlinear mapping layer to determine the feature mapping result; and perform convolution processing on the feature mapping result through the second convolutional layer to obtain the corresponding sharpness estimate. The determining module is also used for: Acquire multiple sample images, each with a corresponding depth label and sharpness label; The depth estimation model and sharpness model to be trained are jointly iteratively trained using the multiple sample images until the iteration stopping condition is met, and the trained depth estimation model and sharpness model are output. Each iteration process includes the following steps: The depth prediction result of the sample image is determined by the depth estimation model to be trained; the sharpness prediction result of the sample image is determined based on the depth prediction result by the sharpness model to be trained. Based on the depth prediction result, the sharpness prediction result, the depth label, and the sharpness label, a target loss value is determined, and the parameters of the depth estimation model and the sharpness model to be trained are adjusted using the target loss value.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1 to 4.

Images