Image processing method and apparatus

Abstract

Embodiments of the present specification provide image processing methods and devices, wherein an image processing method comprises: performing image set construction according to an obtained to-be-detected image and an associated image, calculating an image classification parameter based on image features of each image in a target image set obtained through the construction, updating an evaluation index threshold value and a confidence threshold value if the image classification parameter is within a preset parameter interval, obtaining a target evaluation index threshold value and a target confidence threshold value, calculating an image confidence according to an image evaluation result in a case where the image evaluation result obtained through image evaluation on the to-be-detected image does not meet the target evaluation index threshold value, and finally determining an image detection result of the to-be-detected image by means of the image confidence and the target confidence threshold value.

Description

Technical Field

[0001] This document relates to the field of image processing technology, and in particular to an image processing method and apparatus. Background Technology

[0002] With the continuous development of internet technology, more and more online services have emerged. Users no longer need to go to offline service institutions for service processing, but can directly process the relevant information through online services. In order to improve the security of online services, more and more online services require users to provide corresponding user images for identity verification, such as user-uploaded identity certificate images, driver's license images, etc. In this process, how to better manage user-uploaded user images has gradually become a focus of attention for all parties. Summary of the Invention

[0003] This specification provides one or more embodiments of an image processing method, comprising: acquiring an image to be detected and associated images and constructing an image set; calculating image classification parameters based on image features of each image in the constructed target image set; updating an evaluation index threshold and a confidence threshold if the image classification parameters are within a preset parameter range to obtain a target evaluation index threshold and a target confidence threshold; performing image evaluation on the image to be detected to obtain an image evaluation result; and calculating an image confidence score based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold; and determining an image detection result for the image to be detected based on the image confidence score and the target confidence threshold.

[0004] This specification provides one or more embodiments of an image processing apparatus, comprising: a parameter calculation module configured to acquire an image to be detected and associated images and construct an image set, and calculate image classification parameters based on image features of each image in the constructed target image set; a threshold update module configured to update an evaluation index threshold and a confidence threshold if the image classification parameters are within a preset parameter range, to obtain a target evaluation index threshold and a target confidence threshold; a confidence calculation module configured to perform image evaluation on the image to be detected to obtain an image evaluation result, and calculate an image confidence score based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold; and a result determination module configured to determine the image detection result of the image to be detected based on the image confidence score and the target confidence threshold.

[0005] This specification provides one or more embodiments of an image processing apparatus, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: acquire an image to be detected and associated images and construct an image set; calculate image classification parameters based on image features of each image in the constructed target image set; update an evaluation index threshold and a confidence threshold if the image classification parameters are within a preset parameter range to obtain a target evaluation index threshold and a target confidence threshold; perform image evaluation on the image to be detected to obtain an image evaluation result, and calculate an image confidence score based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold; and determine an image detection result for the image to be detected based on the image confidence score and the target confidence threshold.

[0006] This specification provides one or more embodiments of a storage medium for storing computer-executable instructions, which, when executed by a processor, implement the following process: acquiring an image to be detected and associated images and constructing an image set; calculating image classification parameters based on image features of each image in the constructed target image set; updating evaluation index thresholds and confidence thresholds if the image classification parameters are within a preset parameter range to obtain target evaluation index thresholds and target confidence thresholds; performing image evaluation on the image to be detected to obtain an image evaluation result; and calculating an image confidence score based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold; and determining the image detection result of the image to be detected based on the image confidence score and the target confidence threshold. Attached Figure Description

[0007] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0008] Figure 1 A schematic diagram illustrating the implementation environment of an image processing method provided in one or more embodiments of this specification;

[0009] Figure 2 A flowchart of an image processing method provided for one or more embodiments of this specification;

[0010] Figure 3 A flowchart illustrating an image processing method for an identity credential scenario, provided in one or more embodiments of this specification;

[0011] Figure 4 A schematic diagram of an embodiment of an image processing apparatus provided by one or more embodiments of this specification;

[0012] Figure 5 This is a schematic diagram of the structure of an image processing device provided for one or more embodiments of this specification. Detailed Implementation

[0013] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0014] Reference Figure 1 This specification provides a schematic diagram of the implementation environment for one or more embodiments of the image processing method.

[0015] The image processing method provided in one or more embodiments of this specification is applicable to an implementation environment that performs image detection on the image to be detected and associated images and determines the image detection result. This implementation environment includes at least an image detection server 101.

[0016] In addition, the implementation environment may also include a user terminal 102, which can be configured to interact with the image detection server 101. The client may be an application, a subroutine within an application, a service module within an application, or a web application.

[0017] The image detection server 101 can correspond to a single server, a server cluster consisting of several servers, or one or more cloud servers in a cloud computing platform, and is used to perform image detection on the image to be detected and related images and determine the image detection results.

[0018] User terminal 102 can be a mobile phone, personal computer, tablet computer, e-book reader, device for information interaction based on VR (Virtual Reality) technology, vehicle terminal, IoT device, wearable smart device, laptop computer and desktop computer, etc. User terminal 102 is used to upload the image to be detected to image detection server 101.

[0019] In this implementation environment, after obtaining the image to be detected uploaded by the user terminal 102, the image detection server 101 can obtain the associated images of the image to be detected, and construct an image set based on the image to be detected and its associated images. It can then calculate image classification parameters based on the image features of each image in the constructed target image set. If the image classification parameters are within a preset parameter range, it can update the subsequent evaluation index thresholds and confidence thresholds to obtain the target evaluation index thresholds and target confidence thresholds. If the image evaluation result obtained from evaluating the image to be detected does not meet the target evaluation index thresholds, it can determine the image detection result of the image to be detected based on the target confidence threshold and the image confidence calculated based on the image evaluation result. Thus, by updating the evaluation index thresholds and confidence thresholds through image classification parameters, the accuracy of image detection can be improved. Combining the image to be detected and associated images for image detection enhances the comprehensiveness and flexibility of image detection.

[0020] One or more embodiments of an image processing method provided in this specification are as follows:

[0021] Reference Figure 2 The image processing method provided in this embodiment specifically includes steps S202 to S208.

[0022] Step S202: Obtain the image to be detected and associated images and construct an image set. Based on the image features of each image in the constructed target image set, calculate the image classification parameters.

[0023] The image to be detected in this embodiment refers to an image awaiting detection; the image to be detected includes images uploaded by the user; optionally, the image to be detected includes an identity image to be detected, such as an identity credential image, bank card image, driver's license image, medical insurance certificate image, etc. The image to be detected can be an RGB (Red, Green, Blue) image, and may also include a card image to be detected. Specifically, the card image to be detected may represent the user's card image and / or credential image.

[0024] The image to be detected can be any image used for identity verification. For example, the image to be detected can be an image containing the user's biometric features, such as an image containing the user's iris features, an image containing the user's fingerprint features, an image containing the user's facial features, and so on.

[0025] The associated image refers to an image associated with the image to be detected; the associated image includes historical images of users that failed detection under the target service; for example, if the image to be detected is uploaded through the target service, the associated image represents a previous historical image of the image to be detected that failed detection under the target service; the target service here includes any service that requires image detection of the image to be detected, such as an account registration service that requires collecting user facial images for identity verification. In this embodiment, the associated image obtained can be one or more images.

[0026] Optionally, the associated image can be obtained from a database; the database includes a database storing users' associated images; the database can be determined according to the service type of the target service, and each target service can correspond to a different database. Alternatively, the database can be determined not according to the service type of the target service, and each target service can correspond to the same database. In one optional implementation provided in this embodiment, the associated image is obtained in the following way:

[0027] Determine the user identifier and / or device identifier of the user who uploaded the image to be detected;

[0028] Based on the user identifier and / or the device identifier, read the associated image of the image to be detected from the database.

[0029] The user identifier of the image to be detected includes the user account that uploaded the image to be detected; the device identifier includes the terminal identifier of the user terminal that uploaded the image to be detected.

[0030] Optionally, the database stores associated images corresponding to the user identifier and / or the device identifier; if the number of associated images corresponding to the user identifier and / or the device identifier exceeds the image number threshold, the target images in the associated images corresponding to the user identifier and / or the device identifier are removed in chronological order.

[0031] For example, to reduce the load on the database, the image number threshold is set to m. If the number of associated images corresponding to user identifier a exceeds m, the m images that are later in time can be determined from the associated images according to the time sequence, and the target images other than the m images can be removed from the associated images.

[0032] Furthermore, when all target services correspond to the same database, the associated image can be obtained as follows: determine the user account that uploaded the image to be detected; based on the user account, read the associated image of the image to be detected from the database. The user account includes the user's application account.

[0033] The image classification parameters described in this embodiment include parameters characterizing the image classification of the target image set, such as image classification scores or image classification confidence levels. Specifically, the image classification parameters may include parameters characterizing whether the target image set represents multiple attempts (multiple image uploads) by a normal user or multiple attacks (multiple image uploads) by a malicious user. The image features include feature maps of each image, specifically including feature maps of each image in at least one dimension. These dimensions may include quality dimensions, text dimensions, and / or forgery dimensions. Furthermore, these dimensions may include other types of dimensions. The forgery dimension includes a dimension representing whether each image is a forged image.

[0034] In real-world identity verification scenarios, users often upload images to be verified multiple times, only to fail the verification process each time. This can lead to two possible scenarios: either a regular user repeatedly uploads images for verification, or a malicious user repeatedly uploads images for malicious attacks. Relying solely on real-time uploaded images for verification could repeatedly disturb regular users, reducing their motivation, and would also result in low accuracy in identifying repeated attacks from malicious users.

[0035] To address this, in order to improve the accuracy of identifying multiple image attacks by malicious users, and at the same time improve the accuracy of identifying multiple image attempts by ordinary users, thereby reducing the disturbance to multiple attempts by ordinary users, in addition to obtaining the image to be detected, it is also possible to obtain the associated image of the image to be detected. The associated image represents the image that failed the detection when uploaded by the user before the image to be detected. The image detection result of the image to be detected is determined by combining the image to be detected and the associated image.

[0036] In practical applications, the process of image detection of the image to be detected and related images can be configured with a preset number of images, such as n. If the total number of images of the image to be detected and related images does not reach the preset number n, image padding is performed on the image to be detected and related images to bring the total number of images to the preset number n, thereby improving the convenience of image detection of the image to be detected and related images. In an optional implementation of this embodiment, the image set construction includes:

[0037] An image set is constructed based on the image to be detected and the associated image, and it is detected whether the number of images in the image set is less than a preset number of images;

[0038] If not, the image set shall be used as the target image set.

[0039] The preset number of images here can be the image number threshold used in the above process of removing target images from associated images.

[0040] In one optional implementation of this embodiment, if the result of the above-mentioned detection of whether the number of images in the image set is less than the preset number of images is yes, the following operation is performed:

[0041] The image set is filled with images based on the number of images and the preset number of images, and the image set after image filling is used as the target image set.

[0042] Specifically, the images to be detected and associated images can be combined in chronological order, and the number of images in the combined image set can be checked to see if it is less than a preset number of images. If not, the image set is taken as the target image set. If so, specific images are filled into the image set based on the number of images and the preset number of images, and the filled image set is taken as the target image set. The specific images include blank images. In the process of filling specific images into the combined image set, blank images can be randomly filled into the combined image set. In addition, in the process of filling specific images into the image set based on the number of images and the preset number of images, the number of blank images to be filled can be determined based on the number of images and the preset number of images. The number of blank images to be filled into the combined image set is intermittently filled according to the arrangement order of the images in the combined image set. For example, if the number of filling is 'a', 'a' blank images are intermittently filled into the combined image set from front to back. The number of intervals used here is not specifically limited and is determined according to the actual application scenario.

[0043] In practical applications, when ordinary users upload images multiple times but fail image detection, the differences between the uploaded images may be minor, with only slight variations in shooting angle and lighting. However, malicious users, when uploading images multiple times but failing image detection, may upload images with significant differences, such as different shapes, angles, or occlusion areas of the occluded objects. To address this, image classification parameters can be calculated by analyzing the differences in image features among images in the target image set, thereby improving the accuracy of the image classification parameters. In one optional implementation of this embodiment, the following operations are performed during the calculation of image classification parameters based on the image features of each image in the constructed target image set:

[0044] Based on the image feature maps of each image, calculate the first feature residual in the time dimension and the second feature residual in the spatial dimension;

[0045] Based on the first feature residual and the second feature residual, calculate the image classification parameters of the target image set.

[0046] In the process of calculating the first feature residual in the time dimension and the second feature residual in the spatial dimension based on the image feature maps of the aforementioned images, in an optional implementation of this embodiment, the following operations are performed:

[0047] Target feature blocks are extracted from the image feature maps of each image in chronological order, and the first feature residual is calculated based on the target feature blocks; and,

[0048] The second feature residual is calculated based on the image feature maps of each image.

[0049] The target feature block includes target feature blocks in each image feature map. For example, each image feature map contains 9 feature blocks, and the target feature block is the feature block at the same position in each image feature map. Specifically, the target feature block can be the first, second, and / or third feature block in each image feature map. The first feature residual includes the difference in pixel values ​​of pixels in the target feature blocks of adjacent image feature maps. The second feature residual includes the difference in pixel values ​​of pixels in each feature block of the image feature map.

[0050] For example, each image feature map contains 9 feature blocks. The difference in pixel values ​​of pixels in the same position of target feature blocks in adjacent image feature maps is calculated as the first feature residual, and the difference in pixel values ​​of pixels in each feature block of each image feature map is calculated as the second feature residual. That is, in the process of calculating the first feature residual in the time dimension and the second feature residual in the spatial dimension based on the image feature maps of each image, in order to improve the accuracy of residual calculation and to calculate image classification parameters more accurately, the following operations can also be performed: extract target feature blocks from the image feature maps of each image in chronological order, calculate the difference in pixel values ​​of pixels in the same position of target feature blocks in adjacent image feature maps as the first feature residual, and calculate the difference in pixel values ​​of pixels in each feature block of each image feature map as the second feature residual.

[0051] Furthermore, in the process of calculating the first feature residual in the time dimension and the second feature residual in the spatial dimension based on the image feature maps of each image, the following operations can also be performed: extracting target feature blocks from the image feature maps of each image in chronological order, inputting the extracted target feature blocks into a temporal network to calculate the feature residual in the time dimension, and obtaining the first feature residual in the time dimension; and inputting each feature block in the image feature map of each image into a spatial network to calculate the feature residual in the spatial dimension, and obtaining the second feature residual in the spatial dimension.

[0052] The time network can adopt a transformer neural network structure; the spatial network can also adopt a transformer neural network structure.

[0053] In the process of calculating the image classification parameters of the target image set based on the first feature residual and the second feature residual, the following operations can be performed:

[0054] The first and second feature residuals are fused, and the image classification parameters of the target image set are calculated based on the fused feature residuals.

[0055] In addition, during the process of calculating the image classification parameters of the target image set based on the first feature residual and the second feature residual, the following operation can also be performed: input the first feature residual and the second feature residual into the fully connected layer for parameter calculation to obtain the image classification parameters of the target image set.

[0056] In addition, in the process of calculating image classification parameters based on the image features of each image in the target image set obtained above, in order to improve the flexibility of the image classification parameters, the following operations can also be performed: based on the image feature maps of each image, calculate the first feature residual in the time dimension and / or the second feature residual in the spatial dimension; calculate the image classification parameters of the target image set according to the first feature residual and / or the second feature residual.

[0057] Step S204: If the image classification parameters are within the preset parameter range, update the evaluation index threshold and confidence threshold to obtain the target evaluation index threshold and target confidence threshold.

[0058] The above steps involve acquiring the image to be detected and associated images, constructing an image set, and calculating image classification parameters based on the image features of each image in the constructed target image set. If the image classification parameters are not within the preset parameter range, no processing is required, or the current evaluation index threshold and the current confidence threshold are used as the target evaluation index threshold and the target confidence threshold, respectively, and the following step S206 is executed. In this step, if the image classification parameters are within the preset parameter range, the evaluation index threshold and the confidence threshold are updated to obtain the target evaluation index threshold and the target confidence threshold.

[0059] The preset parameter range mentioned in this embodiment refers to a range of pre-set image classification parameters. For example, if the parameter range is set to ab, bc, and cd, the preset parameter range is ab. Or, if the image classification parameter is classification confidence, the parameter range is a high confidence range, a medium confidence range, and a low confidence range, the preset parameter range is a high confidence range.

[0060] The evaluation index thresholds include index thresholds required for image evaluation of the image to be detected. Optionally, the evaluation index thresholds may include image quality score thresholds, text confidence thresholds, and / or image forgery index thresholds, such as image quality score thresholds, which include sharpness thresholds, integrity thresholds, etc. Furthermore, the evaluation index thresholds may also include other types of index thresholds. The confidence threshold includes the probability or degree to which the image to be detected passes image detection, or the probability that the image to be detected is an image uploaded by a normal user or an image uploaded by a malicious user.

[0061] In specific implementation, in order to improve the image detection pass rate and success rate of the image to be detected when the image classification parameters are within the preset parameter range, in an optional implementation method provided in this embodiment, the following operations are performed during the process of updating the evaluation index threshold and confidence threshold:

[0062] The confidence threshold, the quality score threshold, the text confidence threshold, and / or the image forgery index threshold in the evaluation index threshold are adjusted downwards.

[0063] Specifically, by lowering the indicator threshold and confidence threshold, the image detection pass rate of the images to be detected is improved, preventing situations where ordinary users fail multiple times while malicious users succeed in multiple attacks. This enhances the security of image detection. When the image classification parameters are within the preset parameter range, lowering the indicator threshold and confidence threshold means that the probability of the target image set being multiple attempts or uploads by ordinary users is higher, while the probability of malicious users attacking by uploading images multiple times is lower. Therefore, by lowering the indicator threshold and confidence threshold, the detection requirements for ordinary users are relaxed, thereby improving the flexibility and effectiveness of image detection, enhancing the user experience, and avoiding the disturbance and poor experience caused to ordinary users by prolonged detection failures.

[0064] In addition, during the process of updating the evaluation index threshold and confidence threshold to obtain the target evaluation index threshold and target confidence threshold, the following operation can also be performed: update the evaluation index threshold or the confidence threshold to obtain the target evaluation index threshold and / or the target confidence threshold.

[0065] Step S206: Perform image evaluation on the image to be detected to obtain image evaluation results, and if the image evaluation results do not meet the target evaluation index threshold, calculate the image confidence score based on the image evaluation results.

[0066] The above-mentioned process updates the evaluation index threshold and confidence threshold when the image classification parameters are within the preset parameter range to obtain the target evaluation index threshold and target confidence threshold. In this step, the image to be detected is first evaluated to obtain the image evaluation result. If the image evaluation result does not meet the target evaluation index threshold, the image confidence is calculated with the help of the image evaluation result of the image to be detected.

[0067] The image confidence level described in this embodiment includes the probability or degree to which the image detection of the image to be detected passes. The image evaluation result not meeting the target evaluation index threshold includes an image quality score less than a preset score threshold, a text confidence level less than a preset confidence threshold, and / or an image forgery index less than a preset index threshold.

[0068] In specific implementation, during the process of obtaining image evaluation results for the image to be detected, the following operations can be performed: image quality assessment of the image to be detected to obtain an image quality score; and / or, text parsing of the image to be detected, and calculation of text confidence based on each identified text block; and / or, calculation of the forgery score of the image to be detected; wherein, the image quality score includes clarity score, integrity score, etc.; the forgery index includes a probability or score representing that the image to be detected is a forged image. The text confidence includes the confidence of the image to be detected from a textual perspective, such as identifying textual information (identity card number, address information, etc.) from the image to be detected, and calculating the confidence of the image to be detected based on each identified textual information. Specifically, if the textual information does not match, the text confidence of the image to be detected can be reduced; if the textual information matches, the text confidence of the image to be detected can be increased.

[0069] In one optional implementation of this embodiment, after performing image evaluation on the image to be detected to obtain the image evaluation result, the following operation is also performed: if the image evaluation result meets the target evaluation index threshold, the image detection result of the image to be detected is determined to be detected as passed.

[0070] The image evaluation result meeting the target evaluation index threshold includes: an image quality score greater than or equal to a preset score threshold, a text confidence score greater than or equal to a preset confidence threshold, and / or an image forgery index greater than or equal to a preset index threshold. In other words, if any one or more of the following conditions are met: an image quality score greater than or equal to a preset score threshold, a text confidence score greater than or equal to a preset confidence threshold, or an image forgery index greater than or equal to a preset index threshold, the image evaluation result is determined to meet the target evaluation index threshold.

[0071] In the specific execution process, in order to calculate the image confidence score with finer granularity, this embodiment provides an optional implementation method in which the following operations are performed during the calculation of the image confidence score based on the image evaluation results:

[0072] Determine the distribution of evaluation metrics for each image and / or the difference in evaluation metrics between any two images in each image;

[0073] The image confidence level is calculated based on the distribution of the evaluation indicators and / or the difference between the evaluation indicators.

[0074] Optionally, the evaluation index distribution includes at least one of the following: image quality score distribution, text confidence distribution, and image forgery score distribution.

[0075] Optionally, the evaluation index difference includes at least one of the following: image quality score difference, text confidence score difference, and image forgery score difference.

[0076] Specifically, the distribution of evaluation indicators for each image can be determined, and the difference in evaluation indicators between every two images can be calculated based on the distribution of evaluation indicators for each image. If the difference in evaluation indicators exceeds a preset difference threshold, the image confidence level is determined as a first confidence level; if the difference in evaluation indicators does not exceed the preset difference threshold, the image confidence level is determined as a second confidence level. Optionally, the second confidence level is less than the first confidence level. In addition, during the process of calculating the image confidence level based on the evaluation indicator scores and / or the difference in evaluation indicators, the difference in evaluation indicators between every two images can also be calculated based on the distribution of evaluation indicators for each image, and the corresponding confidence level can be determined based on the calculated difference in evaluation indicators.

[0077] In the specific execution process, in order to improve the effectiveness and comprehensiveness of the calculated image confidence, the image confidence can be calculated based on multimodal features; in another optional implementation provided in this embodiment, in the process of calculating the image confidence based on the image evaluation results, the following operation is performed: calculate the image confidence based on each image and the text recognition results of each image.

[0078] Specifically, image confidence can be calculated based on the image features of each image and the text recognition results of each image; that is, image confidence is calculated from both image and text modalities to improve the effectiveness of image confidence.

[0079] Furthermore, in the process of calculating image confidence based on image evaluation results, the image confidence can be calculated based on the image features of each image and / or the text recognition results of each image. Specifically, the process of calculating image confidence based on the image features of each image is similar to the above. Based on the image feature maps of each image, the first feature residual in the time dimension and the second feature residual in the spatial dimension can be calculated, and the image confidence can be calculated based on the first feature residual and the second feature residual. The above refers to image classification parameters, and here it refers to image confidence. Other processing procedures are similar and can be referred to for further reading. They will not be repeated here.

[0080] In the process of calculating the image confidence score based on the image features and / or the text recognition results of each image, a first confidence score can be calculated based on the image features of each image and / or a second confidence score can be calculated based on the text recognition results of each image. The image confidence score is then calculated based on the first confidence score and / or the second confidence score. The calculation of the first confidence score based on the image features of each image is similar to the process of calculating image classification parameters based on the image features of each image described above, and will not be repeated here. In the process of calculating the second confidence score based on the text recognition results of each image, the matching degree between the text recognition information of each image can be used as the second confidence score. The content of calculating the confidence score based on the text recognition results of each image in this embodiment can refer to the implementation process described here. Calculating confidence scores using data from both image and text modalities (i.e., multimodal data) improves the comprehensiveness and effectiveness of the confidence score calculation.

[0081] Step S208: Based on the image confidence level and the target confidence level threshold, determine the image detection result of the image to be detected.

[0082] After obtaining the image confidence level as described above, this step uses the image confidence level and the target confidence level threshold to determine the image detection result of the image to be detected.

[0083] In one optional implementation of this embodiment, the following operations are performed during the process of determining the image detection result of the image to be detected based on the image confidence score and the target confidence threshold:

[0084] If the image confidence score is greater than or equal to the target confidence threshold, the image detection result is determined to be a successful detection.

[0085] If the image confidence score is less than the target confidence score threshold, the image detection result is determined to be a failed detection.

[0086] It should be added that steps S202 to S206 in this embodiment can be replaced by: acquiring the image to be detected, performing image evaluation on the image to be detected to obtain image evaluation results; if the image evaluation results do not meet the evaluation index threshold (the current evaluation index threshold), reading the associated images of the image to be detected from the database based on the user identifier and / or device identifier of the uploaded image to be detected; constructing an image set based on the image to be detected and the associated images to obtain a target image set; correspondingly, step S208 can be replaced by: calculating the image confidence based on the image features of each image in the target image set, or calculating the image confidence based on the target image set, or calculating the image confidence based on each image and the text recognition results of each image, or calculating the image confidence based on the evaluation index distribution of each image and / or the evaluation index difference between every two images in each image; after this, if the calculated image confidence is greater than or equal to the confidence threshold, the image detection result of the image to be detected is determined to be detected as passed; if the calculated image confidence is less than the confidence threshold, the image detection result of the image to be detected is determined to be detected as failed; and forming a new implementation method with other processing steps provided in this embodiment;

[0087] Alternatively, step S206 can be replaced by performing image evaluation on the image to be detected to obtain an image evaluation result, updating the target confidence threshold based on the image evaluation result to obtain an updated confidence threshold (the updated confidence threshold), and calculating the image confidence based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold. Correspondingly, step S208 can be replaced by determining the image detection result of the image to be detected based on the image confidence and the updated confidence threshold. This, along with other processing steps provided in this embodiment, forms a new implementation. The process of updating the target confidence threshold based on the image evaluation result can be implemented by lowering the target confidence threshold if the image evaluation result meets the target evaluation index threshold, and not processing it if the image evaluation result does not meet the target evaluation index threshold.

[0088] Alternatively, step S204 can be replaced by updating the evaluation index threshold if the image classification parameters are within a preset parameter range to obtain a target evaluation index threshold; step S206 can be replaced by performing image evaluation on the image to be detected to obtain an image evaluation result, updating the confidence threshold based on the image evaluation result to obtain a target confidence threshold, and calculating the image confidence based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold; and forming a new implementation method with other processing steps provided in this embodiment; wherein the process of updating the confidence threshold based on the image evaluation result can be implemented by lowering the confidence threshold if the image evaluation result meets the target evaluation index threshold, and not processing if the image evaluation result does not meet the target evaluation index threshold.

[0089] Alternatively, step S204 can be replaced by updating the confidence threshold if the image classification parameters are within a preset parameter range to obtain the target confidence threshold; step S206 can be replaced by performing image evaluation on the image to be detected to obtain an image evaluation result, and calculating the image confidence based on the image evaluation result if the image evaluation result does not meet the evaluation index threshold; and forming a new implementation method with other processing steps provided in this embodiment.

[0090] Alternatively, step S204 can be replaced by updating the confidence threshold to obtain a target confidence threshold if the image classification parameters are within a preset parameter range; step S206 can be replaced by performing image evaluation on the image to be detected to obtain an image evaluation result, updating the target confidence threshold based on the image evaluation result to obtain an updated confidence threshold, and calculating the image confidence based on the image evaluation result if the image evaluation result does not meet the evaluation index threshold; step S208 can be replaced by determining the image detection result of the image to be detected based on the image confidence and the updated confidence threshold; and forming a new implementation method with other processing steps provided in this embodiment. The process of updating the target confidence threshold based on the image evaluation result can be implemented by lowering the confidence threshold if the image evaluation result meets the evaluation index threshold, and not processing it if the image evaluation result does not meet the evaluation index threshold.

[0091] The aforementioned process of updating the target confidence threshold based on the image evaluation result can be implemented by lowering the target confidence threshold if the image evaluation result meets the target evaluation index threshold, and not processing it if the image evaluation result does not meet the target evaluation index threshold. It should be noted that the process of updating the confidence threshold based on the image evaluation result is similar to the process of updating the target confidence threshold and the process of updating the target confidence threshold based on the image evaluation result, and will not be described again here.

[0092] In summary, the image processing method provided in this embodiment first acquires the image to be detected, reads the associated images of the image to be detected from the database based on the user identifier and / or device identifier of the uploaded image to be detected, constructs an image set based on the image to be detected and the associated images, and calculates image classification parameters based on the image features of each image in the constructed target image set.

[0093] Secondly, if the image classification parameters are within the preset parameter range, the evaluation index threshold and confidence threshold are updated to obtain the target evaluation index threshold and target confidence threshold. The image to be detected is then evaluated to obtain the image evaluation result. If the image evaluation result does not meet the target evaluation index threshold, the image confidence is calculated based on the image evaluation result. If the image confidence is greater than or equal to the target confidence threshold, the image detection result is determined to be a successful detection; if the image confidence is less than the target confidence threshold, the image detection result is determined to be a failed detection. In this way, the evaluation index threshold and confidence threshold are updated using image classification parameters to improve the accuracy of image detection. Combining the image to be detected with related images enhances the comprehensiveness and flexibility of image detection.

[0094] The following description uses the application of an image processing method provided in this embodiment in an identity credential scenario as an example to further illustrate the image processing method provided in this embodiment. (See also...) Figure 3 The image processing method applied to identity credential scenarios includes the following steps.

[0095] Step S302: Obtain the identity credential image and, based on the device identifier of the uploaded identity credential image, read the associated identity credential image from the database.

[0096] Step S304: Construct an image set based on the identity credential image and the associated identity credential image, and calculate image classification parameters based on the image features of each image in the constructed target image set.

[0097] Step S306: If the image classification parameters are within the preset parameter range, update the evaluation index threshold and confidence threshold to obtain the target evaluation index threshold and target confidence threshold.

[0098] Step S308: Perform image evaluation on the identity credential image to obtain the image evaluation result.

[0099] Step S310: If the image evaluation results do not meet the target evaluation index threshold, determine the distribution of evaluation indexes for each image and the difference in evaluation indexes between every two images.

[0100] Step S312: Calculate the image confidence level based on the distribution of evaluation indicators and the difference between evaluation indicators.

[0101] Step S314: If the image confidence level is greater than or equal to the target confidence level threshold, the image detection result of the identity credential image is determined to be a successful detection.

[0102] The above step S314 can be replaced by determining that if the image confidence is less than the target confidence threshold, the image detection result is that the detection failed, and forming a new implementation method with other processing steps provided in this embodiment.

[0103] This specification provides an embodiment of an image processing device as follows:

[0104] In the above embodiments, an image processing method is provided, and correspondingly, an image processing apparatus is also provided, which will be described below with reference to the accompanying drawings.

[0105] Reference Figure 4 This illustration shows a schematic diagram of an embodiment of an image processing apparatus provided in this embodiment.

[0106] Since the apparatus embodiments correspond to the method embodiments, the descriptions are relatively simple. For relevant parts, please refer to the corresponding descriptions of the method embodiments provided above. The apparatus embodiments described below are merely illustrative.

[0107] This embodiment provides an image processing apparatus, including:

[0108] The parameter calculation module 402 is configured to acquire the image to be detected and associated images and construct an image set, and calculate image classification parameters based on the image features of each image in the constructed target image set.

[0109] The threshold update module 404 is configured to update the evaluation index threshold and confidence threshold if the image classification parameters are within a preset parameter range, so as to obtain the target evaluation index threshold and the target confidence threshold.

[0110] The confidence calculation module 406 is configured to perform image evaluation on the image to be detected to obtain an image evaluation result, and calculate the image confidence based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold.

[0111] The result determination module 408 is configured to determine the image detection result of the image to be detected based on the image confidence and the target confidence threshold.

[0112] This specification provides an embodiment of an image processing device as follows:

[0113] Corresponding to the image processing method described above, based on the same technical concept, one or more embodiments of this specification also provide an image processing apparatus for performing the image processing method provided above. Figure 5 This is a schematic diagram of the structure of an image processing device provided for one or more embodiments of this specification.

[0114] This embodiment provides an image processing device, including:

[0115] like Figure 5 As shown, image processing devices can vary considerably due to differences in configuration or performance. They may include one or more processors 501 and memory 502, with memory 502 storing one or more application programs or data. Memory 502 can be temporary or persistent storage. The application programs stored in memory 502 may include one or more modules (not shown), each module including a series of computer-executable instructions from the image processing device. Furthermore, processor 501 may be configured to communicate with memory 502, executing the series of computer-executable instructions stored in memory 502 on the image processing device. The image processing device may also include one or more power supplies 503, one or more wired or wireless network interfaces 504, one or more input / output interfaces 505, one or more keyboards 506, etc.

[0116] In one specific embodiment, the image processing apparatus includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the image processing apparatus, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0117] The image to be detected and associated images are acquired and an image set is constructed. Based on the image features of each image in the constructed target image set, image classification parameters are calculated.

[0118] If the image classification parameters are within the preset parameter range, the evaluation index threshold and confidence threshold are updated to obtain the target evaluation index threshold and target confidence threshold.

[0119] The image to be detected is evaluated to obtain an image evaluation result, and if the image evaluation result does not meet the target evaluation index threshold, the image confidence score is calculated based on the image evaluation result.

[0120] Based on the image confidence level and the target confidence threshold, the image detection result of the image to be detected is determined.

[0121] This specification provides an example of a storage medium as follows:

[0122] Corresponding to the image processing method described above, based on the same technical concept, one or more embodiments of this specification also provide a storage medium.

[0123] The storage medium provided in this embodiment is used to store computer-executable instructions, which, when executed by a processor, implement the following process:

[0124] The image to be detected and associated images are acquired and an image set is constructed. Based on the image features of each image in the constructed target image set, image classification parameters are calculated.

[0125] If the image classification parameters are within the preset parameter range, the evaluation index threshold and confidence threshold are updated to obtain the target evaluation index threshold and target confidence threshold.

[0126] The image to be detected is evaluated to obtain an image evaluation result, and if the image evaluation result does not meet the target evaluation index threshold, the image confidence score is calculated based on the image evaluation result.

[0127] Based on the image confidence level and the target confidence threshold, the image detection result of the image to be detected is determined.

[0128] It should be noted that the embodiments of a storage medium and the embodiments of an image processing method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding method described above, and the repeated parts will not be described again.

[0129] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments. For example, the device embodiment, equipment embodiment, and storage medium embodiment are all similar to the method embodiment, so the description is relatively simple. For reading the relevant content of the device embodiment, equipment embodiment, and storage medium embodiment, please refer to the description of the method embodiment.

[0130] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0131] In the 1930s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement to the methodology cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0132] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0133] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0134] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0135] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0140] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0141] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0142] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0143] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0144] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0145] The above description is merely an embodiment of this document and is not intended to limit the scope of this document. Various modifications and variations can be made to this document by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this document should be included within the scope of the claims of this document.

Claims

1. An image processing method, comprising: The process involves acquiring the image to be detected and associated images, constructing an image set, and calculating image classification parameters based on the image features of each image in the constructed target image set. The associated images include historical images of users that have not passed detection under the target service. The target service includes any service that requires image detection of the image to be detected. The image classification parameters include parameters characterizing the image classification of the target image set, such as a score or confidence level. The image features include feature maps of each image in at least one dimension, including one or more of the following: quality dimension, text dimension, and forgery dimension. If the image classification parameters are within the preset parameter range, the evaluation index threshold and confidence threshold are updated to obtain the target evaluation index threshold and target confidence threshold. The image to be detected is evaluated to obtain an image evaluation result, and if the image evaluation result does not meet the target evaluation index threshold, the image confidence score is calculated based on the image evaluation result. Based on the image confidence level and the target confidence threshold, the image detection result of the image to be detected is determined.

2. The method according to claim 1, wherein calculating the image confidence score based on the image evaluation result comprises: Determine the distribution of evaluation metrics for each image and / or the difference in evaluation metrics between any two images in each image; The image confidence level is calculated based on the distribution of the evaluation indicators and / or the difference between the evaluation indicators.

3. The method according to claim 2, wherein the distribution of the evaluation index includes at least one of the following: Image quality score distribution, text confidence distribution, and image forgery index distribution.

4. The method according to claim 1, wherein calculating image classification parameters based on image features of each image in the constructed target image set includes: Based on the image feature maps of each image, calculate the first feature residual in the time dimension and the second feature residual in the spatial dimension; Based on the first feature residual and the second feature residual, calculate the image classification parameters of the target image set.

5. The method according to claim 4, wherein calculating the first feature residual in the time dimension and the second feature residual in the spatial dimension based on the image feature maps of each image comprises: Target feature blocks are extracted from the image feature maps of each image in chronological order, and the first feature residual is calculated based on the target feature blocks; as well as, The second feature residual is calculated based on the image feature maps of each image.

6. The method according to claim 1, wherein the associated image is obtained in the following manner: Determine the user identifier and / or device identifier of the user who uploaded the image to be detected; Based on the user identifier and / or the device identifier, read the associated image of the image to be detected from the database.

7. The method according to claim 6, wherein the database stores associated images corresponding to the user identifier and / or the device identifier; If the number of associated images corresponding to the user identifier and / or the device identifier exceeds the image number threshold, the target images in the associated images are removed in chronological order.

8. The method according to claim 1, wherein the image set construction comprises: An image set is constructed based on the image to be detected and the associated image, and it is detected whether the number of images in the image set is less than a preset number of images; If not, the image set shall be used as the target image set.

9. The method according to claim 8, if the result of the operation of detecting whether the number of images in the image set is less than a preset number of images is yes, the following operation is performed: The image set is filled with images based on the number of images and the preset number of images, and the image set after image filling is used as the target image set.

10. The method according to claim 1, wherein calculating the image confidence score based on the image evaluation result comprises: The confidence level of each image is calculated based on the image and the text recognition results of each image.

11. The method according to claim 1, wherein determining the image detection result of the image to be detected based on the image confidence level and the target confidence threshold comprises: If the image confidence score is greater than or equal to the target confidence threshold, the image detection result is determined to be a successful detection. If the image confidence score is less than the target confidence score threshold, the image detection result is determined to be a failed detection.

12. The method according to claim 1, further comprising, after performing the image evaluation operation on the image to be detected to obtain the image evaluation result: If the image evaluation result meets the target evaluation index threshold, the image detection result of the image to be detected is determined to be a successful detection.

13. The method according to claim 1, wherein updating the evaluation index threshold and confidence threshold includes: The confidence threshold, the quality score threshold, the text confidence threshold, and / or the image forgery index threshold in the evaluation index threshold are adjusted downwards.

14. An image processing apparatus, comprising: The parameter calculation module is configured to acquire the image to be detected and associated images and construct an image set. Based on the image features of each image in the constructed target image set, it calculates image classification parameters. The associated images include historical images of users that have not passed detection under the target service. The target service includes any service that requires image detection of the image to be detected. The image classification parameters include parameters characterizing the image classification of the target image set. The image classification parameters include a score characterizing the image classification or a confidence level characterizing the image classification. The image features include feature maps of each image in at least one dimension. The dimension includes one or more of the following: quality dimension, text dimension, and forgery dimension. The threshold update module is configured to update the evaluation index threshold and confidence threshold if the image classification parameters are within a preset parameter range, so as to obtain the target evaluation index threshold and the target confidence threshold. The confidence calculation module is configured to perform image evaluation on the image to be detected to obtain an image evaluation result, and calculate the image confidence based on the image evaluation result if the image evaluation result does not meet the target evaluation index threshold. The result determination module is configured to determine the image detection result of the image to be detected based on the image confidence and the target confidence threshold.

15. An image processing apparatus, comprising: processor; And, a memory configured to store computer-executable instructions, which, when executed, cause the processor to: The process involves acquiring the image to be detected and associated images, constructing an image set, and calculating image classification parameters based on the image features of each image in the constructed target image set. The associated images include historical images of users that have not passed detection under the target service. The target service includes any service that requires image detection of the image to be detected. The image classification parameters include parameters characterizing the image classification of the target image set, such as a score or confidence level. The image features include feature maps of each image in at least one dimension, including one or more of the following: quality dimension, text dimension, and forgery dimension. If the image classification parameters are within the preset parameter range, the evaluation index threshold and confidence threshold are updated to obtain the target evaluation index threshold and target confidence threshold. The image to be detected is evaluated to obtain an image evaluation result, and if the image evaluation result does not meet the target evaluation index threshold, the image confidence score is calculated based on the image evaluation result. Based on the image confidence level and the target confidence threshold, the image detection result of the image to be detected is determined.

Images