Image processing method and apparatus

Abstract

Embodiments of the present specification provide image processing methods and devices, wherein an image processing method comprises: after obtaining an application image for project application and a content classification result sent by an upstream node of the project, performing key image block detection on the application image based on the project, extracting image features from the obtained key image blocks, and calculating the image definition of the application image according to the image features; if the content classification result is empty, determining the image content classification of the key image blocks, and if the image content classification is a project adaptive classification, determining the image quality parameter of the application image based on the image definition, and sending to a downstream node of the project for project application processing of the project based on the image quality parameter.

Description

TECHNICAL FIELD

[0001] The present document relates to the technical field of data processing, and particularly relates to an image processing method and device. BACKGROUND

[0002] With the rapid development of Internet technology, project applications of various projects gradually transfer from offline to online, and users do not need to go to offline project institutions to apply for projects. In the process of online project application of various projects, image quality needs to be evaluated. For image quality evaluation, it is divided into objective evaluation and subjective evaluation. Subjective evaluation refers to evaluating image quality according to subjective perception of viewers. Objective evaluation refers to calculating a quantitative value of image quality. For example, a mathematical model can be introduced to calculate the quantitative value of image quality. In this process, how to better realize image processing becomes the focus of all parties. SUMMARY

[0003] One or more embodiments of the present specification provide an image processing method, comprising: acquiring an application image for project application sent by an upstream node of a project, and a content classification result of the application image. Performing key image block detection on the application image based on the project to obtain a key image block. Extracting an image feature from the key image block, and calculating an image definition of the application image based on the image feature. If the content classification result is empty, determining an image content classification of the key image block based on a recognition result obtained by performing image content recognition on the key image block. In the case that the image content classification is a project adaptation classification, determining an image quality parameter of the application image based on the image definition, and sending to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

[0004] One or more embodiments of the present specification provide an image processing device, comprising: an application image acquisition module configured to acquire an application image for project application sent by an upstream node of a project, and a content classification result of the application image. An image block detection module configured to perform key image block detection on the application image based on the project to obtain a key image block. A definition calculation module configured to extract an image feature from the key image block, and calculate an image definition of the application image based on the image feature. If the content classification result is empty, a content classification determination module is run. The content classification determination module is configured to determine an image content classification of the key image block based on a recognition result obtained by performing image content recognition on the key image block. A quality parameter determination module configured to, in the case that the image content classification is a project adaptation classification, determine an image quality parameter of the application image based on the image definition, and send to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

[0005] One or more embodiments of the present specification provide an image processing device, comprising: a processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: acquire an application image for project application sent by an upstream node of a project, and a content classification result of the application image. Perform key image block detection on the application image based on the project to obtain a key image block. Extract an image feature from the key image block, and calculate an image sharpness of the application image based on the image feature. If the content classification result is empty, determine an image content classification of the key image block based on a recognition result obtained by performing image content recognition on the key image block. In the case where the image content classification is a project adaptation classification, determine an image quality parameter of the application image based on the image sharpness, and send to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

[0006] One or more embodiments of the present specification provide a storage medium for storing computer executable instructions, which, when executed by a processor, implement the following processes: acquiring an application image for project application sent by an upstream node of a project, and a content classification result of the application image. Perform key image block detection on the application image based on the project to obtain a key image block. Extract an image feature from the key image block, and calculate an image sharpness of the application image based on the image feature. If the content classification result is empty, determine an image content classification of the key image block based on a recognition result obtained by performing image content recognition on the key image block. In the case where the image content classification is a project adaptation classification, determine an image quality parameter of the application image based on the image sharpness, and send to a downstream node of the project to perform project application processing of the project based on the image quality parameter. BRIEF DESCRIPTION OF DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the one or more embodiments of the present specification or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present specification, and those skilled in the art can also obtain other drawings according to these drawings without paying creative labor;

[0008] Figure 1 A schematic diagram of an image processing method implementation environment provided by one or more embodiments of the present specification;

[0009] Figure 2A flowchart of an image processing method provided by one or more embodiments of the present specification;

[0010] Figure 3 A flowchart of an image processing method applied to a project security scenario provided by one or more embodiments of the present specification;

[0011] Figure 4 A schematic diagram of an embodiment of an image processing device provided by one or more embodiments of the present specification;

[0012] Figure 5 A structural schematic diagram of an image processing device provided by one or more embodiments of the present specification. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present specification, the technical solutions in one or more embodiments of the present specification will be described clearly and completely below with reference to the drawings in one or more embodiments of the present specification. Obviously, the described embodiments are only a part of the embodiments of the present specification, rather than all the embodiments. Based on one or more embodiments of the present specification, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the present document.

[0014] Referring to Figure 1 , a schematic diagram of an implementation environment of an image processing method provided by one or more embodiments of the present specification.

[0015] The image processing method provided by one or more embodiments of the present specification can be applied to a project application implementation environment, which at least includes an image processing system 101. In addition, the implementation environment can also include an upstream node 102, or at least a downstream node 103.

[0016] The image processing system 101 can correspond to a server, or a server cluster composed of several servers, or one or more cloud servers in a cloud computing platform, for image processing in the project application process.

[0017] The upstream node 102 can correspond to a server, or a server cluster composed of several servers, or one or more cloud servers in a cloud computing platform; the downstream node 103 can also correspond to a server, or a server cluster composed of several servers, or one or more cloud servers in a cloud computing platform.

[0018] In the implementation environment, after obtaining the application image for project application sent by the upstream node 102 of the project and the content classification result of the application image, the image processing system 101 can detect a key image block from the application image based on the project, and calculate the image definition of the application image according to the image features extracted from the key image block; in the case that the content classification result is empty, the image content classification of the key image block is determined, and in the case that the image content classification is the project adaptation classification, the image quality parameter of the application image is determined according to the image definition and is sent to the downstream node 103 of the project, so that the downstream node 103 performs project application processing of the project based on the image quality parameter, thereby improving the image quality of the application image and the convenience and effectiveness of the downstream node in performing project application processing.

[0019] One or more embodiments of the image processing method provided in the specification are as follows:

[0020] With reference to Figure 2 The image processing method provided in the embodiment specifically includes steps S202 to S210.

[0021] In step S202, an application image for project application sent by an upstream node of the project is obtained, and a content classification result of the application image is obtained.

[0022] The project in the embodiment includes a related project that needs image quality evaluation, such as a guarantee project (health guarantee project, asset guarantee project, vehicle guarantee project), an examination project, a driving test project, a medical insurance certificate application project, etc.; the upstream node includes an upstream processing node of the project, such as a guarantee project upstream processing node in the process of guarantee application of the guarantee project. The project application includes an application for some services in the project, such as a guarantee application in the guarantee project, and the specific guarantee application can be a claim. The embodiment can be applied to an image processing system, and the order of the execution nodes of the project can be an upstream node, an image processing system, and a downstream node.

[0023] The application image includes an image uploaded by a user for project application, such as a claim image for claim, and specifically, a claim material image photographed by a user in the process of claim in an insurance project; the content classification result includes a classification result of classifying the application image according to the content of the application image; the content classification result can be empty or not empty.

[0024] In actual application, in order to improve the convenience of image processing, the upstream node of the project can perform rough classification on the application image, and for this purpose, the application image for project application sent by the upstream node of the project can be obtained, and the content classification result of the application image. In addition, the application image can also not be roughly classified by the upstream node of the project, so step S202 can be replaced by obtaining the application image for project application sent by the upstream node of the project, and other processing steps provided in the embodiment to form a new implementation mode.

[0025] In step S204, the key image block of the application image is detected based on the project, and the key image block is obtained.

[0026] The application image for project application sent by the upstream node of the project and the content classification result of the application image are obtained in the above step. In this step, in order to improve the accuracy of image quality evaluation of the application image and avoid the interference of information irrelevant to the project in the application image on the image quality evaluation of the application image, the key image block of the application image can be detected based on the project to obtain the key image block.

[0027] The key image block described in the embodiment includes the image block related to the project in the application image. The key image block detection includes detecting the image block related to the project in the application image, such as detecting the image block related to the guarantee application (claim) in the application image, that is, performing background processing on the application image based on the project.

[0028] In specific implementation, in order to avoid the interference of information irrelevant to the project in the application image on the image quality evaluation, the key image block can be detected from the application image, and then the key image block is processed to improve the processing efficiency and processing accuracy. In the first optional implementation provided in the embodiment, in the process of detecting the key image block of the application image based on the project to obtain the key image block, the following operations are performed:

[0029] The key image block of the application image is detected based on the project to obtain the boundary information of a plurality of image blocks in the application image and the corresponding confidence.

[0030] According to the confidence, the candidate boundary information is screened from the plurality of boundary information, and the key image block is extracted from the application image based on the candidate boundary information.

[0031] The boundary information of the plurality of image blocks includes the boundary box of the plurality of image blocks or the coordinate information of the boundary box of the plurality of image blocks. The confidence refers to an index representing the confidence degree of the boundary information of the plurality of image blocks. The plurality of boundary information refers to the boundary information of the plurality of image blocks.

[0032] Specifically, based on the project, key image block detection is performed on the application image to obtain coordinate information of a plurality of image blocks in the application image, and confidence of the plurality of coordinate information is calculated. The plurality of coordinate information is sorted in descending order according to the confidence, and candidate coordinate information before a target position in the sorting result is filtered. The key image block is extracted from the application image according to the candidate coordinate information.

[0033] In the process of extracting the key image block from the application image based on the candidate boundary information, in an optional embodiment of the present embodiment, the following operations are performed:

[0034] Target boundary information with a confidence sorting position at a preset position is determined in the candidate boundary information.

[0035] The image block corresponding to the target boundary information and the image block corresponding to the remaining boundary information are subjected to overlap calculation to obtain an overlap degree.

[0036] Intermediate boundary information is determined in the remaining boundary information based on the overlap degree, and the key image block is extracted from the application image according to the intermediate boundary information and the target boundary information.

[0037] The confidence sorting position includes a sorting position obtained by sorting according to the confidence. The remaining boundary information refers to the boundary information in the candidate boundary information other than the target boundary information. The overlap degree refers to the overlapping degree of the image block corresponding to the target boundary information and the image block corresponding to the remaining boundary information.

[0038] Specifically, target boundary information with a confidence sorting position at a first position is determined in the candidate boundary information. The intersection area and the union area of the image block corresponding to the target boundary information and the image block corresponding to the remaining boundary information are calculated, and the ratio of the intersection area to the union area is calculated. The intermediate boundary information is determined in the remaining boundary information according to the ratio, and the key image block is extracted from the application image according to the intermediate boundary information and the target boundary information.

[0039] In the process of determining the intermediate boundary information from the residual boundary information according to the ratio, the following operation can be performed: if there is first boundary information in the residual boundary information and the ratio of the target boundary information exceeds the ratio threshold value, the first boundary information is deleted from the residual boundary information to obtain target residual boundary information, boundary information with the first confidence ranking position in the target residual boundary information is determined, the overlap degree of the determined boundary information and other boundary information in the target residual boundary information is calculated, if there is target other boundary information in the other boundary information and the overlap degree of the determined boundary information and the target other boundary information is greater than the overlap degree threshold value, the target other boundary information is deleted from the target residual boundary information, and the operation of determining the boundary information with the first confidence ranking position in the target residual boundary information is performed again until the calculated overlap degrees are all greater than the overlap degree threshold value, and the boundary information with the first confidence ranking position screened each time is taken as the intermediate boundary information.

[0040] For example, based on the guarantee project, key image block detection is performed on the application image to obtain the boundary boxes and confidence of 6 image blocks in the application image, the first 4 boundary boxes A, B, C and D are screened from the 6 boundary boxes according to the confidence, the boundary box A with the first confidence ranking position is determined as the target boundary box from the 4 boundary boxes, the overlap degrees between A and B, C and D are calculated respectively, if the overlap degree between B and A is greater than the overlap degree threshold value, B is deleted from the 3 boundary boxes, the boundary box C with the first confidence ranking position is determined from C and D, the overlap degree between D and C is calculated, if the overlap degree is greater than the overlap degree threshold value, D is deleted, and the boundary box C is taken as the intermediate boundary box, and A is taken as the target boundary box, and the key image block is extracted from the application image according to the boundary boxes A and C.

[0041] In addition, in the process of performing key image block detection on the application image based on the project, a target detection model can also be introduced, the application image is input into the target detection model to perform key image block detection based on the project to obtain the key image block in the application image; wherein the target detection model can adopt a CNN (Convolutional Neural Network).

[0042] In addition, in actual application, in the process of performing key image block detection on the application image based on the project, the key image block can not be detected, in this case, the application image can be taken as the key image block to improve the success rate of image processing, in the second optional implementation manner provided in the embodiment, in the process of performing key image block detection on the application image based on the project, the following operation is performed:

[0043] performing key image block detection on the application image based on the project;

[0044] If the detection result is empty, the application image is taken as the key image block; after that, the following step S206 is performed;

[0045] If the detection result is not empty, the following step S206 is performed.

[0046] It should be noted that after the above key image block detection based on the project is performed on the application image to obtain the key image block, the image parameter of the key image block can also be updated, such as cropping and / or scaling the key image block, and then the updated key image block is taken as the key image block for subsequent image feature extraction.

[0047] Step S206, image features are extracted from the key image block, and the image sharpness of the application image is calculated based on the image features.

[0048] The above key image block detection based on the project is performed on the application image to obtain the key image block, in this step, image features are extracted from the key image block, and the image sharpness of the application image is calculated based on the image features.

[0049] The image features described in the embodiment include the image feature vector extracted from the key image block; the image sharpness refers to an index representing the sharpness of the application image.

[0050] In the specific execution process, in order to improve the efficiency and accuracy of feature extraction, a feature extraction model can be introduced, the key image block is input into the feature extraction model for feature extraction to obtain the image features; wherein the feature extraction model can adopt Backbone (main network), and specifically can adopt a split-attention network Split-attention Networks based on a multi-path network Multi-path and a feature map attention Featuremap Attention as Backbone.

[0051] In specific implementation, in order to improve the calculation efficiency and accuracy of calculating the image sharpness, a sharpness calculation model can be introduced, in the first optional implementation provided by the embodiment, in the process of calculating the image sharpness of the application image based on the image features, the following operations are performed:

[0052] The image features are input into the sharpness calculation model for sharpness calculation to obtain the image sharpness.

[0053] In the specific execution process, the sharpness calculation model can be trained in advance, in one optional implementation provided by the embodiment, the sharpness calculation model is trained in the following way:

[0054] input the first image feature in the pair of image feature samples into a first sharpness network in the to-be-trained model to perform sharpness calculation, and input the second image feature into a second sharpness network in the to-be-trained model to perform sharpness calculation;

[0055] calculate a loss value according to a sorting result obtained by sorting the sample sorting result and a sorting result obtained by sorting the first sharpness and the second sharpness, and adjust parameters of the to-be-trained model according to the loss value.

[0056] Optionally, the pair of image feature samples is obtained after key image block detection is performed on a pair of image samples and image feature extraction is performed on a pair of key image blocks obtained by detection; optionally, an image quality level of the first image in the pair of image samples is different from an image quality level of the second image.

[0057] Here, the key image block detection performed on the pair of image samples includes key image block detection performed on the first image and the second image in the pair of image samples, and the implementation process of the key image block detection performed on the first image and the second image is similar to the implementation process of the key image block detection performed on the application image, which will not be described herein again.

[0058] The image quality level of the first image in the pair of image samples can be higher than the image quality level of the second image, or can be lower than the image quality level of the second image; that is, the image sharpness of the first image in the pair of image samples is different from the image sharpness of the second image, and the image sharpness of the first image in the pair of image samples can be greater than the image sharpness of the second image, or can be less than the image sharpness of the second image; in this way, the convenience of model training is improved through the pair of image samples.

[0059] In addition, in order to reduce the number of times of uploading the application image by the user, if the sharpness of a target key image block in each key image block is less than a preset sharpness threshold, the target key image block is subjected to super-resolution processing, and in the second optional implementation manner provided in the embodiment, the following operations are performed in the process of calculating the image sharpness of the application image based on the image features:

[0060] calculate the sharpness of each key image block based on the image features extracted from the key image blocks;

[0061] if the sharpness of a target key image block is less than a preset sharpness threshold, the target key image block is subjected to super-resolution processing;

[0062] calculate the image sharpness of the application image according to the sharpness of the remaining key image blocks and the sharpness of the target key image block subjected to super-resolution processing.

[0063] The remaining key image blocks refer to key image blocks other than the target key image block.

[0064] Specifically, according to the definition of the target key image block obtained by the super-resolution processing and the definition of the remaining key image blocks, the process of calculating the image definition of the application image can be calculated by weighting the definition of the remaining key image blocks, the definition of the target key image block after super-resolution processing and their respective weight values to obtain the image definition of the application image.

[0065] It should be noted that the above implementation process of calculating the image definition of the application image based on the image features can also be replaced by calculating the definition of the key image blocks based on the image features extracted from the key image blocks, performing super-resolution processing on the key image blocks according to the super-resolution level corresponding to the definition of the key image blocks, and calculating the image definition of the application image based on the definition of the key image blocks after super-resolution processing. Optionally, the super-resolution level corresponding to the definition of the key image blocks is negatively correlated with the definition of the key image blocks, that is, the lower the definition, the higher the super-resolution level; the higher the definition, the lower the super-resolution level.

[0066] Or it can also be replaced by calculating the definition of the key image blocks based on the image features extracted from the key image blocks, determining the super-resolution level of the key image blocks according to the definition of the key image blocks and the relevance of the key image blocks to the project, and performing super-resolution processing on the key image blocks according to the super-resolution level of the key image blocks, and calculating the image definition of the application image based on the definition of the key image blocks after super-resolution processing. The relevance refers to the degree of relevance of the key image blocks to the project.

[0067] In addition, in order to realize the pertinence and flexibility of calculating the image definition of the application image, the definition calculation model corresponding to the application image can be determined according to the content classification result of the application image, and the image features are input into the definition calculation model for definition calculation to obtain the image definition. The third optional implementation provided in this embodiment performs the following operations in the process of calculating the image definition of the application image based on the image features:

[0068] According to the content classification result, the definition calculation model corresponding to the application image is determined.

[0069] The image features are input into the definition calculation model for definition calculation to obtain the image definition.

[0070] Specifically, if the content classification result of the application image is the credential classification, the application image is determined to correspond to the credential definition calculation model, the image feature is input into the credential definition calculation model for definition calculation, and the image definition is obtained; if the content classification result of the application image is the list classification, the application image is determined to correspond to the list definition calculation model, the image feature is input into the list definition calculation model for definition calculation, and the image definition is obtained; if the content classification result is empty, the application image is determined to correspond to the target definition calculation model, the image feature is input into the target definition calculation model for definition calculation, and the image definition is obtained.

[0071] In addition, in the process of calculating the image definition of the application image based on the image feature, the image definition of each key image block can be calculated according to the image feature of each key image block.

[0072] In the specific execution process, after the image feature is extracted from the key image block and the image definition of the application image is calculated based on the image feature, in an optional implementation provided by the embodiment, the following operation is further performed:

[0073] If the content classification result is not empty, the image quality parameter of the application image is determined based on the image definition, and the project application processing of the project based on the image quality parameter is sent to the downstream node.

[0074] Specifically, in the case that the content classification result sent by the upstream node is not empty, the content classification of the application image is the project adaptation classification, the image quality parameter of the application image is determined based on the image definition, and the project application processing of the project based on the image quality parameter is sent to the downstream node of the project.

[0075] Step S208, if the content classification result is empty, the image content classification of the key image block is determined based on the recognition result obtained by the image content recognition of the key image block.

[0076] The above extracts the image feature from the key image block and calculates the image definition of the application image based on the image feature; in this step, if the content classification result is empty, the image content classification of the key image block is determined based on the recognition result obtained by the image content recognition of the key image block, if the content classification result is not empty, the image quality parameter of the application image is determined based on the image definition, and the project application processing of the project based on the image quality parameter is sent to the downstream node.

[0077] The identification result described in the embodiment includes an identification result obtained by identifying the image content in the key image block. The image content classification includes a content classification obtained by dividing the image content of the key image block, and the image content classification includes a credential classification, a list classification, and / or other classifications.

[0078] In a specific implementation, if the content classification result is empty, the image content of the key image block is identified to obtain an identification result, and the image content classification of the key image block is determined according to the identification result.

[0079] In a specific implementation process, in an optional implementation provided by the embodiment, after the execution of the operation of determining the image content classification of the key image block based on the identification result obtained by identifying the key image block if the content classification result is empty, the following operations are further performed:

[0080] In the case where the image content classification is not the project adaptation classification, the image quality parameter of the application image is determined based on the image sharpness.

[0081] The determined image quality parameter is marked as a failure of adaptation, and the marked image quality parameter is sent to the downstream node of the project.

[0082] The project adaptation classification refers to an image content classification to which the project is adapted, and the project adaptation classification includes a credential classification and / or a list classification. The image quality parameter refers to a parameter representing the image quality of the application image, such as an image quality score, an image quality level, etc.

[0083] In addition, in the process of calculating the image sharpness of the application image based on the image features, the image sharpness of each key image block can be calculated based on the image features of each key image block. If the content classification result is empty, the image content classification of each key image block can be determined based on the identification result obtained by identifying the image content of each key image block.

[0084] It should be noted that step S202 can be replaced by obtaining the application image for project application sent by the upstream node of the project, and step S208 can be replaced by determining the image content classification of the key image block based on the identification result obtained by identifying the image content of the key image block, and other processing steps provided by the embodiment form a new implementation.

[0085] Step S210, in the case where the image content classification is the project adaptation classification, the image quality parameter of the application image is determined based on the image sharpness, and the image quality parameter is sent to the downstream node of the project to perform the project application processing of the project based on the image quality parameter.

[0086] If the content classification result is empty, the image content classification of the key image block is determined based on the recognition result obtained by performing image content recognition on the key image block. In the case where the image content classification is the project adaptation classification, the image quality parameter of the application image is determined based on the image definition, and is sent to the downstream node of the project, so that the image quality parameter is used by the downstream node to perform the project application processing of the project.

[0087] The image content classification described in the embodiment includes the content classification obtained by dividing the image content of the key image block. The image content classification includes the certificate classification, the list classification, and / or other classifications. The project application processing refers to the relevant application processing of the project, such as the security application processing of the security project.

[0088] In a specific implementation, in order to improve the convenience of the downstream node in performing the project application processing, in an optional implementation provided by the embodiment, the following operation is performed in the process of determining the image quality parameter of the application image based on the image definition:

[0089] If the image definition is greater than the definition threshold, the image quality parameter of the application image is determined to be clear;

[0090] If the image definition is less than or equal to the definition threshold, the image quality parameter of the application image is determined to be blurred;

[0091] Optionally, the definition threshold is obtained from the upstream node or is determined based on the model training result of the definition calculation model.

[0092] In a specific implementation, in an optional implementation provided by the embodiment, the following operation is performed in the process of determining the image quality parameter of the application image based on the image definition and sending the image quality parameter to the downstream node of the project:

[0093] The image quality parameter of the application image is determined according to the image definition;

[0094] An execution action is determined based on the image quality parameter, and it is judged whether the execution action is an application action;

[0095] If yes, the image quality parameter is sent to the downstream node.

[0096] In an optional implementation provided by the embodiment, if the execution result of the operation of judging whether the execution action is an application action is no, the following operation is performed:

[0097] A retry instruction is sent to the upstream node, and the step S202 is returned to be executed.

[0098] Specifically, the process of determining the action to be performed based on the image quality parameter can be implemented in the following manner: if the image quality parameter is clear, determining the action to be performed as an application action; and if the image quality parameter is blurred, determining the action to be performed as a retry action.

[0099] In an optional implementation provided by the embodiment, in the process of performing the project application processing of the project based on the image quality parameter, the following operations are performed:

[0100] If the image quality parameter is the preset quality parameter, performing project verification of the application image under the project, and after the verification is passed, performing the project application for the user;

[0101] If the image quality parameter is not the preset quality parameter, sending a retry reminder to the user, and returning to perform the step S202.

[0102] The preset quality parameter can be clear. The project verification includes verifying the authenticity of the application image. The project application includes performing related application of the project, such as the guarantee application (claim) of the guarantee project.

[0103] In addition, based on the identification result obtained by identifying the image content of each key image block, if the image content classification of the first key image block is the project adaptation classification, the image quality parameter of the key image block is determined based on the image clarity of the key image block, and is sent to the downstream node of the project; if the image quality parameter is the preset quality parameter, the application image is subjected to project verification under the project, and after the verification is passed, the project application is performed for the user; if the image quality parameter is not the preset quality parameter, a retry reminder is sent to the user to obtain the application image corresponding to the key image block uploaded by the user.

[0104] It should be noted that the step S204 can be replaced by detecting the key image block of the application image based on the project to obtain a mask image containing the key image block; the step S206 can be replaced by extracting the image feature from the mask image, and calculating the image clarity of the application image based on the image feature; the step S208 can be replaced by determining the image content classification of the mask image based on the identification result obtained by identifying the image content of the mask image if the content classification result is empty; and the step S210 can be replaced by determining the image quality parameter of the application image based on the image clarity if the image content classification is the project adaptation classification, and sending the image quality parameter to the downstream node of the project to perform the project application processing of the project based on the image quality parameter.

[0105] The key image block detection based on the project is performed on the application image to obtain a mask image containing the key image block, including: performing the key image block detection based on the project on the application image to obtain the key image block, and performing pixel processing on the image blocks other than the key image block in the application image according to a preset pixel value to obtain the mask image. The replacement content of other steps can refer to the execution process in each of the above steps, and the embodiment will not be described here.

[0106] The image processing method provided in the embodiment is further described below by taking the application of the project scene as an example. Figure 3 The image processing method applied to the project scene includes the following steps.

[0107] In step S302, the application image for applying for the guarantee and the content classification result of the application image are obtained from the upstream node of the guarantee project.

[0108] In step S304, the key image block detection based on the project is performed on the application image to obtain the boundary information of the multiple image blocks in the application image and the corresponding confidence.

[0109] In step S306, the candidate boundary information is screened from the multiple boundary information according to the confidence, and the key image block is extracted from the application image based on the candidate boundary information.

[0110] In step S308, the image features are extracted from the key image block, and the definition of the application image is calculated based on the image features.

[0111] In step S310, if the content classification result is empty, the image content classification of the key image block is determined based on the recognition result obtained by the image content recognition on the key image block.

[0112] In step S312, in the case that the image content classification is the project adaptation classification, the image quality parameter of the application image is determined based on the image definition, and the guarantee application processing of the guarantee project based on the image quality parameter is sent to the downstream node of the guarantee project.

[0113] It should be noted that steps S310 to S312 can be replaced by: if the content classification result is not empty, the image quality parameter of the application image is determined based on the image definition, and the guarantee application processing of the guarantee project based on the image quality parameter is sent to the downstream node, and the other processing steps provided in the embodiment form a new implementation mode.

[0114] Further, step S312 can also be replaced by determining the image quality parameter of the application image based on the image sharpness in the case that the image content classification is not the project adaptation classification; marking the determined image quality parameter with a failure of adaptation, and sending the marked image quality parameter to the downstream node, and forming a new implementation mode with other processing steps provided in the embodiment.

[0115] The image processing device provided in the specification implements, for example, the following:

[0116] In the above embodiment, an image processing method is provided, and a corresponding image processing device is also provided, which will be described below with reference to the accompanying drawings.

[0117] Reference Figure 4 which shows a schematic diagram of an embodiment of an image processing device provided in the embodiment.

[0118] Since the device embodiment corresponds to the method embodiment, it is described more simply, and the relevant part can be seen from the above-mentioned corresponding description of the method embodiment. The device embodiment described below is only illustrative.

[0119] The image processing device provided in the embodiment comprises:

[0120] The application image acquisition module 402 is configured to acquire an application image for project application sent by an upstream node of the project, and a content classification result of the application image;

[0121] The image block detection module 404 is configured to perform key image block detection on the application image based on the project to obtain key image blocks;

[0122] The sharpness calculation module 406 is configured to extract image features from the key image blocks, and calculate an image sharpness of the application image based on the image features;

[0123] If the content classification result is empty, the content classification determination module 408 is run, which is configured to determine an image content classification of the key image blocks based on an identification result obtained by performing image content identification on the key image blocks;

[0124] The quality parameter determination module 410 is configured to determine an image quality parameter of the application image based on the image sharpness in the case that the image content classification is the project adaptation classification, and send the image quality parameter to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

[0125] The image processing device provided in the specification implements, for example, the following:

[0126] Corresponding to the above-described image processing method, based on the same technical concept, one or more embodiments of the present specification also provide an image processing device for executing the above-described image processing method, Figure 5 A structural schematic diagram of an image processing device provided by one or more embodiments of the present specification.

[0127] The image processing device provided by the embodiment includes:

[0128] As Figure 5 shown, the image processing device can be quite different due to different configurations or performances, and can include one or more processors 501 and memories 502, and the memories 502 can store one or more storage applications or data. Among them, the memory 502 can be temporary storage or persistent storage. The application stored in the memory 502 can include one or more modules (not shown in the figure), and each module can include a series of computer executable instructions in the image processing device. Further, the processor 501 can be configured to communicate with the memory 502 and execute a series of computer executable instructions in the memory 502 on the image processing device. The image processing device can also include one or more power supplies 503, one or more wired or wireless network interfaces 504, one or more input / output interfaces 505, one or more keyboards 506, etc.

[0129] In one specific embodiment, the image processing device includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and one or more programs can include one or more modules, and each module can include a series of computer executable instructions in the image processing device, and the one or more processors configured to execute the one or more programs include computer executable instructions for:

[0130] Obtaining an application image for applying for a project sent by an upstream node of the project, and a content classification result of the application image;

[0131] Performing key image block detection on the application image based on the project to obtain a key image block;

[0132] Extracting image features from the key image block, and calculating the image definition of the application image based on the image features;

[0133] If the content classification result is empty, determining the image content classification of the key image block based on a recognition result obtained by performing image content recognition on the key image block;

[0134] In a case where the image content classification is the project adaptation classification, an image quality parameter of the application image is determined based on the image definition, and a downstream node of the project is sent to perform project application processing of the project based on the image quality parameter.

[0135] The storage medium provided in the specification implements, for example:

[0136] According to the above description, based on the same technical concept, one or more embodiments of the specification also provide a storage medium.

[0137] The storage medium provided in the embodiment is used to store computer executable instructions, and the computer executable instructions are executed by the processor to implement the following processes:

[0138] An application image for project application sent by an upstream node of the project is obtained, and a content classification result of the application image is obtained;

[0139] The application image is subjected to key image block detection based on the project, and a key image block is obtained;

[0140] Image features are extracted from the key image block, and an image definition of the application image is calculated based on the image features;

[0141] If the content classification result is empty, an image content classification of the key image block is determined based on a recognition result obtained by image content recognition on the key image block;

[0142] In a case where the image content classification is the project adaptation classification, an image quality parameter of the application image is determined based on the image definition, and a downstream node of the project is sent to perform project application processing of the project based on the image quality parameter.

[0143] It should be noted that the embodiments of the storage medium in the specification and the embodiments of the image processing method in the specification are based on the same inventive concept, so the specific implementation of the embodiments can be referred to the foregoing implementation of the corresponding method, and the repeated parts will not be described.

[0144] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments, such as device embodiments, equipment embodiments and storage medium embodiments, which are similar to the method embodiments, so the description is relatively simple. Please refer to the part of the method embodiment for the related content in the device embodiment, the equipment embodiment and the storage medium embodiment.

[0145] The above described embodiments of the present description have been described. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0146] In the 1930s, it was clear to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structure of diodes, transistors, switches, etc.) or in software (e.g., improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, called a hardware description language (HDL), of which there are many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., the most commonly used being VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should be clear to those skilled in the art that, by simply logically programming a method flow in one of the above hardware description languages and programming it into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0147] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, e.g. software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0148] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0149] For the sake of brevity, the above apparatuses are described in functional form in various units. Of course, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0150] Those skilled in the art will appreciate that one or more embodiments of the disclosure can provide a method, a system or a computer program product. Accordingly, one or more embodiments of the disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the disclosure can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer readable program code.

[0151] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0155] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0156] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0157] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0158] One or more embodiments of the specification can be described in the general context of computer-executable instructions being executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the specification can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0159] Each embodiment in the specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0160] The above merely provides the example of the present document and is not intended to limit the present document. For those skilled in the art, the present document can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present document shall be included in the scope of claims of the present document.

Claims

1. An image processing method, comprising: Obtain the application image sent by the upstream node of the project to apply for the project, and the content classification result of the application image; Based on the project, key image blocks are detected in the application image to obtain key image blocks; Image features are extracted from the key image blocks, and the image sharpness of the application image is calculated based on the image features; If the content classification result is empty, the image content classification of the key image block is determined based on the recognition result obtained by performing image content recognition on the key image block; When the image content is classified as a project-adaptive category, the image quality parameters of the application image are determined based on the image clarity, and the application is sent to the downstream node of the project to process the project application based on the image quality parameters.

2. The image processing method according to claim 1, wherein the step of detecting key image blocks in the application image based on the project to obtain key image blocks includes: Based on the project, key image block detection is performed on the application image to obtain the boundary information and corresponding confidence scores of multiple image blocks in the application image; Based on the confidence level, candidate boundary information is filtered from multiple boundary information, and the key image patch is extracted from the application image based on the candidate boundary information.

3. The image processing method according to claim 2, wherein extracting the key image patch from the application image based on the candidate boundary information comprises: Determine the target boundary information whose confidence ranking is in a preset position from the candidate boundary information; The overlap is calculated for the image block corresponding to the target boundary information and the image block corresponding to the remaining boundary information to obtain the overlap. Based on the overlap, intermediate boundary information is determined from the remaining boundary information, and the key image block is extracted from the application image according to the intermediate boundary information and the target boundary information.

4. The image processing method according to claim 1, after the step of determining the image content classification of the key image block based on the recognition result obtained by performing image content recognition on the key image block if the content classification result is empty, further includes: If the image content classification is not a suitable classification for the project, the image quality parameters of the application image are determined based on the image sharpness. The determined image quality parameters are marked as failing to adapt, and the marked image quality parameters are sent to the downstream node.

5. The image processing method according to claim 1, after the step of extracting image features from the key image block and calculating the image sharpness of the application image based on the image features is performed, it further includes: If the content classification result is not empty, the image quality parameters of the application image are determined based on the image clarity, and the application is sent to the downstream node to process the project application based on the image quality parameters.

6. The image processing method according to claim 1, wherein determining the image quality parameters of the application image based on the image sharpness includes: If the image definition is greater than a definition threshold, determining that an image quality parameter of the application image is clear; If the image definition is less than or equal to the definition threshold, determining that the image quality parameter of the application image is blurred; Wherein, the definition threshold is obtained from the upstream node or determined based on a model training result of a definition calculation model.

7. The image processing method of claim 1, wherein the calculating the image definition of the application image based on the image feature comprises: inputting the image feature into a definition calculation model for definition calculation to obtain the image definition; wherein, the definition calculation model is trained in the following manner: inputting a first image feature in an image feature sample pair into a first definition network in a to-be-trained model for definition calculation, and inputting a second image feature into a second definition network in the to-be-trained model for definition calculation; calculating a loss value according to a sample ranking result and a ranking result obtained by ranking a first definition and a second definition calculated from the pair, and adjusting parameters of the to-be-trained model according to the loss value.

8. The image processing method of claim 7, wherein the image feature sample pair is obtained after key image block detection on an image sample pair and image feature extraction on a key image block pair detected. wherein The image quality level of the first image in the image sample pair is different from the image quality level of the second image.

9. The image processing method of claim 1, wherein the performing project application processing of the item based on the image quality parameter comprises: if the image quality parameter is a preset quality parameter, performing project verification of the application image under the item, and if the verification passes, performing project application to the user; if the image quality parameter is not the preset quality parameter, sending a retry reminder to the user.

10. The image processing method of claim 1, wherein the performing key image block detection on the application image based on the item to obtain a key image block comprises: performing key image block detection on the application image based on the item; if the detection result is empty, taking the application image as the key image block.

11. The image processing method of claim 1, wherein the calculating the image definition of the application image based on the image feature comprises: calculating the definition of each key image block based on the image feature extracted from each key image block; if the definition of a target key image block is less than a preset definition threshold, performing super-resolution processing on the target key image block; calculating the image definition of the application image according to the definition of the remaining key image blocks and the definition of the target key image block after super-resolution processing.

12. The image processing method of claim 1, wherein the calculating the image definition of the application image based on the image feature comprises: determining a definition calculation model corresponding to the application image according to the content classification result; inputting the image feature into the definition calculation model for definition calculation to obtain the image definition.

13. The image processing method of claim 1, wherein the determining the image quality parameter of the application image based on the image sharpness and sending to the downstream node of the project comprises: determining the image quality parameter of the application image based on the image sharpness; determining an execution action based on the image quality parameter, and determining whether the execution action is an application action; and if yes, sending the image quality parameter to the downstream node.

14. The image processing method of claim 13, wherein if the execution result of the determining whether the execution action is the application action is no, the following operation is performed: sending a retry instruction to the upstream node, and returning to the step of obtaining the application image for project application sent by the upstream node of the project and the content classification result of the application image.

15. An image processing apparatus, comprising: an application image obtaining module configured to obtain an application image for project application sent by an upstream node of a project and a content classification result of the application image; an image block detecting module configured to perform key image block detection on the application image based on the project to obtain key image blocks; a sharpness calculating module configured to extract image features from the key image blocks and calculate an image sharpness of the application image based on the image features; a content classification determining module configured to, if the content classification result is empty, determine an image content classification of the key image blocks based on a recognition result obtained by performing image content recognition on the key image blocks; and a quality parameter determining module configured to, if the image content classification is a project adaptation classification, determine an image quality parameter of the application image based on the image sharpness and send to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

16. An image processing device, comprising: a processor; and a memory configured to store computer executable instructions that, when executed, cause the processor to: obtain an application image for project application sent by an upstream node of a project and a content classification result of the application image; perform key image block detection on the application image based on the project to obtain key image blocks; extract image features from the key image blocks and calculate an image sharpness of the application image based on the image features; if the content classification result is empty, determine an image content classification of the key image blocks based on a recognition result obtained by performing image content recognition on the key image blocks; and if the image content classification is a project adaptation classification, determine an image quality parameter of the application image based on the image sharpness and send to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

17. A storage medium for storing computer executable instructions that, when executed by a processor, implement the following processes: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ obtain an application image for project application sent by an upstream node of a project, and a content classification result of the application image; perform key image block detection on the application image based on the project, and obtain a key image block; extract an image feature from the key image block, and calculate an image definition of the application image based on the image feature; if the content classification result is empty, determine an image content classification of the key image block based on a recognition result obtained by performing image content recognition on the key image block; in a case where the image content classification is a project adaptation classification, determine an image quality parameter of the application image based on the image definition, and send the image quality parameter to a downstream node of the project to perform project application processing of the project based on the image quality parameter.

Images