Image Processing Method, Apparatus, Device, and Medium

Abstract

The present disclosure provides an image processing method, which can be applied to the fields of artificial intelligence technology and finance. The method includes: in response to a service processing request, obtaining a to-be-processed image carried in the service processing request; segmenting the to-be-processed image to obtain multiple frames of sub-images; using a feature extraction model to process the image data of the multiple frames of sub-images respectively to determine a first target sub-image from the multiple frames of sub-images; and using an image processing model to process the image data of the first target sub-image to obtain a target image. In addition, the present disclosure also provides an image processing apparatus, an electronic device, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology and the financial field, and more particularly, to an image processing method, apparatus, device, and medium. Background Art

[0002] A bank intelligent customer service Q&A system refers to a system for users to ask questions to human customer service or intelligent customer service when they encounter problems during the process of using a bank application for bank-related services. During the user's question-asking process, there is often a situation where it is necessary to take a picture of an entity object to provide an image related to the service. During the shooting process, due to factors such as inaccurate focusing and aberration of the optical system, the captured image may have problems such as more noise and lower resolution, resulting in the system being unable to accurately recognize the image, increasing the ineffective communication time, and reducing the communication efficiency. Summary of the Invention

[0003] In view of this, the present disclosure provides an image processing method, apparatus, electronic device, readable storage medium, and computer program product.

[0004] One aspect of the present disclosure provides an image processing method, including: in response to a service processing request, obtaining a to-be-processed image carried in the service processing request; splitting the to-be-processed image to obtain multiple frames of sub-images; using a feature extraction model to process the image data of multiple frames of the sub-images respectively to determine a first target sub-image from multiple frames of the sub-images; and using an image processing model to process the image data of the first target sub-image to obtain a target image.

[0005] According to an embodiment of the present disclosure, the service processing request further carries a service type; wherein, the using a feature extraction model to process the image data of multiple frames of the sub-images respectively to determine a first target sub-image from multiple frames of the sub-images includes: based on the service type, determining a reference feature vector, where the reference feature vector includes the image data obtained by using the feature extraction model to process a reference image associated with the service type; using the feature extraction model to process the image data of multiple frames of the sub-images respectively to obtain multiple sub-feature vectors; for each sub-feature vector among the multiple sub-feature vectors, determining the similarity between the sub-feature vector and the reference feature vector to obtain multiple similarities; and based on the multiple similarities, determining the first target sub-image from multiple frames of the sub-images.

[0006] According to an embodiment of the present disclosure, the above method further includes: intercepting a second target sub-image from the to-be-processed image based on the above first target sub-image; wherein, processing the image data of the above first target sub-image using the image processing model to obtain the target image includes: processing the image data of the above second target sub-image using the above image processing model to obtain the above target image.

[0007] According to an embodiment of the present disclosure, the above intercepting a second target sub-image from the to-be-processed image based on the above first target sub-image includes: determining the position of the above first target sub-image in the above to-be-processed image to obtain position information; and intercepting the above second target sub-image from the above to-be-processed image based on the above position information.

[0008] According to an embodiment of the present disclosure, the size of the above second target sub-image is larger than the size of the above first target sub-image.

[0009] According to an embodiment of the present disclosure, the above method further includes: processing the image data of the above target image using an image recognition model to obtain the image recognition result of the above to-be-processed image.

[0010] According to an embodiment of the present disclosure, the above image processing model is trained by the following method: obtaining a training sample set, wherein the above training sample set includes a plurality of sample images; for each of the plurality of above sample images, segmenting the above sample image to obtain multiple frames of sub-sample images; using a feature extraction model to process the image data of multiple frames of the above sub-sample images respectively to determine a first sub-sample image from multiple frames of the above sub-sample images; generating a second sub-sample image based on the above first sub-sample image; and using the image data of the above first sub-sample image and the image data of the above second sub-sample image to train an initial image processing model to obtain the above image processing model.

[0011] According to an embodiment of the present disclosure, the above generating a second sub-sample image based on the above first sub-sample image includes: performing a blurring process on the above first sub-sample image to obtain the above second sub-sample image; wherein the above blurring process includes Gaussian blurring and / or motion blurring.

[0012] According to an embodiment of the present disclosure, the above-mentioned initial image processing model includes a generator and a discriminator; training the initial image processing model using the image data of the above-mentioned first sub-sample image and the image data of the above-mentioned second sub-sample image to obtain the above-mentioned image processing model, including: using the above-mentioned generator to process the image data of the above-mentioned second sub-sample image to obtain the image data of the first generated image; using the above-mentioned discriminator to process the image data of the above-mentioned first sub-sample image and the image data of the above-mentioned first generated image to obtain discriminant data; and using the above-mentioned discriminant data to train the above-mentioned generator and the above-mentioned discriminator to obtain the above-mentioned image processing model.

[0013] According to an embodiment of the present disclosure, the above-mentioned discriminator includes a global discriminator and a local discriminator, and the above-mentioned discriminant data includes first discriminant data and second discriminant data; using the above-mentioned discriminator to process the image data of the above-mentioned first sub-sample image and the image data of the above-mentioned first generated image to obtain discriminant data, including: using the above-mentioned global discriminator to process the image data of the above-mentioned first sub-sample image and the image data of the above-mentioned first generated image to obtain the above-mentioned first discriminant data; respectively processing the above-mentioned first sub-sample image and the above-mentioned first generated image based on a predicted local cropping strategy to obtain a third sub-sample image and a second generated image; and using the above-mentioned local discriminator to process the image data of the above-mentioned third sub-sample image and the image data of the above-mentioned second generated image to obtain the above-mentioned second discriminant data.

[0014] According to an embodiment of the present disclosure, the above-mentioned method further includes: obtaining an initial sample image, where the above-mentioned initial sample set includes multiple initial sample images; for each of the multiple above-mentioned initial sample images, performing data augmentation processing on the above-mentioned initial sample image to obtain multiple above-mentioned sample images; where the above-mentioned data augmentation processing includes any one or more of the following: image cropping, image rotation, scale change, color jitter, contrast transformation, and adding noise.

[0015] Another aspect of the present disclosure discloses an image processing device, including: a response module, configured to obtain a to-be-processed image carried in the above-mentioned service processing request in response to a service processing request; a first splitting module, configured to split the above-mentioned to-be-processed image to obtain multiple frames of sub-images; a first processing module, configured to respectively process the image data of multiple frames of the above-mentioned sub-images using a feature extraction model to determine a first target sub-image from multiple frames of the above-mentioned sub-images; a second processing module, configured to process the image data of the above-mentioned first target sub-image using an image processing model to obtain a target image.

[0016] Another aspect of the present disclosure provides an electronic device, including: one or more processors; a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method as described above.

[0017] Another aspect of the present disclosure provides a computer-readable storage medium, on which executable instructions are stored, and when the executable instructions are executed by a processor, the processor implements the method as described above.

[0018] Another aspect of the present disclosure provides a computer program product, which includes computer-executable instructions, and the computer-executable instructions are used to implement the method as described above when being executed.

[0019] According to an embodiment of the present disclosure, by segmenting the image to be processed, multiple frames of sub-images are obtained; a feature extraction model is used to determine a first target sub-image from the multiple frames of sub-images, and then the first target sub-image is processed by a graphics processing model to obtain a target image. Before generating the target image, by adopting the technical means of segmenting the image into multiple frames of sub-images and then determining the target image from the multiple frames of sub-images, the influence of features and noises irrelevant to the target image on the target image generation process is reduced, and the resolution of the target image is effectively improved, thereby at least partially overcoming the technical problem that the images to be recognized in the business system cannot be accurately recognized due to more noises and lower resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Through the following description of the embodiments of the present disclosure with reference to the drawings, the above and other objects, features and advantages of the present disclosure will become clearer. In the drawings:

[0021] Figure 1 Schematically shows an exemplary system architecture to which the image processing method and apparatus according to the embodiments of the present disclosure can be applied;

[0022] Figure 2 Schematically shows a flowchart of the image processing method according to the embodiments of the present disclosure;

[0023] Figure 3 Schematically shows a flowchart of the image processing model training method according to the embodiments of the present disclosure;

[0024] Figure 4 Schematically shows a block diagram of the image processing apparatus according to the embodiments of the present disclosure; and

[0025] Figure 5 Schematically shows a block diagram of an electronic device suitable for implementing the image processing method according to the embodiments of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0027] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0028] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0029] In cases where expressions such as "at least one of A, B, and C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, and C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.). In cases where expressions such as "at least one of A, B, or C, etc." are used, generally, it should be interpreted according to the meaning commonly understood by those of ordinary skill in the art (for example, "a system having at least one of A, B, or C" should include, but not be limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C, etc.).

[0030] Image deblurring refers to repairing a generated blurred image through a certain algorithm and ultimately improving the visual effect of the image. For the application occasion of a given image, the overall and local characteristics of the image are purposefully emphasized, the original unclear image is made clear or certain business characteristics are emphasized, the differences between the characteristics of different objects in the image are enlarged, non-business characteristics are suppressed, the image quality is improved, the information content is enriched, and the image interpretation and recognition effects are enhanced.

[0031] Currently, during the process of user questions, it is often necessary to photograph physical objects and send the captured images to the customer so that the customer can answer based on the images. During the photographing process, due to factors such as inaccurate focusing and aberrations of the optical system, problems such as more noise and lower resolution may occur in the captured images, resulting in the system being unable to accurately recognize the images. In addition, affected by external light and environmental noise, the captured images will also have problems such as more noise and lower resolution. In this case, it is often necessary for the user to take pictures again multiple times, which not only increases the ineffective communication time but also reduces the communication efficiency.

[0032] In view of this, in response to the situation of blurred photographed images, the present disclosure can combine a generative adversarial network, and through an image deblurring model optimized by continuous learning in the early stage, perform deblurring processing on the images uploaded by the user to generate images with high image resolution and high system recognition rate, enhance the actual user experience, avoid repeated operations by the user, and at the same time increase the accuracy of the system in recognizing problems, further improving the solution rate of problems encountered by users during use.

[0033] Specifically, the embodiments of the present disclosure provide an image processing method, apparatus, electronic device, readable storage medium, and computer program product, which can effectively reduce the noise and interference in the image to be recognized, effectively solve the problem of image blurring, and improve the success rate of the image to be recognized. Among them, the image processing method includes: in response to a service processing request, obtaining the image to be processed carried in the service processing request; splitting the image to be processed to obtain multiple frames of sub-images; using a feature extraction model to process the image data of the multiple frames of sub-images respectively to determine a first target sub-image from the multiple frames of sub-images; and using an image processing model to process the image data of the first target sub-image to obtain a target image.

[0034] It should be noted that the image processing method, apparatus, electronic device, storage medium, and program product determined in the embodiments of the present disclosure can be used in the fields of artificial intelligence technology and finance, and can also be used in any field other than the fields of artificial intelligence technology and finance. The specific application fields are not limited.

[0035] In the technical solution of the present disclosure, the processing of the collection, storage, use, processing, transmission, provision, disclosure, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good customs.

[0036] In the technical solution of the present disclosure, before obtaining or collecting the user's personal information, the authorization or consent of the user is obtained.

[0037] Figure 1Schematically illustrated is an exemplary system architecture to which the image processing method and apparatus according to embodiments of the present disclosure can be applied; it should be noted that, Figure 1 The illustration is only an example of the system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0038] As Figure 1 shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0039] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as financial applications, shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software, etc. (only as examples).

[0040] The terminal devices 101, 102, 103 may be various electronic devices having a camera device and / or a display screen, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.

[0041] The server 105 may be a server providing various services, such as a background management server (only as an example) that can support service processing requests carrying images to be processed sent by users using the terminal devices 101, 102, 103. The background management server may analyze and process data such as received service processing requests, and feedback the processing results (for example, the background management server may first perform denoising and deblurring processing on the images to be processed in the service processing requests to obtain clear images) to the terminal devices.

[0042] It should be noted that the image processing method provided by the embodiments of the present disclosure can generally be executed by the server 105. Correspondingly, the image processing apparatus provided by the embodiments of the present disclosure can generally be disposed in the server 105. The image processing method provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103, and / or the server 105. Correspondingly, the image processing apparatus provided by the embodiments of the present disclosure can also be disposed in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103, and / or the server 105. Alternatively, the image processing method provided by the embodiments of the present disclosure can also be executed by the terminal devices 101, 102, or 103, or can also be executed by other terminal devices different from the terminal devices 101, 102, or 103. Correspondingly, the image processing apparatus provided by the embodiments of the present disclosure can also be disposed in the terminal devices 101, 102, or 103, or disposed in other terminal devices different from the terminal devices 101, 102, or 103.

[0043] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0044] Figure 2 is only illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0045] As Figure 2 shown, the method includes operations S201 to S204.

[0046] In operation S201, in response to a service processing request, the to-be-processed image carried in the service processing request is obtained.

[0047] In operation S202, the to-be-processed image is segmented to obtain multiple frames of sub-images.

[0048] In operation S203, the image data of multiple frames of sub-images are respectively processed using a feature extraction model to determine a first target sub-image from the multiple frames of sub-images.

[0049] In operation S204, the image data of the first target sub-image is processed using an image processing model to obtain a target image.

[0050] According to the embodiments of the present disclosure, the to-be-processed image can be an image associated with the service represented by the service processing request. The to-be-processed image can include a to-be-identified object related to the service, and the to-be-identified object can be, for example, a bank card, a debit card, various service documents, etc. Identifying the to-be-identified object can be to identify information such as text, patterns, colors, etc. in the to-be-processed image.

[0051] According to an embodiment of the present disclosure, the manner of segmenting the image to be processed is not limited herein. For example, the image to be processed may be segmented into multiple sub-images of the same size. Specifically, the image to be processed may be evenly divided into M×N rectangular blocks, where M and N are both integers greater than or equal to 0, and each rectangular block represents a sub-image. For another example, the type of the object to be recognized in the image to be processed may be determined first, and segmentation may be performed according to the region segmentation rule corresponding to the type. Specifically, when the image to be processed is an image containing a bank card, the region segmentation rule corresponding to the type of bank card may be adopted to segment the key regions of the bank card. For example, the regions such as the bank card title line, bank card number, bank card logo, and background may be segmented respectively. The obtained multiple sub-images may be sub-images including regions such as the bank card title line, bank card number, bank card logo, and background respectively.

[0052] According to an embodiment of the present disclosure, the feature extraction model may output a feature vector according to the input image data. The feature extraction model may be trained based on any existing feature extraction network architecture. The existing feature extraction network architecture may include but is not limited to MobileNetV2, ShuffleNetV2, PVANet, etc.

[0053] According to an embodiment of the present disclosure, the image processing model may be used to generate a target image according to the input first target sub-image. The image processing model may be any type of image generation model, such as pixelRNN / CNN, GAN (generative adversarial network model), etc.

[0054] According to an embodiment of the present disclosure, by segmenting the image to be processed, multiple sub-images are obtained; the feature extraction model is used to determine the first target sub-image from the multiple sub-images, and then the first target sub-image is processed by the graphics processing model to obtain the target image. Before generating the target image, by adopting the technical means of segmenting the image into multiple sub-images and then determining the target image from the multiple sub-images, the influence of features and noises irrelevant to the target image on the target image generation process is reduced, and the resolution of the target image is effectively improved, thereby at least partially overcoming the technical problem that the image to be recognized in the business system cannot be accurately recognized due to a large amount of noise and low resolution.

[0055] According to an embodiment of the present disclosure, the service type may also be carried in the service processing request.

[0056] According to an embodiment of the present disclosure, the service type may include, for example, transfer, deposit, loan, fee settlement, service agency, etc.

[0057] According to an embodiment of the present disclosure, operation S203 may include the following operations: determining a reference feature vector based on a service type, where the reference feature vector is obtained by processing image data of a reference image associated with the service type using a feature extraction model; processing image data of multiple frames of sub-images respectively using the feature extraction model to obtain multiple sub-feature vectors; for each sub-feature vector among the multiple sub-feature vectors, determining the similarity between the sub-feature vector and the reference feature vector to obtain multiple similarities; and determining a first target sub-image from the multiple frames of sub-images based on the multiple similarities.

[0058] According to an embodiment of the present disclosure, the reference image may be a standard image associated with the service type indicated by the service processing request. For example, when processing a transfer service, it is necessary to identify a bank card number, and the corresponding reference image may be an image including the bank card number and having a high recognition accuracy.

[0059] According to an embodiment of the present disclosure, the similarity may be obtained by calculating the distance between the sub-feature vector and the reference feature vector in the vector space. Specifically, the closer the distance between the sub-feature vector and the reference feature vector in the vector space, the higher the similarity between the sub-feature vector and the reference feature vector can be represented; the farther the distance between the sub-feature vector and the reference feature vector in the vector space, the lower the similarity between the sub-feature vector and the reference feature vector can be represented. Calculating the distance between feature vectors in the vector space can be implemented by methods such as the Euclidean distance algorithm, Manhattan distance algorithm, Mahalanobis distance algorithm, etc., which are not limited herein.

[0060] According to an embodiment of the present disclosure, the similarity may also be calculated by methods such as the Pearson correlation coefficient algorithm, cosine similarity algorithm, etc.

[0061] According to an embodiment of the present disclosure, the first target sub-image may be the sub-image among the multiple frames of sub-images that is closest to the reference image, that is, it can be determined that among the multiple calculated similarities, the sub-image corresponding to the maximum similarity is the first target sub-image.

[0062] According to an embodiment of the present disclosure, through segmentation and similarity calculation, there may be a situation where image information is missing in the determined first target sub-image, or there is more useless image information. Therefore, in some embodiments, before generating the target image, the following operations may further be included: intercepting a second target sub-image from the image to be processed based on the first target sub-image. Wherein, operation S204 may include the following operations: processing image data of the second target sub-image using an image processing model to obtain a target image.

[0063] According to an embodiment of the present disclosure, obtaining a second target sub-image from a to-be-processed image based on a first target sub-image may include the following operations: determining the position of the first target sub-image in the to-be-processed image to obtain position information; and intercepting the second target sub-image from the to-be-processed image based on the position information.

[0064] According to an embodiment of the present disclosure, according to the position information of the first target sub-image, the area around the first target sub-image can be proportionally enlarged or reduced to obtain the position information of the second target sub-image, and then the second target sub-image can be obtained by intercepting. The settings of the enlargement or reduction and the ratio can be adaptively adjusted according to actual needs.

[0065] According to an embodiment of the present disclosure, wherein, the size of the second target sub-image is larger than the size of the first target sub-image.

[0066] According to an embodiment of the present disclosure, specifically, when the to-be-processed image is a bank card and the first target sub-image is a partial UnionPay card number, the second target sub-image can be the entire UnionPay card number; when the first target sub-image is a partial UnionPay logo, the second target sub-image can be the entire UnionPay logo.

[0067] According to an embodiment of the present disclosure, alternatively, the size of the second target sub-image may be smaller than the size of the first target sub-image. For example, when the to-be-processed image is a bank card and the first target sub-image has other invalid information in addition to the entire UnionPay card number, the second target sub-image can be the entire UnionPay card number after removing other invalid information; when the first target sub-image has other invalid information in addition to the entire UnionPay logo, the second target sub-image can be the entire UnionPay logo.

[0068] According to an embodiment of the present disclosure, operation S204 may further include the following operations:

[0069] Processing the image data of the second target sub-image using an image processing model to obtain a target image. That is, using the re-determined sub-image to generate a target image, thereby further improving the recognition probability of the generated image.

[0070] According to an embodiment of the present disclosure, the generated target image can be called by other systems in the business system. For example, the other system is an intelligent customer service Q&A system of a bank, and this system can process the target image to obtain the information required by the business. Specifically, an image recognition model can be used to process the image data of the target image to obtain the image recognition result of the to-be-processed image.

[0071] Figure 3 Schematically shows a flowchart of a method for training an image processing model according to an embodiment of the present disclosure.

[0072] As Figure 3 shown, the method includes operations S301 to S305.

[0073] In operation S301, a training sample set is obtained, where the training sample set includes multiple sample images.

[0074] In operation S302, for each of the multiple sample images, the sample image is segmented to obtain multiple frames of sub-sample images.

[0075] In operation S303, the feature extraction model is used to process the image data of the multiple frames of sub-sample images respectively to determine the first sub-sample image from the multiple frames of sub-sample images.

[0076] In operation S304, a second sub-sample image is generated based on the first sub-sample image.

[0077] In operation S305, the image data of the first sub-sample image and the image data of the second sub-sample image are used to train the initial image processing model to obtain the image processing model.

[0078] According to an embodiment of the present disclosure, the training sample set can be selected according to the actual scenarios of bank and financial industry Q&A customer service. The actual scenarios can include bank card number reading, taking photos of the front and back of an ID card, taking photos of problems encountered by other mobile phones when using an APP, etc. The sample image can be an image associated with the business represented by the business processing request. The sample image can include an object to be recognized related to the business. The object to be recognized can be, for example, a bank card, a debit card, various business documents, etc. Recognizing the object to be recognized can be recognizing information such as text, pattern, color, etc. in the sample image.

[0079] According to an embodiment of the present disclosure, the manner of segmenting the sample image is not limited herein. For example, the sample image can be segmented into multiple frames of sub-images with the same size. Specifically, the sample image can be evenly divided into M×N rectangular blocks, where M and N are both integers greater than or equal to 0, and each rectangular block represents a frame of sub-image. For another example, the type of the object to be recognized in the sample image can be determined first, and segmentation can be performed according to the region segmentation rule corresponding to the type. Specifically, when the sample image is an image containing a bank card, the region segmentation rule corresponding to the type of bank card can be adopted to segment the key regions of the bank card. For example, the title line of the bank card, the bank card number, the bank card logo, the background, etc. can be segmented respectively. The multiple frames of sub-images obtained can be sub-images including regions such as the title line of the bank card, the bank card number, the bank card logo, the background, etc.

[0080] According to an embodiment of the present disclosure, the feature extraction model can output a feature vector based on the input image data. In another embodiment, the feature extraction network can also be added element-wise to the sample image to obtain a final output, and the output result can be a result of determining whether it is a real image. The feature extraction model can be a lightweight feature extraction network model, or can be trained based on any existing feature extraction network architecture. The existing feature extraction network architectures can include but are not limited to MobileNetV2, ShuffleNetV2, PVANet, etc.

[0081] According to an embodiment of the present disclosure, the first sub-sample image is determined from multiple frames of sub-sample images by using the feature extraction model; the feature extraction can be performed on multiple frames of sub-sample images in the training sample set to obtain the first sub-sample image, which can reduce the parameters of the image processing model and reduce the training duration of the image processing model, thereby improving the efficiency of training the image processing model.

[0082] According to an embodiment of the present disclosure, operation S304 may further include the following operations: generating a second sub-sample image based on the first sub-sample image, including: performing a blurring process on the first sub-sample image to obtain the second sub-sample image; wherein, the blurring process includes Gaussian blurring and / or motion blurring.

[0083] According to an embodiment of the present disclosure, the first sub-sample image can be a clear image, and the second sub-sample image can be a blurred image corresponding to the clear image. Specifically, the target object can be video-shot by using a high-speed camera in a moving state, and then the video can be frame-cropped to find out the clear image and the blurred image corresponding to the clear image to obtain the first sub-sample image and the second sub-sample image. The blurred image can be generated in a simulated manner according to the reasons for the generation of the blurred image in the actual scenario. For example, it can be Gaussian blurring with different intensities and / or motion blurring with different intensities. Gaussian blurring and motion blurring are two main reasons for the blurred pictures taken. Specifically, when using Gaussian blurring, a Gaussian blur kernel can be used to generate a blurred image of the first sub-sample image as the second sub-sample image. Among them, the larger the convolution kernel used, the more obvious the blurring effect on the first sub-sample image. Preferably, since the sample images are mostly clear images, the albumentations library can be used to perform blurring processing on the first sub-sample image, and a 3×3 convolution kernel can be used during the processing.

[0084] According to an embodiment of the present disclosure, the initial image processing model can adopt the model architecture of the generative adversarial module. Taking the initial image processing model as the generative adversarial module as an example, the training method of the image processing model will be further described below.

[0085] According to an embodiment of the present disclosure, the initial image processing model may be composed of a generator and a discriminator.

[0086] According to an embodiment of the present disclosure, operation S305 may further include the following operations: processing the image data of the second sub-sample image using the generator to obtain the image data of the first generated image; processing the image data of the first sub-sample image and the image data of the first generated image using the discriminator to obtain discriminant data; and using the discriminant data to train the generator and the discriminator to obtain an image processing model.

[0087] According to an embodiment of the present disclosure, the initial image processing model may be a generative adversarial neural network model including a generator and a discriminator. The generative adversarial neural network model can be trained by the first sub-sample image and the second sub-sample image. Specifically, the generator and the discriminator can be trained according to the discriminant data obtained by the discriminator, and the loss of the generative adversarial neural network model can be determined by using the generator loss function and the discriminator loss function respectively. By minimizing the model loss, the training of the generative adversarial neural network model is completed.

[0088] According to an embodiment of the present disclosure, the discriminator (D) can be used to determine whether the image generated by the generator (G) is a fake picture or a real picture, that is, to judge the authenticity of the picture. The loss function of the discriminator (D) can be as shown in formula (1).

[0089] loss D =D fake -D real (1)

[0090] Wherein, loss D can represent the loss of the discriminator (D), D fake can represent the probability of judging a fake picture, D real can represent the probability of judging a real picture. Therefore, when D fake is smaller and at the same time D real is larger, it can be shown that the discriminator (D) can more accurately judge the authenticity of the picture. Therefore, the smaller the loss function loss_D, the better.

[0091] According to an embodiment of the present disclosure, the generator (G) is used to make the gap between the generated image and the clear image smaller and smaller. The loss function of the generator (G) can be as shown in formula (2).

[0092] loss G =content_loss+t×adv_loss (2)

[0093] Wherein, loss GIt can represent the loss of the generator (G). content_loss can represent that the gap between the generated image and the clear image becomes smaller and smaller. t can represent a hyperparameter set as needed; so that the discriminator can judge the generated image as a real image, and finally achieve the effect of image deblurring.

[0094] According to an embodiment of the present disclosure, in the generative adversarial neural network model, the feature extraction network architecture input to the discriminator can be replaced with a lightweight feature extraction network. For example, the mobileNet feature extraction network is used to extract the image data features of the first sub-sample image and the first generated image. By using the lightweight feature extraction network, not only a large number of training sample sets are not required when training the image processing model, but also the parameters of the image processing model can be reduced. Therefore, the training duration of the image processing model is reduced, and the image processing model is lightweighted.

[0095] According to an embodiment of the present disclosure, wherein the discriminator includes a global discriminator and a local discriminator, and the discriminant data includes first discriminant data and second discriminant data; using the discriminator to process the image data of the first sub-sample image and the image data of the first generated image to obtain discriminant data, including: using the global discriminator to process the image data of the first sub-sample image and the image data of the first generated image to obtain first discriminant data; respectively processing the first sub-sample image and the first generated image based on the predicted local cropping strategy to obtain a third sub-sample image and a second generated image; and using the local discriminator to process the image data of the third sub-sample image and the image data of the second generated image to obtain second discriminant data.

[0096] According to an embodiment of the present disclosure, adv_loss in formula (2) can be as shown in formula (3).

[0097] adv_loss = -D local(fake) -0.5×D global(fake) (3)

[0098] Wherein, D local(fake) can represent the probability that the local discriminator judges the pseudo-image as a pseudo-image, and D global(fake) can represent the probability that the global discriminator judges the pseudo-image as a pseudo-image. For adv_loss, since the purpose of the generator is to generate a pseudo-image closer to the real image so that the discriminator can judge the pseudo-image as a real image, the smaller adv_loss is, the better. The smaller this loss is, the more delicate texture generation can be guaranteed in the generated image, thereby improving the effect of image deblurring.

[0099] According to an embodiment of the present disclosure, taking the generative adversarial neural network model adopted by the initial image model as an example, the generative adversarial neural network model may include a generator and a discriminator. The generator hopes to generate an image closer to the real picture through continuous iteration, so that the discriminator judges the pseudo-picture generated by the generator as a real picture; the discriminator hopes to be able to distinguish the pseudo-picture generated by the generator. Therefore, after a large number of iterations, the pseudo-images generated by the generator are getting closer and closer to the real pictures, and the discriminator cannot recognize the authenticity of the generated pseudo-pictures.

[0100] According to an embodiment of the present disclosure, the first generated image may be a clear image different from the first sub-sample image generated by the generator according to the blurred second sub-sample image. By comparing the image data of the first sub-sample image and the image data of the first generated image by the discriminator, discriminant data can be obtained. Then, based on the discriminant data, the generator and the discriminator are trained through a large number of iterative calculations, adjusting model parameters and other operations to obtain a trained image processing model.

[0101] According to an embodiment of the present disclosure, the discriminator in the generative adversarial neural network model can be divided into a global discriminator and a local discriminator. An image in the training sample set can be input into the global discriminator as the whole image, and the local image obtained by randomly cropping the image locally can be input into the local discriminator. After a series of convolution operations, the probability that the discriminator judges the picture generated by the generator as a real picture can be output.

[0102] According to an embodiment of the present disclosure, the first sub-sample image can be used as the whole image and input into the global discriminator. Correspondingly, the first generated image can also be the whole image.

[0103] According to an embodiment of the present disclosure, the predicted local cropping strategy can be cropped according to the principle of the same region. Specific cropping methods can be random cropping, regional cropping, etc. For example, random cropping can be randomly cropping the first sub-sample image to obtain a third sub-sample image, and randomly cropping the first generated image to obtain a second generated image. Regional cropping can be cropping according to the key region. For example, when the first sub-sample image and the first generated image respectively contain a bank card, regions such as the bank card title line, bank card number, bank card logo, and background can be cropped to obtain a third sub-sample image and a second generated image. The third sub-sample image can be input into the local discriminator as the cropped local image. Correspondingly, the second generated image can also be the cropped local image.

[0104] According to an embodiment of the present disclosure, the loss of the initial image processing model can be determined by using the generator loss function and the discriminator loss function. By minimizing the model loss, the training of the initial image processing model is completed.

[0105] According to an embodiment of the present disclosure, during the training process of an image processing model, the existing training sample set can also be divided into a training set and a test set. The image processing model can be trained through the training set, and the discrimination ability of the model for the image to be processed can be evaluated through the test set. This can avoid the situation that the image processing model regards the characteristics of the training samples themselves as the general characteristics of potential samples during the training process, resulting in a good learning and training process for the training samples and causing the problem of overfitting.

[0106] According to an embodiment of the present disclosure, the cross-validation method can be used to divide the training sample set. Specifically, the training sample set can be divided into k parts, where k - 1 parts can be used as the training set, and the remaining 1 part can be used as the test set. In this way, k groups of training and testing are performed according to the divided k - 1 groups of training sets and 1 group of test sets, and finally the average value of k test results can be returned. The selection of the k value can be adaptively adjusted according to actual needs or experience. Preferably, k can be selected to be greater than or equal to 10 to ensure the stability of the output result.

[0107] According to an embodiment of the present disclosure, the training process of the image processing model can also include creating a virtual environment and installing relevant dependencies in the virtual environment. Specifically, a virtual environment of python3.9 can be created first, and then modules such as pytorch1.0.1, opencv, and numpy can be installed in this environment.

[0108] According to an embodiment of the present disclosure, the training process of the image processing model can also include modifying the configuration file. Specifically, initialization settings can be performed on multiple training parameters in the config file of the image processing model, such as the path of the training sample set, the number of epochs, batch_size, learning rate, and other information.

[0109] According to an embodiment of the present disclosure, the training process of the image processing model can also be visualized. Specifically, the tensorboard visualization tool can be used to display the index changes of tensors and network diagrams during the model training process. The saving path of the file and the loading path of the used training sample set can also be set when using it.

[0110] According to an embodiment of the present disclosure, during the process of training the model, it can also be judged whether optimization is needed based on the fitting situation during the training process. Specifically, it can be judged based on the change situation of the loss function of the generator and / or discriminator during the training of the model. If the situation of the loss function oscillating occurs, it can indicate that the entire model is relatively unstable. It can be judged based on the accuracy of the training set and the test set during the training process. If it is found that the accuracy of the training set is relatively high while the accuracy of the test set is relatively low, it can indicate that the image processing model has the situation of overfitting and the model needs to be optimized.

[0111] According to an embodiment of the present disclosure, data augmentation techniques can also be used to optimize the image processing model. Specifically, data augmentation techniques can be used on the training set, for example, by randomly pruning, color jittering, scale variation, contrast transformation, etc. to increase the data, which can not only improve the generalization ability of the model, but also increase the noisy data and enhance the robustness of the model.

[0112] According to an embodiment of the present disclosure, the model can also be optimized by modifying the learning rate. By using a varying learning rate, the fitting ability of the image processing model can be increased, thereby improving the accuracy of the model. Specifically, a relatively large learning rate can be used for training first, and the convergence curves of the loss function curve and the test set accuracy can be observed. When the descent rate of the loss function curve and the ascent rate of the convergence curve of the test set accuracy slow down, the learning rate can be decreased, and the decrease of the learning rate can be repeated multiple times until the descent rate of the loss function curve and the ascent rate of the convergence curve of the test set accuracy are no longer affected.

[0113] According to an embodiment of the present disclosure, the image processing model can also be optimized by increasing the number of network layers. Specifically, the number of neural network layers can be appropriately increased to find the turning point where the accuracy rises and then falls during the training process of the test set, and the performance of the model can be optimized by increasing the depth of the network layers.

[0114] According to an embodiment of the present disclosure, the trained image processing model can also be verified. First, an appropriate model can be selected. Specifically, after training the training sample set using a GPU (Graphics Processing Unit), multiple available model parameter files will be generated, and an appropriate model can be selected for verification by judgment.

[0115] According to an embodiment of the present disclosure, after selecting the model, the original test images to be deblurred can be input into the model one by one, and the model can output multiple sharp images corresponding to the test images to be stored in a folder.

[0116] According to an embodiment of the present disclosure, the deblurring effect can be viewed after the sharp images are output. Specifically, the output sharp images can be saved in a folder, and the comparison effect between the original test images and the deblurred sharp images can be viewed in the folder, which can realize the verification of the trained image processing model.

[0117] According to an embodiment of the present disclosure, an initial sample image is obtained, where the initial sample set includes multiple initial sample images; for each initial sample image in the multiple initial sample images, the initial sample image is subjected to data augmentation processing to obtain multiple sample images; where the data augmentation processing includes any one or more of the following: image cropping, image rotation, scale variation, color jittering, contrast transformation, and adding noise.

[0118] According to an embodiment of the present disclosure, the sample image may be an initial sample image after data augmentation. The data augmentation may include any one or more of image cropping, image rotation, scale change, color jitter, contrast transformation, and adding noise. For example, for image rotation, the initial sample image may be rotated with the upper left corner of the image as the rotation center to obtain the sample image, and the random rotation angle may be -45° to 45°; for scale change, the initial image may be randomly scaled to obtain the sample image, and the scaling range may be 0.5 - 1.5 times the size of the initial image; for adding noise, salt-and-pepper noise may be added to obtain the sample image, for example, by randomly changing some pixel values of the initial image to 0 or 255 to achieve adding salt-and-pepper noise.

[0119] According to an embodiment of the present disclosure, before blurring the first sub-sample image, data augmentation operations may also be performed on the first sub-sample image. For example, operations such as rotation, scaling, and shearing may be performed on the first sub-sample image, and then the first sub-sample image after data augmentation is blurred to generate the second sub-sample image.

[0120] According to an embodiment of the present disclosure, by performing data augmentation on the basis of the initial sample image, a model trained using the training sample set after data augmentation can improve the defuzzification effect and generate an image with higher resolution under the condition that other training conditions remain unchanged.

[0121] It should be noted that unless it is clearly stated that there is a sequential execution order between different operations in the flowcharts shown in the embodiments of the present disclosure, or there is a sequential execution order between different operations in terms of technical implementation, the execution order between multiple operations may be unsequenced, and multiple operations may also be executed simultaneously.

[0122] Figure 4 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is schematically shown.

[0123] As Figure 4 shown, the image processing apparatus 400 includes a response module 410, a first segmentation module 420, a first processing module 430, and a second processing module 440.

[0124] The response module 410 is configured to obtain the image to be processed carried in the service processing request in response to the service processing request.

[0125] The first segmentation module 420 is configured to segment the image to be processed to obtain multiple frames of sub-images.

[0126] The first processing module 430 is configured to respectively process the image data of multiple frames of the sub-images using a feature extraction model to determine a first target sub-image from multiple frames of the sub-images.

[0127] The second processing module 440 is configured to process the image data of the first target sub-image using an image processing model to obtain a target image.

[0128] According to an embodiment of the present disclosure, the first processing module 430 further includes a first determination unit, a first processing unit, a second determination unit, and a third determination unit.

[0129] The first determination unit is configured to determine a reference feature vector based on the service type, where the reference feature vector is obtained by processing the image data of a reference image associated with the service type using the feature extraction model.

[0130] The first processing unit is configured to process the image data of multiple frames of the sub-images respectively using the feature extraction model to obtain multiple sub-feature vectors.

[0131] The second determination unit is configured to determine the similarity between each sub-feature vector among the multiple sub-feature vectors and the reference feature vector to obtain multiple similarities.

[0132] The third determination unit is configured to determine the first target sub-image from multiple frames of the sub-images based on the multiple similarities.

[0133] According to an embodiment of the present disclosure, the image processing apparatus 400 further includes a cropping module.

[0134] The cropping module is configured to crop a second target sub-image from the image to be processed based on the first target sub-image.

[0135] According to an embodiment of the present disclosure, the second processing unit 440 includes a second processing unit.

[0136] The second processing unit is configured to process the image data of the second target sub-image using the image processing model to obtain the target image.

[0137] According to an embodiment of the present disclosure, the cropping module includes a fourth determination unit and a cropping unit.

[0138] The fourth determination unit is configured to determine the position of the first target sub-image in the image to be processed to obtain position information.

[0139] The cropping unit is configured to crop the second target sub-image from the image to be processed based on the position information.

[0140] According to an embodiment of the present disclosure, the image processing apparatus further includes a third processing unit.

[0141] The third processing unit is configured to process the image data of the target image using an image recognition model to obtain an image recognition result of the image to be processed. According to an embodiment of the present disclosure, the image processing apparatus further includes a first acquisition module, a second segmentation module, a third processing module, a generation module, and a training module.

[0142] The first acquisition module is configured to acquire a training sample set, where the training sample set includes a plurality of sample images.

[0143] The second segmentation module is configured to segment each of the plurality of sample images to obtain a plurality of frames of sub-sample images.

[0144] The third processing module is configured to process the image data of the plurality of frames of sub-sample images respectively using a feature extraction model to determine a first sub-sample image from the plurality of frames of sub-sample images.

[0145] The generation module is configured to generate a second sub-sample image based on the first sub-sample image.

[0146] The training module is configured to train an initial image processing model using the image data of the first sub-sample image and the image data of the second sub-sample image to obtain an image processing model.

[0147] According to an embodiment of the present disclosure, the generation module includes a fourth processing unit.

[0148] The fourth processing unit is configured to perform a blurring process on the first sub-sample image to obtain the second sub-sample image.

[0149] According to an embodiment of the present disclosure, the training module includes a fifth processing unit, a sixth processing unit, and a training unit.

[0150] The fifth processing unit is configured to process the image data of the second sub-sample image using the generator to obtain the image data of a first generated image.

[0151] The sixth processing unit is configured to process the image data of the first sub-sample image and the image data of the first generated image using the discriminator to obtain discriminant data.

[0152] The training unit is configured to train the generator and the discriminator using the discriminant data to obtain the image processing model.

[0153] According to an embodiment of the present disclosure, the sixth processing unit further includes a first processing subunit, a second processing subunit, and a third processing subunit.

[0154] The first processing subunit is configured to process the image data of the first sub-sample image and the image data of the first generated image using the global discriminator to obtain the first discriminant data.

[0155] The second processing subunit is configured to process the first sub-sample image and the first generated image respectively based on a predicted local cropping strategy to obtain a third sub-sample image and a second generated image.

[0156] The third processing subunit is configured to process the image data of the third sub-sample image and the image data of the second generated image by using the local discriminator to obtain the second discriminant data.

[0157] According to an embodiment of the present disclosure, the image processing apparatus further includes a second acquisition module and a fourth processing module.

[0158] The second acquisition module is configured to acquire an initial sample image, where the initial sample set includes a plurality of initial sample images.

[0159] The fourth processing module is configured to perform data augmentation processing on each of the plurality of initial sample images to obtain a plurality of the sample images.

[0160] According to an embodiment of the present disclosure, any plurality of modules, sub-modules, units, and sub-units, or at least part of the functions of any of them can be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure can be split into multiple modules for implementation. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by any other reasonable way of integrating or packaging circuits in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware or in an appropriate combination of any several of them. Alternatively, one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding functions can be executed.

[0161] For example, any combination of the response module 410, the first segmentation module 420, the first processing module 430, and the second processing module 440 can be implemented in one module / unit / sub-unit, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more of these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the response module 410, the first segmentation module 420, the first processing module 430, and the second processing module 440 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in a suitable combination of any several of them. Alternatively, at least one of the response module 410, the first segmentation module 420, the first processing module 430, and the second processing module 440 can be at least partially implemented as a computer program module, which can execute corresponding functions when the computer program module is run.

[0162] It should be noted that the part of the image processing apparatus in the embodiments of the present disclosure corresponds to the part of the image processing method in the embodiments of the present disclosure. For the description of the image processing apparatus part, please refer to the image processing method part for details, and will not be elaborated herein.

[0163] Figure 5 A block diagram of an electronic device suitable for implementing the image processing method according to an embodiment of the present disclosure is schematically shown. Figure 5 The shown electronic device is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0164] As Figure 5 shown, the computer electronic device 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to the program stored in the read only memory (ROM) 502 or the program loaded from the storage section 508 into the random access memory (RAM) 503. The processor 501 can include, for example, a general microprocessor (such as a CPU), an instruction set processor and / or a related chipset, and / or a dedicated microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 501 can also include on-board memory for caching purposes. The processor 501 can include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiments of the present disclosure.

[0165] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 502 and / or the RAM 503. It should be noted that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0166] According to an embodiment of the present disclosure, the electronic device 500 may further include an input / output (I / O) interface 505, and the input / output (I / O) interface 505 is also connected to the bus 504. The electronic device 500 may further include one or more of the following components connected to the I / O interface 505: an input portion 506 including a keyboard, a mouse, etc.; an output portion 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage portion 508 including a hard disk, etc.; and a communication portion 509 including a network interface card such as a LAN card, a modem, etc. The communication portion 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed so that a computer program read from it can be installed into the storage portion 508 as needed.

[0167] According to an embodiment of the present disclosure, the method flow according to the embodiments of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication portion 509, and / or installed from the removable medium 511. When the computer program is executed by the processor 501, the above-described functions defined in the system according to the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described system, device, apparatus, module, unit, etc. can be implemented by computer program modules.

[0168] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist separately without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the methods according to the embodiments of the present disclosure are implemented.

[0169] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0170] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include one or more memories other than the above-described ROM 502 and / or RAM 503 and / or ROM 502 and RAM 503.

[0171] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program code for executing the methods provided in the embodiments of the present disclosure. When the computer program product runs on an electronic device, the program code is used to cause the electronic device to implement the image processing method provided in the embodiments of the present disclosure.

[0172] When the computer program is executed by the processor 501, the above functions defined in the system / apparatus of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0173] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and be downloaded and installed through the communication part 509, and / or be installed from the removable medium 511. The program code included in the computer program may be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0174] According to embodiments of the present disclosure, program code for executing the computer programs provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, Python, the "C" language, or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0175] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks can occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and combinations of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions. Those skilled in the art can understand that the features described in various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0176] The above describes the embodiments of the present disclosure. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. An image processing method, comprising: In response to a service processing request, obtaining the image to be processed carried in the service processing request; Segmenting the image to be processed to obtain multiple frames of sub-images; Using a feature extraction model to process the image data of multiple frames of the sub-images respectively to determine a first target sub-image from multiple frames of the sub-images; Using an image processing model to process the image data of the first target sub-image to obtain a target image; The method further comprises: Based on the first target sub-image, intercepting a second target sub-image from the image to be processed, wherein the second target sub-image is obtained by intercepting the image to be processed based on a region that is proportionally enlarged or reduced around the first target sub-image with the first target sub-image as the center; Wherein, the using an image processing model to process the image data of the first target sub-image to obtain a target image comprises: Using the image processing model to process the image data of the second target sub-image to obtain the target image.

2. The method according to claim 1, wherein The service processing request also carries a service type; Wherein, the using a feature extraction model to process the image data of multiple frames of the sub-images respectively to determine a first target sub-image from multiple frames of the sub-images comprises: Based on the service type, determining a reference feature vector, wherein the reference feature vector is obtained by using the feature extraction model to process the image data of a reference image associated with the service type; Using the feature extraction model to process the image data of multiple frames of the sub-images respectively to obtain multiple sub-feature vectors; For each sub-feature vector among the multiple sub-feature vectors, determining the similarity between the sub-feature vector and the reference feature vector to obtain multiple similarities; and Based on the multiple similarities, determining the first target sub-image from multiple frames of the sub-images.

3. The method according to claim 1, wherein, The intercepting a second target sub-image from the image to be processed based on the first target sub-image comprises: Determining the position of the first target sub-image in the image to be processed to obtain position information; and Based on the position information, intercepting the second target sub-image from the image to be processed.

4. The method according to claim 3, wherein, The size of the second target sub-image is larger than the size of the first target sub-image.

5. The method according to claim 1, further comprising: Using an image recognition model to process the image data of the target image to obtain an image recognition result of the image to be processed.

6. The method according to any one of claims 1 to 5, wherein The image processing model is obtained by training through the following method: Obtaining a training sample set, wherein the training sample set includes multiple sample images; For each sample image among the multiple sample images, segmenting the sample image to obtain multiple frames of sub-sample images; Using a feature extraction model to process the image data of multiple frames of the sub-sample images respectively to determine a first sub-sample image from multiple frames of the sub-sample images; Based on the first sub-sample image, generating a second sub-sample image; and Using the image data of the first sub-sample image and the image data of the second sub-sample image to train an initial image processing model to obtain the image processing model.

7. The method according to claim 6, wherein, Generating a second sub-sample image based on the first sub-sample image includes: Performing blurring processing on the first sub-sample image to obtain the second sub-sample image; Wherein, the blurring processing includes Gaussian blurring processing and / or motion blurring processing.

8. The method according to claim 6, wherein The initial image processing model includes a generator and a discriminator; Training the initial image processing model using the image data of the first sub-sample image and the image data of the second sub-sample image to obtain the image processing model includes: Processing the image data of the second sub-sample image using the generator to obtain the image data of a first generated image; Processing the image data of the first sub-sample image and the image data of the first generated image using the discriminator to obtain discriminant data; and Training the generator and the discriminator using the discriminant data to obtain the image processing model.

9. The method according to claim 8, wherein The discriminator includes a global discriminator and a local discriminator, and the discriminant data includes first discriminant data and second discriminant data; Processing the image data of the first sub-sample image and the image data of the first generated image using the discriminator to obtain discriminant data includes: Processing the image data of the first sub-sample image and the image data of the first generated image using the global discriminator to obtain the first discriminant data; Processing the first sub-sample image and the first generated image respectively based on a predicted local cropping strategy to obtain a third sub-sample image and a second generated image; and Processing the image data of the third sub-sample image and the image data of the second generated image using the local discriminator to obtain the second discriminant data.

10. The method according to claim 6, further comprising: Obtaining an initial sample image, wherein the initial sample set includes multiple initial sample images; For each initial sample image among the multiple initial sample images, performing data augmentation processing on the initial sample image to obtain multiple sample images; Wherein, the data augmentation processing includes any one or more of the following: image cropping, image rotation, scale change, color jitter, contrast transformation, and adding noise.

11. An image processing apparatus, comprising: A response module, configured to obtain a to-be-processed image carried in the service processing request in response to a service processing request; A first splitting module, configured to split the to-be-processed image to obtain multiple frames of sub-images; A first processing module, configured to process the image data of multiple frames of the sub-images respectively using a feature extraction model to determine a first target sub-image from multiple frames of the sub-images; A second processing module, configured to process the image data of the first target sub-image using an image processing model to obtain a target image; The apparatus further includes: An intercepting module, configured to intercept a second target sub-image from the to-be-processed image based on the first target sub-image, wherein the second target sub-image is obtained by intercepting the to-be-processed image with a region that proportionally expands or shrinks around the first target sub-image with the first target sub-image as the center; Wherein, the second processing module includes: A second processing unit, configured to process the image data of the second target sub-image by using the image processing model to obtain the target image.

12. An electronic device, comprising: One or more processors; A memory, configured to store one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, having stored thereon executable instructions, which when executed by a processor cause the processor to implement the method according to any one of claims 1 to 10.

14. A computer program product, the computer program product comprising computer-executable instructions, which when executed are used to implement the method according to any one of claims 1 to 10.

Images