Image processing method, device, equipment and computer readable storage medium

Abstract

The embodiment of the present application provides an image processing method, apparatus, device and computer-readable storage medium, wherein the method comprises: extracting features of an image to be processed to obtain directional features and image features of the image to be processed; identifying the directional features to obtain the category of the image to be processed; performing image reconstruction processing on the image to be processed according to the category and the image features to obtain a processed image; and outputting the processed image. Through the present application, accurate image reconstruction processing can be performed on the image to be processed according to the accurate category to obtain a processed image that meets the user's processing requirements.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and relate to but are not limited to an image processing method, apparatus, device, and computer-readable storage medium. Background Art

[0002] At present, most of the image processing is done on image recognition, such as face recognition, license plate recognition, text recognition, etc. Image processing mainly focuses on coloring black and white photos, image content prediction, and fuzzy photo refinement. The processing method mainly combines traditional rule-based image recognition methods with statistical processing methods. Photographic post-processing is a relatively niche field in image processing, and its target audience is mainly the processing of original photos by photographers or built-in software in cameras.

[0003] Most of the photography post-processing methods in the related technology rely on image processing software (such as Photoshop or Lightroom). For photographers, the processing of landscape photography images is often a tedious and repetitive task, but good post-processing requires a certain amount of inspiration, which requires photographers to spend a lot of time learning various software operation steps; for cameras, a good built-in post-processing software can also greatly increase customers' desire to buy.

[0004] However, the image processing in the related technology is still rarely used in photography post-processing. Traditional technical solutions are difficult to accurately reconstruct images. Therefore, the image processing methods in the related technology will have the problems of low accuracy and poor processing effect in the post-processing of landscape photography images. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, apparatus, device and computer-readable storage medium, which can reduce the time cost of learning various post-processing software, accurately identify the category of the image to be processed, and accurately reconstruct the image to be processed according to the accurate category, so as to obtain a processed image that meets the user's processing requirements, thereby improving the accuracy and processing effect of image post-processing.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides an image processing method, comprising:

[0008] Performing feature extraction on the image to be processed to obtain directional features and image features of the image to be processed;

[0009] Identifying the directional features to obtain the category of the image to be processed;

[0010] According to the category and the image features, performing image reconstruction processing on the image to be processed to obtain a processed image;

[0011] The processed image is output.

[0012] The present application provides an image processing device, including:

[0013] A feature extraction module, used to extract features from the image to be processed, and obtain directional features and image features of the image to be processed;

[0014] A recognition module, used for recognizing the directional features to obtain the category of the image to be processed;

[0015] An image reconstruction module, used for performing image reconstruction processing on the image to be processed according to the category and the image features to obtain a processed image;

[0016] An output module is used to output the processed image.

[0017] The present application provides an image processing device, including:

[0018] The memory is used to store executable instructions; the processor is used to implement the above-mentioned image processing method when executing the executable instructions stored in the memory.

[0019] An embodiment of the present application provides a storage medium storing executable instructions for causing a processor to execute the above-mentioned image processing method.

[0020] The image processing method, apparatus, device and computer-readable storage medium provided in the embodiments of the present application can accurately identify the category of the image to be processed, perform accurate image reconstruction processing on the image to be processed according to the accurate category, improve the accuracy and processing effect of image post-processing, and obtain a processed image that meets the user's processing requirements by extracting the directional features and image features of the image to be processed and processing the image to be processed based on the directional features and image features. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0022] Figure 2A is a schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0023] Figure 2B It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0024] Figure 3 It is an optional flowchart of the image processing method provided in the embodiment of the present application;

[0025] Figure 4 This is an optional flowchart of the image processing model training method provided in the embodiment of the present application;

[0026] Figure 5 This is an optional flowchart of the image processing model training method provided in the embodiment of the present application;

[0027] Figure 6 This is an optional flowchart of the image processing model training method provided in the embodiment of the present application;

[0028] Figure 7 It is a schematic diagram of the implementation process of the model construction stage provided in the embodiment of the present application;

[0029] Figure 8 It is a schematic diagram of the structure of the first convolutional neural network model provided in an embodiment of the present application;

[0030] Fig. 9 is a schematic diagram of the structure of a second convolutional neural network model provided in an embodiment of the present application;

[0031] Fig.10 It is a schematic diagram of the implementation process of the model training phase provided in the embodiment of the present application;

[0032] Fig.11 It is a flowchart of the image recognition stage provided in an embodiment of the present application;

[0033] Fig.12 is a schematic diagram of the composition structure of an image processing device provided in an embodiment of the present application;

[0034] Fig.13 It is a schematic diagram of the composition structure of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0036] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0037] In order to better understand the image processing method provided in the embodiments of the present application, the image processing method in the related art and the existing problems are first described:

[0038] In related technologies, image processing is mostly focused on image recognition, and image processing mainly focuses on colorization of black and white photos, image content prediction, and refinement of blurred photos. The processing methods mainly combine traditional rule-based image recognition methods with statistical-based processing methods.

[0039] The methods in the related art have at least the following problems:

[0040] 1) Traditional rule-based image recognition methods are difficult to achieve good recognition results in image content classification. When the complexity of the image content reaches a certain level, the recognition rate decreases sharply. For example, for landscape photography, the content is complex and diverse, which is more difficult to recognize than traditional simple images such as faces and text.

[0041] 2) In the related technologies, there are few post-processing of the whole image, and most of them process the features of a certain part of the image or perform image content recognition, such as image sharpness processing, black and white photo colorization, face recognition, and text recognition, which rarely involve post-processing of photographic images.

[0042] 3) Although a single convolutional neural network has achieved good results in the classification of some images, it is prone to overfitting on the training set and falls into local optimality, and the recognition rate on the verification set is often relatively low.

[0043] 4) In the camera software post-processing technology in the related art, although the software can perform corresponding post-processing according to the corresponding scene and directly output photos, the techniques are still relatively simple. For example, a certain brand of mobile phone can only perform content recognition according to a few scenes such as blue sky, flowers, sunset, etc., and then perform post-processing. The processing method is also a simple high dynamic range image (HDR, High-Dynamic Range), saturation, contrast, exposure, etc. The adjustment is difficult to achieve the post-processing techniques of general landscape photographers.

[0044] 5) Some image processing software such as Lightroom and Photoshop use presets and filters to perform some automated processing, but they are more targeted and only applicable to photographs of the same style. For landscape photography, the content and form are more complex and varied, and simply using presets and other related techniques cannot meet the photographer's post-production needs.

[0045] 6) In various industrialized photography organizations, photography post-processing requires a huge amount of manpower, and photography post-processing is a highly repetitive job. Most of the work steps are almost the same. For a large number of homogeneous photos, how to retouch the photos quickly and well is an urgent problem to be solved.

[0046] Based on at least one of the above problems existing in the related art, in order to better integrate valuable photo editing resources and allow photographers to free up their hands for more creative work (such as more pre-shooting), a neural network is combined with post-processing of images to provide an image processing method that uses a convolutional neural network to perform post-processing of landscape photography. Perform feature extraction on the image to be processed to obtain the directional features and image features of the image to be processed; identify the directional features to obtain the category of the image to be processed; perform image reconstruction processing on the image to be processed according to the category and the image features to obtain a processed image; and output the processed image. In this way, the category of the image to be processed can be accurately identified, so that the image to be processed can be accurately reconstructed according to the accurate category to obtain a processed image that meets the user's processing requirements.

[0047] The following describes an exemplary application of the image processing device provided by the embodiment of the present application. The image processing device provided by the embodiment of the present application can be implemented as various types of terminals such as a laptop computer, a tablet computer, a desktop computer, a mobile device (for example, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device), and can also be implemented as a server. The following describes an exemplary application when the program execution device is implemented as a server.

[0048] See also Figure 1 , Figure 1 is an optional flow chart of the image processing method provided in the embodiment of the present application, which will be combined with Figure 1 The steps shown are explained.

[0049] Step S101 , extracting features from an image to be processed to obtain directional features and image features of the image to be processed.

[0050] Here, the image to be processed may be a picture downloaded from a website, or may be a picture taken by a user using a camera device, for example, may be a landscape picture taken by a user.

[0051] In the embodiment of the present application, when performing feature extraction, different feature extraction tools can be used to extract directional features and image features of the image to be processed. Among them, directional features refer to features that characterize the directionality of the image. The extraction of directional features is actually the identification of features in different directions of the image, which simulates the selectivity of the human eye for direction in image recognition. In the embodiment of the present application, the extraction of directional features is mainly to obtain some local features of the image, that is, the directional features extracted in the embodiment of the present application are local features on the image to be processed. Image features refer to features that characterize the correlation between multiple image pixels, which can be overall features.

[0052] Step S102: identifying the directional features to obtain the category of the image to be processed.

[0053] Here, the category of the image to be processed is determined according to the directional features, and the category may be any image category, for example, the category may be a landscape category such as mountains, water, buildings, night scenes, sunsets, starry skies, plants, roads, snow scenes, flowers, magnificent, mysterious, tiny, beaches, etc., or a character category such as expressions, faces, etc.

[0054] Step S103: performing image reconstruction processing on the image to be processed according to the category and the image feature to obtain a processed image.

[0055] Here, the image post-processing of the image to be processed is realized by image reconstruction processing, wherein different image categories correspond to different processing methods. For example, for landscape photography pictures, the content included in the picture is more complex and has more information, while for character pictures, the content included in the picture is simple and has less information. Therefore, different processing methods are required, and the overall effect of the picture is also different for landscape photography pictures and character pictures. Therefore, the image to be processed can be reconstructed according to the category of the image to be processed, so as to obtain a processed image that meets the image requirements.

[0056] Step S104: output the processed image.

[0057] The image processing method provided in the embodiment of the present application extracts the directional features and image features of the image to be processed, and processes the image to be processed based on the directional features and image features. In this way, it is possible to accurately identify the category of the image to be processed, and accurately reconstruct the image to be processed according to the accurate category, thereby improving the accuracy and processing effect of image post-processing, thereby obtaining a processed image that meets the user's processing requirements.

[0058] The image processing method of the embodiment of the present application can be applied to post-processing of landscape photography, wherein the image processing method of the embodiment of the present application can be applied to an image processing system. Figure 2A is a schematic diagram of the structure of the image processing system provided in the embodiment of the present application, such as Figure 2A As shown, the image processing system 20 includes: a terminal 100, a network 200 and a server 300. After the photographer takes a landscape photography image, the image is input into the terminal 100 as an image to be processed. The terminal 100 is installed with an image processing application (APP). The terminal 100 sends the image to be processed to the server 300 of the image processing application through the network 200, so that the server 300 performs image post-processing on the image to be processed to obtain a processed image. At least one of the image to be processed and the processed image can be displayed on the current interface 100-1 of the terminal 100.

[0059] Figure 2B is an optional flow chart of the image processing method provided in the embodiment of the present application, such as Figure 2B As shown, based on Figure 2A The image processing system comprises the following steps:

[0060] Step S201: The user inputs a landscape photography image as an image to be processed into a terminal.

[0061] Step S202: the terminal sends the image to be processed to the server.

[0062] Step S203: the server extracts features of the image to be processed to obtain local features and overall features of the image to be processed.

[0063] Here, the local features correspond to the above-mentioned directional features, and the overall features correspond to the above-mentioned image features.

[0064] Step S204: the server identifies local features and overall features to obtain the category of the image to be processed.

[0065] In some embodiments, when determining the category of the image to be processed, it can be achieved by the following steps:

[0066] Step S2041, performing convolution processing and pooling processing on the local features in sequence to obtain pooling processing features.

[0067] Here, local features are local features of local areas in the image to be processed. Since not all image features in all areas of the entire image to be processed are valid features and cannot all be used as the basis for determining the image category, image category recognition is performed by obtaining local features of the image to be processed.

[0068] In an embodiment of the present application, the local feature may be a feature of an area in the image to be processed with more feature data, or a feature of an area with larger variations in feature data. For example, if there is an exposed area in the image to be processed, the feature data of the area is less, and the feature of the area cannot be used as a valid feature for image category recognition. Alternatively, if there is an area in the image to be processed with more objects (such as people, scenery, and buildings), the area is the most capable of accurately determining the image category, and the variations in the feature data of the area are larger (i.e., the variation difference in the pixel values ​​of the pixel points is larger), so the feature of the area is determined as the above-mentioned local feature.

[0069] In an embodiment of the present application, the local features can be convolved by a convolutional layer in a convolutional neural network, and the local features after the convolutional processing can be pooled by a pooling layer in the convolutional neural network.

[0070] Step S2042: performing identification processing on the pooled processing features to determine the category corresponding to the pooled processing features.

[0071] Here, by comparing the pooled processing features with the preset features of different categories preset in the convolutional neural network, the feature difference between the pooled processing features and each preset feature is determined, and the preset feature with the smallest feature difference is determined as the similar feature of the pooled processing feature. Finally, the category of the similar feature is determined as the category of the pooled processing feature.

[0072] Step S2043: Determine the category corresponding to the pooling processing feature as the category of the image to be processed.

[0073] Step S205: the server performs image reconstruction processing on the image to be processed according to the category and overall features to obtain a processed image.

[0074] Step S206: the server sends the processed image to the terminal.

[0075] Step S207: the terminal displays the processed image on the current interface.

[0076] In the image processing method provided in the embodiment of the present application, the terminal sends the image to be processed to the server for processing. Since the server extracts local features and overall features of the image to be processed, and processes the image to be processed based on the local features and the overall features, the category of the image to be processed can be accurately identified according to the local features and the overall features, so that the image to be processed is accurately reconstructed according to the accurate category, and a processed image that meets the user's processing requirements is obtained, and the processed image that meets the user's processing requirements is displayed on the terminal.

[0077] In some embodiments, the image processing method of the present application embodiment can be implemented by combining filters and convolutional neural networks. Figure 1 , Figure 3 is an optional flow chart of the image processing method provided in the embodiment of the present application, such as Figure 3 As shown, step S101 can be implemented by the following steps:

[0078] Step S301: Perform Fourier transform processing on the image to be processed by using a filter to obtain directional features of the image to be processed.

[0079] Here, the filter may be a Gabor filter. In an embodiment of the present application, a Gabor filter may be used to perform Fourier transform processing on the image to be processed to obtain the directional features of the image to be processed. The Gabor filter is obtained by superimposing a trigonometric function (such as a sine function) with a Gaussian function. The Gabor filter may be used to perform trigonometric function and Gaussian function transformation processing on the pixels of the image to be processed, and the transformed value is determined as the directional feature.

[0080] Step S302: Use a convolutional neural network to extract features of the image to be processed to obtain image features of the image to be processed.

[0081] Here, a convolutional neural network is used to extract features from the image to be processed, wherein the convolutional neural network used to realize feature extraction includes at least a convolution layer and a pooling layer, and convolution processing is performed on the image to be processed through a certain number of convolution layers, and pooling processing is performed on the image to be processed through a certain number of pooling layers, so as to obtain image features of the image to be processed, and the image features are basic features for image processing.

[0082] Please continue to refer to Figure 3 Step S103 can be implemented by the following steps:

[0083] Step S303: determine the convolutional neural network corresponding to the category as the target convolutional neural network.

[0084] Step S304: Use the target convolutional neural network to perform image reconstruction processing on the image features to obtain the processed image.

[0085] In an embodiment of the present application, after determining the category of the image to be processed, a target convolutional neural network is obtained according to the category of the image to be processed, and the obtained target convolutional neural network is used to perform image reconstruction processing.

[0086] In some embodiments, it is also possible to determine whether to process the image to be processed based on the convolutional neural network, that is, the type corresponding to the convolutional neural network is fixed, and the convolutional neural network is used to perform image reconstruction processing on a specific type of image to be processed. If the type of the image to be processed is a specific type, the convolutional neural network is used to process the image to be processed. The specific category is a preset category, and the preset category can be a category preset by the user. The preset category can be set according to the actual processing needs of the user, or according to the image category that can be processed by the software used to implement image processing. For example, the specific category can be a landscape photography picture category, or the specific category can also be a mountain category in a landscape photography picture, etc.

[0087] In some embodiments, the image processing method of the embodiment of the present application can also be implemented using an image processing model, that is, using the image processing model to perform image reconstruction processing on the image to be processed. Among them, the image processing model includes a filter, an image category recognition sub-model and an image reconstruction sub-model, wherein the filter is used to extract features from the image to obtain the directional features of the image, the image category recognition sub-model is used to identify the type of the image, and the image reconstruction sub-model is used to identify the content of the image and reconstruct the image. The extraction of directional features in the image category recognition sub-model is mainly to obtain some local features of the image, and its ultimate goal is to extract the overall features of the image in combination with the image category recognition sub-model, and finally combine the local features and the overall features to identify the image category.

[0088] Here, a training method for an image processing model is provided, such as Figure 4 FIG. 1 is a schematic diagram of an optional flow chart of an image processing model training method provided in an embodiment of the present application, wherein the method comprises:

[0089] Step S401: input a sample image into a filter to obtain a directional feature of the sample image.

[0090] Here, the sample image may be a picture downloaded by the user from a website, or may be a picture taken by the user using a photographic device.

[0091] The filter is used to perform Fourier transform processing on the sample image, so as to obtain the directional features of the sample image. Among them, the extraction of the directional features of the sample image is actually to identify the features of the sample image in different directions, which simulates the selectivity of the human eye to direction in image recognition. In the embodiment of the present application, the directional features of the sample image are extracted mainly to obtain some local features of the sample image, that is, the directional features of the sample image extracted in the embodiment of the present application are local features on the sample image.

[0092] Step S402: input the directional features into an image category recognition sub-model, and perform image category recognition processing on the directional features through the image category recognition sub-model to obtain the category of the sample image.

[0093] Here, the image category recognition sub-model is used to recognize the category of the sample image according to the directional features. In the implementation process, the directional features may be subjected to convolution and pooling processing to finally obtain the category of the sample image.

[0094] Step S403: obtaining an image reconstruction sub-model corresponding to the category.

[0095] Step S404: input the sample image into the image reconstruction sub-model, and perform image reconstruction processing on the sample image through the image reconstruction sub-model to obtain a processed sample image.

[0096] Here, the image reconstruction sub-model is used to extract features from the sample image to obtain image features, and to perform image reconstruction processing on the image features to obtain an image after image reconstruction, that is, a processed sample image.

[0097] Step S405: input the category of the sample image and the processed sample image into a preset loss model to obtain a loss result.

[0098] In some embodiments, the preset loss model includes a first preset loss model and a second preset loss model; the loss result includes a first loss result and a second loss result. The first preset loss model is used to verify the output result of the image category recognition submodel to obtain a first loss result; the second preset loss model is used to verify the output result of the image reconstruction submodel to obtain a second loss result. Correspondingly, step S405 can be implemented by the following steps:

[0099] Step S4051: input the category of the sample image into the first preset loss model to obtain the first loss result.

[0100] Here, the first preset loss model is used to compare the sample image category with the preset category to obtain a first loss result, wherein the preset category can be obtained after the user manually identifies the category of the sample image.

[0101] The first preset loss model includes a first loss function, through which the first similarity between the category of the sample image and the preset category can be calculated, and the first loss result can be determined according to the first similarity.

[0102] Step S4052: input the processed sample image into the second preset loss model to obtain the second loss result.

[0103] Here, the second preset loss model is used to compare the processed sample image with the preset processed image to obtain a second loss result, wherein the preset processed image can be an image obtained after the user manually processes the sample image.

[0104] The second preset loss model includes a second loss function, through which the second similarity between the sample image and the preset processed image can be calculated, and the second loss result can be determined according to the second similarity.

[0105] Step S406: modify the image category recognition sub-model and the image reconstruction sub-model according to the loss result until the image category recognition sub-model and the image reconstruction sub-model can accurately perform image processing on the sample image.

[0106] Here, when the above similarity is greater than the preset similarity threshold, the loss result indicates that the image category recognition submodel in the current image processing model cannot accurately identify the category of the sample image, or indicates that the image reconstruction submodel in the current image processing model cannot accurately process the sample image. Therefore, it is necessary to correct the image category recognition submodel or the image reconstruction submodel in the current image processing model. Then, the image reconstruction submodel can be corrected according to the above similarity until the similarity between the processed sample image output by the image processing model and the preset processed image meets the preset conditions, and the corresponding image processing model is determined as the trained image processing model.

[0107] Of course, in some embodiments, the image reconstruction sub-model in steps S403 to S406 may also be a model corresponding to any other category, for example, it may be a model corresponding to a high mountain category, or it may be a model corresponding to a large tree category, and this embodiment of the present application does not limit this.

[0108] The training method of the image processing model provided in the embodiment of the present application, because the sample image is input into the filter to obtain the directional features of the sample image; the directional features are input into the image category recognition sub-model to obtain the category of the sample image, and the category of the sample image is input into the first preset loss model to obtain a first loss result; the sample image is input into the image reconstruction sub-model to obtain a processed sample image; and the processed sample image is input into the second preset loss model to obtain a second loss result. Therefore, the image category recognition sub-model and the image reconstruction sub-model can be corrected according to the first loss result and the second loss result, respectively, and the obtained corrected image category recognition sub-model can accurately identify the category of the image, and the obtained corrected image reconstruction sub-model can accurately perform image post-processing on the image, so as to obtain a processed image that meets the user's processing requirements and improve the user experience.

[0109] In the embodiment of the present application, the image category recognition submodel and the image reconstruction submodel can be two independent training processes, by inputting the directional features generated by the Gabor filter into the image category recognition submodel, after several iterations, comparing with the pre-labeled image category, and then correcting the image category recognition submodel according to the first loss result; the image reconstruction submodel training process is to input the sample image with known category information into the image reconstruction submodel, after multiple iterations, evaluate the second loss result through the loss function, and continuously correct the model parameters. Therefore, corresponding to multiple categories of images, multiple categories of image reconstruction submodels (such as mountains, sunrises, etc.) can be trained; in the image category recognition process, it is only necessary to input the image category recognized by the image category recognition submodel into the corresponding image reconstruction submodel (for example, if the result of the image category recognition submodel is the mountain category, then the image can be input into the image reconstruction submodel of the mountain category), and the corresponding category processing can be performed.

[0110] In some embodiments, the image reconstruction sub-model can not only realize the image feature extraction of the sample image, but also realize the image reconstruction processing of the sample image. Figure 4 ,like Figure 5 FIG. 4 is an optional flow chart of the image processing model training method provided in an embodiment of the present application. Step S404 can also be implemented by the following steps:

[0111] Step S501, determine the image reconstruction sub-model corresponding to the category of the sample image, input the sample image into the image reconstruction sub-model, extract features of the sample image through the feature extraction layer in the image reconstruction sub-model, and obtain image features of the sample image.

[0112] Step S502: performing image reconstruction processing on the image features through the image reconstruction processing layer in the image reconstruction sub-model to obtain the processed sample image.

[0113] Here, the image reconstruction processing layer includes a convolution layer and an upsampling layer; correspondingly, step S502 can be implemented by the following steps:

[0114] Step S5021, performing convolution processing on the image features through the convolution layer to obtain convolution processing features.

[0115] Step S5022: upsample the convolution processing features through the upsampling layer to obtain upsampled features.

[0116] Step S5023: determining the processed sample image according to the up-sampling feature.

[0117] based on Figure 4 , Figure 6 is an optional flow chart of the image processing model training method provided in the embodiment of the present application, such as Figure 6 As shown, the method also includes:

[0118] Step S601: input a preset category corresponding to the sample image into the first preset loss model, and input a processed image corresponding to the sample image into the second preset loss model.

[0119] It should be noted that step S601 can be executed at any time before step S405, and this embodiment of the present application does not limit this.

[0120] Correspondingly, step S405 can be implemented by the following steps:

[0121] Step S602: Determine a first similarity between the category of the sample image and the preset category through the first preset loss model.

[0122] Step S603: determine the first loss result according to the first similarity.

[0123] Step S604: Determine a second similarity between the processed sample image and the processed image by using the second preset loss model.

[0124] Step S605: determine the second loss result according to the second similarity.

[0125] Correspondingly, step S406 can be implemented by the following steps:

[0126] Step S606: modify the image category recognition sub-model according to the first loss result until the image category recognition sub-model can accurately determine the category of the sample image.

[0127] Step S607: modify the image reconstruction sub-model according to the second loss result until the image reconstruction sub-model can accurately perform image processing on the sample image.

[0128] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0129] The embodiment of the present application provides an image processing method, which uses a convolutional neural network to perform post-processing of landscape photography, mainly using a convolutional neural network and a Gabor filter. A convolutional neural network is a deep feedforward neural network with multiple convolutional layers and pooling layers. The local features of the image are extracted through the convolutional layer, and the local features are input into the activation function. After normalization, the classification result of the image is output. The convolutional neural network mainly includes two processes: the training stage and the classification stage. A large number of sample sets are required in the training stage. The recognition results of a large number of sample sets are compared with the pre-agreed labels, and reverse derivatives are taken to update the weights in the network. After a large number of iterative updates of the weights, the optimal solution is obtained to achieve the purpose of recognizing the image. The Gabor filter uses Fourier transform to extract features from different directions of the image.

[0130] The method of the embodiment of the present application mainly includes the following stages: a model construction stage, a model training stage and an image recognition stage.

[0131] Figure 7 is a schematic diagram of the implementation process of the model construction stage provided in the embodiment of the present application, such as Figure 7 As shown, the following steps are included:

[0132] Step S701, classify the image.

[0133] Here, image datasets (e.g., ImageNet datasets) and user-collected image collections can be used together to identify the overall features of the image and classify the subject matter of the image. For example, in this application, landscape images can be classified into 15 different categories, mainly divided into two categories: emotions and scenery, including mountains, water, buildings, night scenes, sunsets, stars, plants, roads, snow scenes, flowers, magnificent, mysterious, small, beaches, quiet, etc. These labels can basically cover most of the subject matter in landscape photography. This classification work is completed before image training, and the image types are manually marked.

[0134] Step S702: construct a Gabor filter.

[0135] Adding Gabor filter to the convolutional neural network model enables the convolutional neural network to add recognition of image direction and increase recognition accuracy. Gabor filter is a window function that introduces time local variation by Fourier transform, and obtains windowed Fourier transform. The real number form of Gabor filter is shown in the following formula (1-1):

[0136]

[0137] Wherein, x′=xcosθ+ysinθ, y′=-xsinθ+ycosθ; x, y represent the coordinate values ​​of the Gabor filter on the horizontal and vertical coordinates respectively.

[0138] λ represents wavelength; θ represents direction, ranging from 0 to 360 degrees; ψ represents phase shift, ranging from -180 to 180 degrees; σ represents the standard deviation of the Gabor function Gaussian factor; γ represents the spatial aspect ratio. When γ=1, the shape is circular. When γ<1, the shape is elongated in the direction of parallel stripes, and the value can usually be 0.5.

[0139] In order to extract Gabor features (i.e., directional features) in eight directions, namely, 0, π / 8, π / 4, 3π / 8, π / 2, 5π / 8, 3π / 4, and 7π / 8, the embodiment of the present application constructs Gabor filters of four different scales, each scale differing by 90 degrees and having a phase difference of eight different directions.

[0140] Step S703, constructing a first convolutional neural network model (corresponding to the above-mentioned image category recognition sub-model).

[0141] The classification model here uses a convolutional neural network model that includes three convolutional layers, three pooling layers, and a normalization layer (softmax layer). The three convolutional layers and three pooling layers mainly recognize the local features of the image. The softmax layer contains 15 neurons, which classifies the subject matter of the image after Gabor feature extraction and local feature extraction.

[0142] like Figure 8 As shown, it is a structural schematic diagram of the first convolutional neural network model provided in an embodiment of the present application. The first convolutional neural network model 80 includes an input layer 801, a first convolutional layer 802, a first pooling layer 803, a second convolutional layer 804, a second pooling layer 805, a third convolutional layer 806, a third pooling layer 807, a first fully connected layer 808, a hyperbolic tangent function layer (Tanh layer) 809, a second fully connected layer 810 and a softmax layer 811 connected in sequence.

[0143] Among them, the input layer 801 inputs the feature H×W, and the first convolution layer 802 performs convolution processing on the n features H×W to obtain 2n features H×W; the first pooling layer 803 performs pooling processing on the 2n features H×W to obtain 2n features H / 2×W / 2; the second convolution layer 804 performs convolution processing on the 2n features H / 2×W / 2 to obtain 4n features H / 2×W / 2; the second pooling layer 805 performs pooling processing on the 4n features H / 2×W / 2 to obtain 4n features H / 4×W / 4; the third convolution layer 806 performs pooling processing on the 4n features H / 4 ×W / 4 is convolved to obtain 8n features H / 4×W / 4; the 8n features H / 4×W / 4 are pooled by the third pooling layer 807 to obtain 8n features H / 8×W / 8; the 8n features H / 8×W / 8 are fully connected by the first fully connected layer 808 to obtain 8n features 1×1; the 8n features 1×1 are then successively passed through the Tanh layer 809, the second fully connected layer 810 and the softmax layer 811 to perform hyperbolic tangent function calculation, full connection processing and normalization processing on the 8n features 1×1 to obtain the final features used to determine the image type and output them. n is a positive integer, for example, n can be 32.

[0144] Step S704, construct a second convolutional neural network model (corresponding to the above-mentioned image reconstruction sub-model).

[0145] The second convolutional neural network is mainly divided into two stages, feature extraction stage and image reconstruction stage. The feature extraction stage includes three convolutional layers, and the image reconstruction stage includes three convolutional layers and three upsampling layers.

[0146] like Fig. 9 As shown, it is a structural schematic diagram of the second convolutional neural network model provided in an embodiment of the present application. The second convolutional neural network model 90 includes a feature extraction stage 91 and an image reconstruction stage 92, including an input layer 901, a first convolutional layer 902, a second convolutional layer 903, a first pooling layer 904, a third convolutional layer 905, a second pooling layer 906, a convolutional pooling layer 907, a first upsampling layer 908, a second upsampling layer 909, a fourth convolutional layer 910, a third upsampling layer 911, a fifth convolutional layer 912, a sixth convolutional layer 913 and an output layer 914 connected in sequence.

[0147] Among them, in the feature extraction stage, the input layer 901 inputs m features H×W, and the first convolution layer 902 performs convolution processing on the m features H×W to obtain 2m features H×W; the second convolution layer 903 performs convolution processing on the 2m features H×W to obtain 4m features H×W; the first pooling layer 904 performs pooling processing on the 4m features H×W to obtain 4m features H / 2×W / 2; the third convolution layer 905 performs convolution processing on the 4m features H / 2×W / 2 to obtain 8m features H / 2×W / 2; the second pooling layer 906 performs pooling processing on the 8m features H / 2×W / 2 to obtain 8m features H / 4×W / 4; the convolution pooling layer 907 performs convolution pooling processing on the 8m features H / 4×W / 4 to obtain 16m features H / 8×W / 8, and the feature extraction process is completed.

[0148] In the image reconstruction stage, the 16m features H / 8×W / 8 extracted in the feature extraction stage are upsampled by the first upsampling layer 908 to obtain 8m features H / 4×W / 4; the 8m features H / 4×W / 4 are upsampled by the second upsampling layer 909 to obtain 8m features H / 2×W / 2; the 8m features H / 2×W / 2 are convolved by the fourth convolutional layer 910 to obtain 4m features H / 2×W / 2; the 4m features H / 2×W / 2 are upsampled by the third upsampling layer 911 to obtain 4m features H×W; the 4m features H×W are convolved by the fifth convolutional layer 912 to obtain 2m features H×W; the 2m features H×W are convolved by the sixth convolutional layer 913 to obtain m features H×W; finally, the m features H×W are output by the output layer 914. m is a positive integer. For example, m can be 32.

[0149] Based on the above model construction stage, Fig.10 Schematic diagram of the implementation process of the model training phase provided in the embodiment of the present application, such as Fig.10 As shown, the following steps are included:

[0150] Step S1001, input the sample image into the Gabor filter to extract the directional features, and input the directional features into the first convolutional neural network model to perform category recognition.

[0151] Here, a picture H×W×3 with RGB three channels can be input into the Gabor filter, and after convolution with 4×8×3 filters, a graphic feature (i.e., directional feature) with a thickness of H×W×8 and 8 channels is obtained.

[0152] This directional feature is input into the content recognition convolutional neural network model (i.e., the first convolutional neural network model), and is first convolved with a convolution kernel of size H×3×3, with a sliding step of 1, to obtain an image of size H×W×H; then the image is maximum sampled (pooled) with a sliding step of 2, and the image size is reduced to 1 / 2 of the original, to obtain an image of size H / 2×W / 2×H; two more convolutions and two more pooling operations are performed according to the above steps, and finally the image is output to the softmax layer, to obtain 15 graphic features of size 1×1×1, and the softmax function is used as the activation function for normalization, to finally obtain the category of the image.

[0153] Step S1002: input the sample image into the second convolutional neural network model for image post-processing.

[0154] Here, the second convolutional neural network model is used for image post-processing, and the original piece of RGB channel (i.e., sample image) that has not been post-processed is input into the second convolutional neural network model, which is equivalent to inputting a H×W×3 graph with 3-channel thickness into the network model. For the convolutional neural network, what needs to be done is to predict the distribution of the value of each pixel in the RGB three channels from 0 to 255 according to a certain parameter model. The image processing process here mainly includes two stages, namely the feature extraction stage and the image reconstruction stage. First, the H×W×3 image is convolved with the convolution kernels of 3×3×64 and 3×3×128 to obtain an H×W×128 image. Then it is convolved with a 3×3 convolution kernel with a sliding step of 2 to obtain an H / 2×W / 2×128 image. After three convolution and pooling operations, the image size is H / 8×W / 8×512. This is the feature extraction process. The image reconstruction process uses a 3×3 convolution kernel with a sliding step of 1. After three convolutions and multiple upsampling, the H×W×3 image is regained.

[0155] Step S1003, optimizing the first convolutional neural network model and the second convolutional neural network model.

[0156] In the image training stage, the reconstructed image is compared with the target image (which can be an image that has been manually post-processed), and the weights in the network are updated through back propagation. After multiple iterations, the optimal parameter model is obtained, and the training of the second convolutional neural network is completed.

[0157] It should be noted that the training process of the first convolutional neural network model is to continuously correct the parameter model by predicting the label category of the training image; the training process of the second convolutional neural network model is to correct the parameter model by comparing the content of the training image with the target image. The training processes of the two models are two relatively independent processes.

[0158] In the image recognition stage, the model constructed and trained in the above embodiment can be used for image recognition. In the image recognition process, the image to be recognized is first passed through the first convolutional neural network, and after the image subject recognition is completed, the image is input into the second convolutional neural network model for image post-processing, and finally a picture that has undergone certain post-processing is obtained.

[0159] Fig.11 is a flow chart of the image recognition stage provided in the embodiment of the present application, such as Fig.11 As shown, the following steps are included:

[0160] In step S1101, images crawled from ImageNet and major photography websites are used to form an image training set, and the image training set is input into a Gabor filter to obtain directional features.

[0161] Step S1102, inputting the directional features into the first convolutional neural network model to obtain the category of the image.

[0162] Step S1103, verifying the category of the image according to the verification set for the image category to obtain a parameter model of the first convolutional neural network model.

[0163] Step S1104, obtaining image classification according to the parameter model.

[0164] In step S1105, images crawled from major photography websites are formed into a picture set, and the picture set is input into the second convolutional neural network model to obtain processed image features.

[0165] Step S1106, according to the acquired image classification, the corresponding type parameter model processes the processed image features to obtain a post-processing result.

[0166] Step S1107, outputting the post-processing result.

[0167] The image processing method provided in the embodiment of the present application combines a convolutional neural network with a Gabor filter to perform image recognition, classification and post-processing.

[0168] By combining convolutional neural networks with Gabor filters, the content of images can be better recognized. This is mainly because the introduction of Gabor filters can recognize the directional features of images, making up for the fact that convolutional neural networks can only recognize local features of images, which is similar to the recognition principle of the human eye.

[0169] Since post-processing of photographic images is rarely performed in image processing techniques in related technologies, or the processing methods are relatively simple, post-processing of photographic images through convolutional neural networks is more advantageous than rule-based recognition methods in related technologies.

[0170] Compared with the manually operated photography post-processing technology in the related art, the method of the embodiment of the present application can reduce the time cost of people in learning various post-processing software. By adopting convolutional neural networks to train a large number of image data sets, the purpose of automated post-processing of images is achieved, which can liberate productivity to a certain extent. Moreover, since the convolutional neural network and the Gabor filter can be integrated in the same image processing software, one-click processing of the image to be processed can be achieved.

[0171] Based on the foregoing embodiments, an embodiment of the present application provides an image processing device, which includes the modules included and the components included in each module, and can be implemented by a processor in an image processing device; of course, it can also be implemented by a logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Micro Processor Unit), a digital signal processor (DSP, Digital Signal Processing) or a field programmable gate array (FPGA, Field Programmable Gate Array), etc.

[0172] Fig.12 is a schematic diagram of the composition structure of the image processing device provided in the embodiment of the present application, such as Fig.12 As shown, the image processing device 1200 includes:

[0173] The feature extraction module 1201 is used to extract features from the image to be processed to obtain directional features and image features of the image to be processed;

[0174] The recognition module 1202 is used to recognize the directional feature and obtain the category of the image to be processed;

[0175] An image reconstruction module 1203 is used to perform image reconstruction processing on the image to be processed according to the category and the image feature to obtain a processed image;

[0176] The output module 1204 is used to output the processed image.

[0177] In some embodiments, the directional feature is a local feature of a local area in the image to be processed; the recognition module is also used to: perform convolution processing and pooling processing on the local feature in sequence to obtain a pooling processing feature; perform recognition processing on the pooling processing feature to determine the category corresponding to the pooling processing feature; and determine the category corresponding to the pooling processing feature as the category of the image to be processed.

[0178] In some embodiments, the feature extraction module is also used to: use a filter to perform Fourier transform processing on the image to be processed to obtain the directional features of the image to be processed; use a convolutional neural network to perform feature extraction on the image to be processed to obtain the image features of the image to be processed.

[0179] In some embodiments, the image reconstruction module is further used to: determine the convolutional neural network corresponding to the category as the target convolutional neural network; and use the target convolutional neural network to perform image reconstruction processing on the image features to obtain the processed image.

[0180] In some embodiments, the device also includes: a control module, which is used to use an image processing model to perform image reconstruction processing on the image to be processed; wherein the image processing model is trained through the following steps: inputting a sample image into the image processing model to be trained; performing image processing on the sample image through the image processing model to be trained to obtain the category of the sample image and the processed sample image; inputting the category of the sample image and the processed sample image into a preset loss model to obtain a loss result; according to the loss result, correcting the parameters in the image processing model to be trained to obtain a corrected image processing model.

[0181] In some embodiments, the image processing model to be trained includes a filter and an image category recognition sub-model; the image processing model is trained by the following steps: performing Fourier transform processing on the sample image through the filter to obtain the directional features of the sample image; performing image category recognition processing on the directional features through the image category recognition sub-model to obtain the category of the sample image.

[0182] In some embodiments, the image processing model to be trained also includes: an image reconstruction sub-model corresponding to the category of the sample image; correspondingly, the image processing model is trained through the following steps: performing image reconstruction processing on the sample image through the image reconstruction sub-model to obtain the processed sample image.

[0183] In some embodiments, the image reconstruction submodel includes a feature extraction layer and an image reconstruction processing layer; the image processing model is trained through the following steps: performing feature extraction on the sample image through the feature extraction layer to obtain image features of the sample image; performing image reconstruction processing on the image features through the image reconstruction processing layer to obtain the processed sample image.

[0184] In some embodiments, the image reconstruction processing layer includes a convolution layer and an upsampling layer; the image processing model is trained by the following steps: performing convolution processing on the image features through the convolution layer to obtain convolution processing features; performing upsampling processing on the convolution processing features through the upsampling layer to obtain upsampling features; and determining the processed sample image based on the upsampling features.

[0185] In some embodiments, the apparatus further comprises: an input module, configured to input a preset category corresponding to the sample image and a processed image corresponding to the sample image into the preset loss model;

[0186] Correspondingly, the image processing model is trained through the following steps: determining a first similarity between the category of the sample image and the preset category; determining a second similarity between the processed sample image and the processed image; and determining the loss result based on the first similarity and the second similarity.

[0187] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0188] It should be noted that in the embodiment of the present application, if the above-mentioned image processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for a terminal to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read Only Memory), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0189] Correspondingly, an embodiment of the present application provides an image processing device, Fig.13is a schematic diagram of the composition structure of the image processing device provided in the embodiment of the present application, such as Fig.13 As shown, the image processing device 1300 at least includes: a processor 1301, a communication interface 1302 and a computer-readable storage medium 1303 configured to store executable instructions, wherein the processor 1301 generally controls the overall operation of the image processing device. The communication interface 1302 can enable the image processing device to communicate with other terminals or servers through a network. The computer-readable storage medium 1303 is configured to store instructions and applications executable by the processor 1301, and can also cache data to be processed or processed by each module in the processor 1301 and the image processing device 1300, which can be implemented by flash memory (FLASH) or random access memory (RAM, Random Access Memory).

[0190] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0191] In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0192] The above-mentioned units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment of the present application. It can be understood by ordinary technicians in this field that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, read-only memories, magnetic disks or optical disks. Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a terminal to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage media include: mobile storage devices, ROM, magnetic disks, optical disks, and other media that can store program codes.

Claims

1. An image processing method, characterized in that: include: Using a filter to perform Fourier transform processing on the image to be processed to obtain the directional characteristics of the image to be processed; The directional feature is a local feature of a local area in the image to be processed; Using a convolutional neural network to extract features of the image to be processed to obtain image features of the image to be processed; Performing convolution processing and pooling processing on the local features in sequence to obtain pooling processing features; Performing recognition processing on the pooling processing feature to determine the category corresponding to the pooling processing feature; determining the category corresponding to the pooling processing feature as the category of the image to be processed; According to the category of the image to be processed and the image features, an image processing model is used to perform image reconstruction processing on the image to be processed to obtain a processed image; the image processing model includes the filter, an image category recognition submodel and an image reconstruction submodel corresponding to the category; the image category recognition submodel is used to identify the type of the image to be processed; the image reconstruction submodel includes a feature extraction layer and an image reconstruction processing layer, the image reconstruction processing layer includes a convolution layer and an upsampling layer, and the image reconstruction submodel is used to identify the content of the image to be processed and perform image reconstruction processing on the image to be processed; The processed image is output.

2. The method according to claim 1, characterized in that The method further comprises: Determine the convolutional neural network corresponding to the category as the target convolutional neural network; The target convolutional neural network is used to perform image reconstruction processing on the image features to obtain the processed image.

3. The method according to claim 1, characterized in that The training method of the image processing model comprises the following steps: Input the sample image into the image processing model to be trained; Performing image processing on the sample image by using the image processing model to be trained to obtain the category of the sample image and the processed sample image; Inputting the category of the sample image into a first preset loss model to obtain a first loss result; Inputting the processed sample image into a second preset loss model to obtain a second loss result; According to the first loss result and the second loss result, the parameters in the image processing model to be trained are corrected to obtain a corrected image processing model.

4. The method according to claim 3, characterized in that The image processing model to be trained includes a filter and an image category recognition sub-model; The method further comprises: Performing Fourier transform processing on the sample image through the filter to obtain the directional characteristics of the sample image; The image category recognition sub-model is used to perform image category recognition processing on the directional features to obtain the category of the sample image.

5. The method according to claim 4, characterized in that The image processing model to be trained further includes: an image reconstruction sub-model corresponding to the category of the sample image; Correspondingly, performing image processing on the sample image by using the image processing model to be trained to obtain a processed sample image includes: The sample image is reconstructed by using the image reconstruction sub-model to obtain the processed sample image.

6. The method according to claim 5, characterized in that The image reconstruction sub-model includes a feature extraction layer and an image reconstruction processing layer; The performing image reconstruction processing on the sample image by using the image reconstruction sub-model to obtain the processed sample image includes: Extracting features of the sample image through the feature extraction layer to obtain image features of the sample image; The image reconstruction processing layer performs image reconstruction processing on the image features to obtain the processed sample image.

7. The method according to claim 6, characterized in that The image reconstruction processing layer includes a convolution layer and an upsampling layer; The image reconstruction processing layer performs image reconstruction processing on the image features to obtain the processed sample image, including: Performing convolution processing on the image features through the convolution layer to obtain convolution processing features; Performing upsampling processing on the convolution processing features through the upsampling layer to obtain upsampling features; The processed sample image is determined according to the up-sampling feature.

8. The method according to claim 3, characterized in that The method further comprises: inputting a preset category corresponding to the sample image into the first preset loss model, and inputting a processed image corresponding to the sample image into the second preset loss model; Correspondingly, the inputting the category of the sample image into a first preset loss model to obtain a first loss result includes: Determining a first similarity between the category of the sample image and the preset category by using the first preset loss model; determining the first loss result according to the first similarity; The step of inputting the processed sample image into a second preset loss model to obtain a second loss result includes: Determining a second similarity between the processed sample image and the processed image by using the second preset loss model; The second loss result is determined according to the second similarity.

9. An image processing device, characterized in that: include: A feature extraction module, used for performing Fourier transform processing on the image to be processed using a filter to obtain the directional features of the image to be processed; The directional feature is a local feature of a local area in the image to be processed; a convolutional neural network is used to extract features of the image to be processed to obtain image features of the image to be processed; A recognition module, used for sequentially performing convolution processing and pooling processing on the local features to obtain pooling processing features; Performing recognition processing on the pooling processing feature to determine the category corresponding to the pooling processing feature; determining the category corresponding to the pooling processing feature as the category of the image to be processed; An image reconstruction module, used to perform image reconstruction processing on the image to be processed using an image processing model according to the category of the image to be processed and the image features, so as to obtain a processed image; the image processing model includes the filter, an image category recognition submodel and an image reconstruction submodel corresponding to the category; the image category recognition submodel is used to identify the type of the image to be processed; the image reconstruction submodel includes a feature extraction layer and an image reconstruction processing layer, the image reconstruction processing layer includes a convolution layer and an upsampling layer, and the image reconstruction submodel is used to identify the content of the image to be processed and perform image reconstruction processing on the image to be processed; An output module is used to output the processed image.

10. An image processing device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the method according to any one of claims 1 to 8 when executing the executable instructions stored in the memory.

11. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute and implement the method described in any one of claims 1 to 8.

Images