Image processing method and terminal

Abstract

The present application is applicable to the field of computer technology, and provides an image processing method and terminal, including: obtaining a set of images to be processed; using the non-exposed images in the set of images to be processed as a first set of images; determining similar images in the first set of images based on feature vectors of the images in the first set of images, and removing the similar images to obtain a second set of images; detecting blurred images in the second set of images to obtain a non-blurred image set. In the above manner, the terminal obtains the set of images to be processed, detects the characteristics of each image in the set of images to be processed, quickly and accurately detects the non-exposed images, similar images, and blurred images in the set of images to be processed, and thus improves the speed of cleaning images when cleaning these images.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and in particular, relates to an image processing method and terminal. Background Art

[0002] As the functions of terminal devices become more and more abundant, the images stored in terminal devices are also increasing, and a large amount of image data is generated every day. At the same time, a large number of low-quality images and similar images are also stored in terminal devices; for example, due to the movement of the photographed object, the camera is out of focus, the light is too bright or too dark, the captured image is blurred, underexposed or overexposed. These images occupy a large amount of memory, wasting the storage space of the terminal device.

[0003] In the prior art, these images are detected through a classification network, so that users can quickly clean up the images. However, this detection method itself takes up a lot of memory, runs slowly, is not comprehensive, has a low detection accuracy, and is not convenient for users to clean up images. Summary of the invention

[0004] In view of this, an embodiment of the present application provides an image processing method and terminal to solve the problem that the existing detection method itself occupies a large amount of memory, runs slowly, has incomplete detection, has low detection accuracy, and leads to slow image cleaning speed.

[0005] A first aspect of an embodiment of the present application provides an image processing method, including:

[0006] Get the image set to be processed;

[0007] taking the non-exposed images in the image set to be processed as the first image set;

[0008] Determining similar images in the first image set according to feature vectors of images in the first image set, and removing the similar images to obtain a second image set;

[0009] The blurred images in the second image set are detected to obtain a non-blurred image set.

[0010] Further, the taking the non-exposed images in the to-be-processed image set as the first image set comprises:

[0011] For each image in the to-be-processed image set, reducing the image based on a first preset ratio value to obtain a first target image corresponding to the image;

[0012] Calculating an average value of channel values ​​of a preset channel corresponding to the first target image;

[0013] Based on the average value and a preset exposure threshold, determining a non-exposed image in the set of images to be processed;

[0014] The first image set is generated based on all the non-exposed images in the image set to be processed.

[0015] Further, calculating the average value of the channel value of the preset channel corresponding to the first target image includes:

[0016] Convert the RGB color mode corresponding to the first target image into the HSV color mode;

[0017] An average value of channel values ​​of the V channel corresponding to the first target image is calculated based on the HSV color model.

[0018] Further, determining similar images in the first image set according to feature vectors of images in the first image set, and removing the similar images to obtain the second image set includes:

[0019] For each image in the first image set, reducing the image based on a second preset ratio value to obtain a second target image corresponding to the image;

[0020] Inputting each of the second target images into a neural network model for feature extraction processing to obtain a feature vector corresponding to each of the second target images;

[0021] Calculate a similarity value between any two images in the first image set based on each of the feature vectors;

[0022] Determining similar images in the first image set based on the similarity value and a preset similarity threshold;

[0023] The similar images in the first image set are removed to obtain the second image set.

[0024] Further, the detecting the blurred images in the second image set to obtain the non-blurred image set includes:

[0025] For each image in the second image set, reducing the image based on a third preset ratio value to obtain a reduced image corresponding to the image;

[0026] Calculating a blur value corresponding to the reduced image;

[0027] Determining a blurred image in the second image set based on the blur value and a preset blur threshold;

[0028] All the blurred images in the second image set are filtered to obtain the non-blurred image set.

[0029] Further, the calculating the blur value corresponding to the reduced image includes:

[0030] Performing a convolution operation based on a preset matrix and a matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image;

[0031] Calculate the standard deviation corresponding to the reduced image based on the target matrix;

[0032] The standard deviation is mapped based on a preset function to obtain a blur value corresponding to the reduced image.

[0033] Furthermore, the preset function includes a nonlinear activation function, and mapping the standard deviation based on the preset function to obtain a blur value corresponding to the reduced image includes:

[0034] The standard deviation is input into the nonlinear activation function for calculation to obtain a blur value corresponding to the reduced image.

[0035] Furthermore, in order to facilitate the user to clean up the image, the present application also includes: when an image cleaning request is received, cleaning the image based on the cleaning condition in the image cleaning request.

[0036] A second aspect of an embodiment of the present invention provides an image processing terminal, the terminal comprising:

[0037] An acquisition unit, used for acquiring a set of images to be processed;

[0038] A first processing unit, configured to use the non-exposed images in the image set to be processed as a first image set;

[0039] a second processing unit, configured to determine similar images in the first image set according to feature vectors of images in the first image set, and remove the similar images to obtain a second image set;

[0040] The third processing unit is used to detect blurred images in the second image set to obtain a non-blurred image set.

[0041] Furthermore, the first processing unit includes:

[0042] A first reduction unit, configured to reduce each image in the to-be-processed image set based on a first preset ratio value to obtain a first target image corresponding to the image;

[0043] A calculation unit, used to calculate an average value of channel values ​​of a preset channel corresponding to the first target image;

[0044] A first determining unit, configured to determine a non-exposed image in the set of images to be processed based on the average value and a preset exposure threshold;

[0045] The first filtering unit is configured to generate the first image set based on all the non-exposed images in the image set to be processed.

[0046] Furthermore, the computing unit is specifically used for:

[0047] Convert the RGB color mode corresponding to the first target image into the HSV color mode;

[0048] An average value of channel values ​​of the V channel corresponding to the first target image is calculated based on the HSV color model.

[0049] Furthermore, the second processing unit is specifically configured to:

[0050] For each image in the first image set, reducing the image based on a second preset ratio value to obtain a second target image corresponding to the image;

[0051] Inputting each of the second target images into a neural network model for feature extraction processing to obtain a feature vector corresponding to each of the second target images;

[0052] Calculate a similarity value between any two images in the first image set based on each of the feature vectors;

[0053] Determining similar images in the first image set based on the similarity value and a preset similarity threshold;

[0054] The similar images in the first image set are removed to obtain the second image set.

[0055] Furthermore, the third processing unit includes:

[0056] A second reduction unit, configured to reduce each image in the second image set based on a third preset ratio value to obtain a reduced image corresponding to the image;

[0057] A fuzzy value calculation unit, used to calculate a fuzzy value corresponding to the reduced image;

[0058] A second determining unit, configured to determine a blurred image in the second image set based on the blur value and a preset blur threshold;

[0059] The second filtering unit is used to filter all the blurred images in the second image set to obtain the non-blurred image set.

[0060] Furthermore, the fuzzy value calculation unit includes:

[0061] A convolution operation unit, configured to perform a convolution operation based on a preset matrix and a matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image;

[0062] A standard deviation calculation unit, used for calculating the standard deviation corresponding to the reduced image based on the target matrix;

[0063] A mapping unit is used to map the standard deviation based on a preset function to obtain a blur value corresponding to the reduced image.

[0064] Furthermore, the preset function includes a nonlinear activation function, and the mapping unit is specifically used to: input the standard deviation into the nonlinear activation function for calculation to obtain a blur value corresponding to the reduced image.

[0065] Furthermore, the terminal further includes:

[0066] The cleaning unit is used to clean the image based on the cleaning condition in the image cleaning request when receiving the image cleaning request.

[0067] A third aspect of an embodiment of the present invention provides another image processing terminal, comprising a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are connected to each other, wherein the memory is used to store a computer program that supports the terminal to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to perform the following steps:

[0068] Get the image set to be processed;

[0069] taking the non-exposed images in the image set to be processed as the first image set;

[0070] Determining similar images in the first image set according to feature vectors of images in the first image set, and removing the similar images to obtain a second image set;

[0071] The blurred images in the second image set are detected to obtain a non-blurred image set.

[0072] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0073] Get the image set to be processed;

[0074] taking the non-exposed images in the image set to be processed as the first image set;

[0075] Determining similar images in the first image set according to feature vectors of images in the first image set, and removing the similar images to obtain a second image set;

[0076] The blurred images in the second image set are detected to obtain a non-blurred image set.

[0077] The image processing method and terminal provided by the embodiment of the present application have the following beneficial effects:

[0078] In an embodiment of the present application, the terminal obtains a set of images to be processed; uses the non-exposed images in the set of images to be processed as the first image set; determines similar images in the first image set based on the feature vectors of the images in the first image set, and removes the similar images to obtain a second image set; detects blurred images in the second image set to obtain a non-blurred image set. In the above manner, the terminal obtains the set of images to be processed, detects the characteristics of each image in the set of images to be processed, quickly and accurately detects the non-exposed images, similar images, and blurred images in the set of images to be processed, thereby improving the speed of cleaning images when cleaning these images. This detection method itself occupies little memory and runs fast, which speeds up the speed of image detection; and this detection method has more comprehensive detection, and can quickly and accurately detect various types of images to meet the different needs of users; further, this detection method improves the accuracy of detection, thereby improving the speed of cleaning images when cleaning images. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0080] Figure 1 is a flowchart of an implementation method of an image processing method provided by an embodiment of the present application;

[0081] Figure 2 It is a schematic diagram of a neural network model provided by this application;

[0082] Figure 3 is a flowchart of an implementation method of an image processing method provided by another embodiment of the present application;

[0083] Figure 4 is a schematic diagram of an image processing terminal provided by an embodiment of the present application;

[0084] Figure 5 It is a schematic diagram of an image processing terminal provided by another embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0086] See also Figure 1 , Figure 1 is a schematic flow chart of an image processing method provided by an embodiment of the present invention. The execution subject of the image processing method in this embodiment is a terminal, which includes but is not limited to mobile terminals such as smart phones, tablet computers, personal digital assistants (PDAs), and may also include terminals such as desktop computers. Figure 1 The image processing method shown may include:

[0087] S101: Obtain a set of images to be processed.

[0088] The terminal obtains a set of images to be processed. Specifically, the terminal may obtain the set of images to be processed when detecting an instruction to detect images. The instruction to detect images may be triggered by a user, such as when the user clicks a detection option in the terminal. The set of images to be processed is a collection of multiple images that need to be processed, and the terminal may detect images in the set of images to be processed, for example, detecting overexposed images, underexposed images, similar images, blurred images, etc. The set of images to be processed may be a gallery thumbnail in the terminal; or it may be a set of images to be processed uploaded by a user to the terminal; this is only an exemplary description and is not limited to this.

[0089] S102: Using the non-exposed images in the to-be-processed image set as the first image set.

[0090] The non-exposed image corresponds to the exposed image, and the exposed image may include an over-exposed image and an under-exposed image; wherein the over-exposed image is an image obtained when the film sensitivity is too high, the flash light is too strong, or the exposure time is too long when the image is taken; and the under-exposed image is an image obtained when the film sensitivity is too low, the flash light is too dim, or the exposure time is too short when the image is taken. The remaining images after removing the exposed images from the image set to be processed are non-exposed images, that is, the first image set consists of non-exposed images.

[0091] The terminal detects overexposed images and / or underexposed images in the image set to be processed, filters the detected overexposed images and / or underexposed images from the image set to be processed, and obtains a first image set; specifically, the terminal reduces the images in the image set to be processed according to a preset ratio to obtain a first target image corresponding to the images in the image set to be processed; obtains the RGB color mode corresponding to the first target image, and converts the color mode of the first target image from RGB to HSV; at this time, performs channel splitting on the image after the color mode conversion, calculates and records the average value of the channel value of the V channel corresponding to the first target image; compares the average value corresponding to the first target image with the preset exposure threshold, and determines whether the image corresponding to the average value is an exposed image according to the comparison result. The terminal filters out all exposed images in the image set to be processed to obtain a first image set; that is, the terminal filters out overexposed images in the image set to be processed, and filters out underexposed images in the image set to be processed, and the remaining images constitute the first image set.

[0092] Further, in order to accurately detect whether the image is an overexposed image or an underexposed image, S102 may include S1021-S1024, which are specifically as follows:

[0093] S1021: For each image in the set of images to be processed, reduce the image based on a first preset ratio value to obtain a first target image corresponding to the image.

[0094] For each image in the image set to be processed, the terminal reduces these images based on the first preset ratio value to obtain a first target image corresponding to the image in the image set to be processed. The first preset ratio value can be preset. For example, the first preset ratio value can be 64×64. This is only an exemplary description and is not limited to this. Specifically, the terminal reduces the image in the image set to be processed according to the first preset ratio value to obtain the first target image corresponding to the image in the image set to be processed. For example, when the image set to be processed is a gallery thumbnail, the terminal obtains the image in the gallery thumbnail, reduces the image in the gallery thumbnail based on the first preset ratio value, and obtains the scaled-down image.

[0095] S1022: Calculate an average value of channel values ​​of a preset channel corresponding to the first target image.

[0096] The terminal calculates the average value of the channel value of the preset channel corresponding to the first target image. Specifically, the terminal converts the color mode of the first target image, performs channel splitting on the first target image after the color mode is converted, and calculates and records the average value of the channel value of the preset channel corresponding to the first target image. For example, the initial color mode of the first target image is the RGB color mode, and the RGB color mode of the first target image is converted to the HSV color mode; the first target image converted to the HSV color mode is channel split, and the average value of the channel value of the V channel corresponding to the first target image is calculated and recorded.

[0097] Further, in order to accurately and quickly calculate the average value corresponding to the first target image, S1022 may include S10221-S10222, which are specifically as follows:

[0098] S10221: Convert the RGB color mode corresponding to the first target image into the HSV color mode.

[0099] The terminal converts the RGB color mode corresponding to the first target image into the HSV color mode. Specifically, the terminal obtains the RGB color mode corresponding to the first target image, obtains the channel values ​​corresponding to each channel of the first target image based on the RGB color mode, and converts the RGB color mode into the HSV color mode based on each channel value and the channel conversion method. For example, the color mode conversion can be implemented by the following method:

[0100] R′=R / 255, G′=G / 255, B′=B / 255

[0101] Cmax=max(R′,G′,B′), Cmin=min(R′,G′,B′)

[0102] Δ=Cmax-Cmin

[0103]

[0104]

[0105] V=Cmax, where R, G, and B correspond to the R channel, G channel, and B channel respectively; and H, S, and V correspond to the H channel, S channel, and V channel respectively.

[0106] S10222: Calculate the average value of the channel value of the V channel corresponding to the first target image based on the HSV color model.

[0107] The terminal calculates the average value of the channel value of the V channel corresponding to the first target image based on the HSV color mode. Specifically, the terminal performs channel splitting on the first target image converted into the HSV color mode, splitting it into an H channel, an S channel, and a V channel, and calculates the average value of the channel value of the V channel corresponding to the first target image.

[0108] S1023: Determine a non-exposed image in the set of images to be processed based on the average value and a preset exposure threshold.

[0109] The exposure threshold may include an overexposure threshold and an underexposure threshold; wherein the overexposure threshold is used to determine an overexposed image in the image set to be processed, and the underexposure threshold is used to determine an underexposed image in the image set to be processed. When the terminal detects an overexposed image and an underexposed image in the image set to be processed, the remaining image is a non-exposed image.

[0110] When the average value corresponding to the first target image is greater than or equal to the overexposure threshold, the first target image is an overexposed image. The overexposure threshold can be preset. Specifically, the terminal obtains a preset number of sample images with overexposure image marks or underexposure image marks, calculates the average values ​​corresponding to these sample images by the method in S1021-S1022, and sets the overexposure threshold based on the calculated average values ​​and the image mark corresponding to each sample image. The average value corresponding to the first target image is compared with the preset overexposure threshold, and whether the image corresponding to the average value is an overexposed image is determined according to the comparison result; when the average value is greater than or equal to the overexposure threshold, the image corresponding to the average value is determined to be an overexposed image; when the average value is less than the overexposure threshold, the size between the average value and the underexposure threshold is compared, and whether the image corresponding to the average value is an underexposed image is determined according to the comparison result. Specifically, the terminal compares the size between the average value and the overexposure threshold, and when the average value corresponding to a certain image is greater than the overexposure threshold, the image corresponding to the average value is obtained, and the image is determined to be an overexposed image. For example, the overexposure threshold can be set to 245, and when the average value corresponding to a certain image is greater than 245, the image is determined to be an overexposed image.

[0111] When the average value corresponding to the first target image is less than the overexposure threshold and when the average value corresponding to the first target image is less than the underexposure threshold, the first target image is an underexposure image.

[0112] The underexposure threshold can be set in advance, and the setting method is similar to the overexposure threshold, which will not be repeated here. When the average value is less than the overexposure threshold, the average value is compared with the underexposure threshold, and whether the image corresponding to the average value is an underexposed image is determined based on the comparison result. When the average value is less than the overexposure threshold and the average value is less than the underexposure threshold, the image corresponding to the average value is an underexposed image. Specifically, the terminal compares the average value with the underexposure threshold. When the average value corresponding to a certain image is less than the underexposure threshold, the image corresponding to the average value is obtained, and the image is determined to be an underexposed image. For example, the underexposure threshold can be set to 35. When the average value corresponding to a certain image is less than 35, the image is determined to be an underexposed image.

[0113] The terminal may mark the overexposed images and underexposed images in the image set to be processed, and the remaining unmarked images are non-exposed images.

[0114] S1024: Generate the first image set based on all the non-exposed images in the image set to be processed.

[0115] The terminal filters out all the exposed images in the image set to be processed, and generates a first image set based on the remaining non-exposed images. That is, the terminal filters out the overexposed images in the image set to be processed, and filters out the underexposed images in the image set to be processed, and the remaining images constitute the first image set. The terminal may also mark the exposed images in the image set to be processed, and the remaining unmarked images constitute the first image set. The images in the first image set are all non-exposed images.

[0116] S103: Determine similar images in the first image set according to feature vectors of images in the first image set, and remove the similar images to obtain a second image set.

[0117] The terminal detects similar images in the first image set, filters out these similar images, and the remaining images form the second image set. Specifically, the terminal can extract feature vectors of images in the first image set based on a pre-trained neural network model, and identify similar images in the first image set based on the feature vectors. It is also possible to calculate the histograms corresponding to the images in the first image set, and calculate the similarity values ​​between any two images in the first image set based on each histogram; compare each similarity value with a preset similarity threshold, and determine whether the two images corresponding to this similarity value are similar images according to the comparison result. The terminal filters out similar images in the first image set, and the remaining images form the second image set; it is also possible to mark similar images in the first image set, and unmarked images form the second image set. It is worth noting that the similar images here do not include the original images used as the basis for comparison in two similar images or multiple similar images. For example, the first image set includes image 1, image 2, image 3, image 4, and image 5, wherein image 1 is compared with image 2, image 3, image 4, and image 5 as the basis for comparison, and it is obtained that image 1 is similar to image 2 and image 3, and image 4 and image 5 are not similar to any image in the first image set. Then, image 2 and image 3 are marked as similar images, and the similar images are filtered out. The remaining image 1, image 4, and image 5 form the second image set.

[0118] Further, when similar images are determined based on a pre-trained neural network model, S103 may include S1031-S1035, which are specifically as follows:

[0119] S1031: For each image in the first image set, reduce the image based on a second preset ratio value to obtain a second target image corresponding to the image.

[0120] The terminal reduces each image in the first image set based on the second preset ratio value to obtain a second target image corresponding to each image in the first image set. The second preset ratio value can be preset, and the second preset ratio value can be the same as the first preset ratio value or different, and there is no limitation on this. For example, the second preset ratio value can be 128×128, 400×300, etc. Specifically, the terminal reduces each image in the first image set according to the second preset ratio value to obtain a reduced image of each image in the first image set, that is, obtains a second target image corresponding to each image in the first image set.

[0121] S1032: Input each of the second target images into a neural network model for feature extraction processing to obtain a feature vector corresponding to each of the second target images.

[0122] The neural network model can be pre-trained by this terminal, or pre-trained by other terminals and the corresponding file of the neural network model can be transplanted to this terminal. In other words, the execution subject of training the neural network model and the execution subject of using the neural network model to extract feature vectors can be the same or different. The neural network model that can be used to extract the feature vector of the image can also be downloaded on the Internet, and there is no limitation on the type of the neural network model.

[0123] For example, the neural network model may be a VGG16 model, which may include a convolutional layer, a pooling layer, a fully connected layer, and a softmax layer. Figure 2 As shown in the figure, 224×224×64 represents a convolution layer (conwolution+ReLU) containing 64 convolution kernels, and there are two such convolution layers. 112×112×128 represents a convolution layer containing 128 convolution kernels, and there are two such convolution layers. 56×56×256 represents a convolution layer containing 256 convolution kernels, and there are three such convolution layers. By analogy, the model also includes 28×28×512, 14×14×512, 7×7×512 convolution layers, 1×1×4096, 1×1×1000 fully connected layers (fully connected+ReLU), and the last softmax layer. Among them, each convolution layer of different specifications is connected to a pooling layer (max pooling) to reduce the image size.

[0124] In this embodiment, the second target image can be processed by three convolution layers of 224×224×64, 112×112×128, and 56×56×256 and the pooling layers after these convolution layers. Specifically, the second target image is convolved by the 224×224×64 convolution layer to extract the feature vector, the pooling layer after the convolution layer reduces the size of the second target image, the 112×112×128 convolution layer performs convolution on the reduced size second target image, and so on, and finally outputs the feature vector extracted by the 56×56×256 convolution layer.

[0125] S1033: Calculate a similarity value between any two images in the first image set based on each of the feature vectors.

[0126] The terminal calculates the similarity value between any two images in the first image set based on each feature vector. The similarity value between the two images can be represented by the Pearson correlation coefficient between the feature vectors corresponding to the two images. For example, there are 20 second target images, and the terminal obtains the feature vector corresponding to each second target image, and calculates the Pearson correlation coefficient between the first image and the second image based on the feature vector corresponding to the first image and the feature vector corresponding to the second image; further calculates the Pearson correlation coefficient between the first image and each of the remaining images; further calculates the Pearson correlation coefficient between the second image and each of the remaining images; and calculates the Pearson correlation coefficient between every two images in the first image set in the same way. The Pearson correlation coefficient between two images can be calculated by the following formula:

[0127] Among them, X and Y represent the feature vectors corresponding to the two images, that is, the feature vectors corresponding to the two images whose similarity values ​​need to be calculated, ρ x,y represents the Pearson correlation coefficient between two images, cov(X,Y) represents the covariance of X and Y, σ X represents the standard deviation of X, σ Y represents the standard deviation of Y.

[0128] S1034: Determine similar images in the first image set based on the similarity value and a preset similarity threshold.

[0129] The preset similarity threshold can be set in advance as a standard for determining whether two images are similar. The terminal compares the similarity value with the preset similarity threshold. When the similarity value is greater than the preset similarity threshold, the two images corresponding to the similarity value are determined to be similar images; when the similarity value is less than or equal to the preset similarity threshold, no processing is performed. When the similarity value between two images is represented by the Pearson correlation coefficient, the Pearson correlation coefficient is compared with the preset similarity threshold. When the Pearson correlation coefficient is greater than the preset similarity threshold, the two images corresponding to the Pearson correlation coefficient are determined to be similar images; when the Pearson correlation coefficient is less than or equal to the preset similarity threshold, no processing is performed. For example, the preset similarity threshold can be set to 0.84. When a certain Pearson correlation coefficient is greater than 0.84, the two images corresponding to the Pearson correlation coefficient are determined to be similar images.

[0130] S1035: Remove the similar images from the first image set to obtain the second image set.

[0131] The terminal removes similar images from the first image set, and the remaining images form a second image set. The terminal may also mark all similar images in the first image set, and the unmarked images form the second image set.

[0132] Furthermore, each image in the first image set may be reduced based on a second preset ratio value to obtain a second target image corresponding to each image in the first image set; the histogram corresponding to each first target image is calculated; the similarity value between any two images in the first image set is calculated based on each histogram; the similarity value is compared with a preset similarity threshold, and when the similarity value is greater than the preset similarity threshold, the two images corresponding to the similarity value are determined to be similar images; similar images are filtered out from the first image set to obtain a second image set, or similar images in the first image set are marked, and the unmarked images are combined into a second image set.

[0133] Specifically, the terminal reduces each image in the first image set based on the second preset ratio value to obtain the second target image corresponding to each image in the first image set. The histogram corresponding to each second target image is calculated by a preset histogram function; the histogram corresponding to each second target image can also be calculated by a preset program. The similarity value between any two images in the first image set is calculated based on each histogram, that is, the terminal calculates the similarity value between every two images in the first image set based on each histogram. For example, there are 10 second target images, and the terminal obtains the histogram corresponding to each second target image, calculates the similarity value between the first image and the second image based on the histogram corresponding to the first image and the histogram corresponding to the second image; further calculates the similarity value between the first image and each of the remaining images; further calculates the similarity value between the second image and each of the remaining images; and calculates the similarity value between every two images in the first image set in the same way. Among them, when calculating the similarity value between two images, the second image can be called a comparison image. For example, the terminal can determine the image similarity by calculating the Buffett distance, and specifically calculates the similarity value between two images by the following formula:

[0134] Among them, p(i) represents the histogram corresponding to the input image, and p′(i) represents the histogram corresponding to the image compared with the input image.

[0135] The preset similarity threshold can be set in advance as a standard for determining whether two images are similar. The terminal compares the similarity value with the preset similarity threshold. When the similarity value is greater than the preset similarity threshold, the two images corresponding to the similarity value are determined to be similar images; when the similarity value is less than or equal to the preset similarity threshold, no processing is performed. For example, the preset similarity threshold can be set to 0.145. When a certain similarity value is greater than 0.145, the two images corresponding to the similarity value are determined to be similar images. The terminal filters out similar images from the first image set to obtain a second image set; similar images in the first image set can also be marked, and unmarked images form a second image set. It is worth noting that when the method for determining the similarity of two images is different, the setting of the preset similarity threshold is also different; for example, the preset similarity threshold of the former method can be set to 0.84, and the preset similarity threshold of the latter method can be set to 0.145. The actual implementation shall prevail and there is no limitation on this.

[0136] In this embodiment, the sensitivity of the trained neural network model to the high-level features of the image is utilized to effectively extract the feature vectors in the image, so that semantically similar images can be effectively detected. The Pearson correlation coefficient between the two images is calculated based on the feature vector, and then the Pearson correlation coefficient is compared with a preset similarity threshold to determine whether the two images are similar, thereby improving the accuracy of detecting image similarity.

[0137] S104: Detect blurred images in the second image set to obtain a non-blurred image set.

[0138] The terminal obtains a second image set, detects and filters blurred images in the second image set, and obtains a non-blurred image set. The second image set consists of images remaining after similar images are filtered out in the first image set. Specifically, the terminal reduces each image in the second image set according to a preset ratio to obtain a reduced image corresponding to each image; calculates a blur value corresponding to each reduced image; compares each blur value with a preset blur threshold, and determines whether the image corresponding to this blur value is a blurred image based on the comparison result. In the same way, all blurred images in the second image set are detected, these blurred images are filtered out, and the remaining images constitute a non-blurred image set; it is also possible to mark the blurred images in the second image set, and to make up the non-blurred images from the unmarked images.

[0139] Further, in order to accurately and quickly calculate the blur value corresponding to each image in the second image set, S104 may include S1041-S1044, which are specifically as follows:

[0140] S1041: For each image in the second image set, reduce the image based on a third preset ratio value to obtain a reduced image corresponding to the image.

[0141] For each image in the second image set, the terminal reduces the image in the second image set based on the third preset ratio value, and obtains a reduced image corresponding to the image in the second image set. The third preset ratio value can be set in advance, and the third preset ratio value can be the same as the first preset ratio value and the second preset ratio value, or can be different, and there is no limitation on this. For example, the third preset ratio value can be set to a width-to-height ratio of 4:3. Specifically, the terminal reduces the image in the second image set according to the third preset ratio value, and obtains the image after the image in the second image set is reduced, that is, the reduced image corresponding to the image in the second image set is obtained.

[0142] S1042: Calculate the blur value corresponding to the reduced image.

[0143] The terminal calculates the blur value corresponding to the reduced image. The blur value can be used to indicate the blur degree corresponding to the reduced image. Specifically, the terminal performs edge gradient on the reduced image through the Laplace operator, and then calculates the blur value corresponding to the reduced image. For example, the terminal obtains the matrix corresponding to the reduced image, performs convolution operation on the matrix corresponding to the reduced image based on the preset matrix, and obtains the target matrix corresponding to the reduced image; calculates the standard deviation corresponding to the reduced image based on the target matrix; maps the standard deviation based on the preset function, and obtains the blur value corresponding to the reduced image.

[0144] Further, in order to accurately and quickly calculate the blur value corresponding to the image in the second image set, S1042 may include S10421-S10423, which are specifically as follows:

[0145] S10421: Perform a convolution operation based on a preset matrix and a matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image.

[0146] The terminal performs a convolution operation based on the preset matrix and the matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image. Specifically, the preset matrix may be an L matrix, such as: The terminal performs a convolution operation on the matrix corresponding to the reduced image and the L matrix to obtain a target matrix corresponding to the reduced image.

[0147] S10422: Calculate the standard deviation corresponding to the reduced image based on the target matrix.

[0148] The terminal calculates the standard deviation of the reduced image based on the target matrix. Specifically, D(f)=∑ y ∑ x|G(x,y)|,(G(x,y)>T) calculates the standard deviation of the reduced image; where D(f) represents the cumulative sum function, and G(x,y) represents the grayscale coordinate value of the image at that point. It can be understood as summing up the (x,y) coordinates according to D(f), calculating the standard deviation of the cumulative summation result, and obtaining the standard deviation of the reduced image.

[0149] S10423: Map the standard deviation based on a preset function to obtain a blur value corresponding to the reduced image.

[0150] The terminal maps the standard deviation based on a preset function to obtain a blur value corresponding to the reduced image. Specifically, the preset function may be a nonlinear activation function, through which the standard deviation is mapped within the range of [0, 1] to obtain a blur value corresponding to the reduced image. For example, the calculated standard deviation is input into a nonlinear activation function for calculation to obtain a blur value corresponding to the reduced image.

[0151] Furthermore, the preset function may include a nonlinear activation function, and mapping the standard deviation based on the preset function to obtain the blur value corresponding to the reduced image specifically includes: inputting the standard deviation into the nonlinear activation function for calculation to obtain the blur value corresponding to the reduced image.

[0152] The terminal inputs the calculated standard deviation into a nonlinear activation function for calculation to obtain the blur value corresponding to the reduced image. The nonlinear activation function refers to a function that runs on the neurons of the artificial neural network and is responsible for mapping the input of the neuron to the output. For example, the nonlinear activation function can be a sigmoid function The function maps the standard deviation within the range of [0, 1] to obtain the blur value corresponding to the reduced image. For example, the calculated standard deviation is used as input x to calculate S(x), and the blur value corresponding to the reduced image is obtained.

[0153] S1043: Determine a blurred image in the second image set based on the blur value and a preset blur threshold.

[0154] The preset blur threshold can be preset. Specifically, the terminal obtains a preset number of sample images with blur image marks and clear image marks, calculates the blur values ​​corresponding to these sample images by the method in S1041-S1042, and sets the blur threshold based on the calculated blur values ​​and the image mark corresponding to each sample image. For example, the blur value corresponding to the sample image with the clear image mark is used as the minimum reference value of the blur threshold, and the blur value corresponding to the sample image with the blur image mark is used as the maximum reference value of the blur threshold. The size of the blur threshold is adjusted based on the minimum reference value and the maximum reference value to obtain the final preset blur threshold. The terminal compares the size between the blur value and the preset blur threshold. When the blur value is greater than the preset blur threshold, it determines that the image corresponding to the blur value is a blurred image; when the blur value is less than or equal to the preset blur threshold, no processing is performed. All blurred images in the second image set are detected in the same way.

[0155] S1044: Filter all the blurred images in the second image set to obtain the non-blurred image set.

[0156] The terminal filters out all blurred images in the second image set, and the remaining images form a non-blurred image set. The terminal may also mark all blurred images in the second image set, and the unmarked images form a non-blurred image set.

[0157] In the embodiment of the present application, the terminal obtains a set of images to be processed; the non-exposed images in the set of images to be processed are used as the first set of images; similar images in the first set of images are determined according to the feature vectors of the images in the first set of images, and the similar images are removed to obtain the second set of images; blurred images in the second set of images are detected to obtain a set of non-blurred images. In the above manner, the terminal obtains the set of images to be processed, detects the characteristics of each image in the set of images to be processed, and quickly and accurately detects the non-exposed images, similar images and blurred images in the set of images to be processed, thereby improving the speed of cleaning images when cleaning these images. This detection method itself occupies a small amount of memory, runs fast, and speeds up the speed of image detection; and this detection method has more comprehensive detection, and can quickly and accurately detect various types of images to meet the different needs of users; further, the terminal can also mark the exposed images, similar images and blurred images in the set of images to be processed, and can clear these marked images with one click when cleaning images, thereby improving the speed of cleaning images; and this detection method can accurately detect the exposed images, similar images and blurred images in the set of images to be processed, thereby improving the cleaning accuracy when cleaning images.

[0158] See also Figure 3 , Figure 31 is a schematic flow chart of an image processing method provided by another embodiment of the present invention. The execution subject of the image processing method in this embodiment is a terminal, which includes but is not limited to mobile terminals such as smart phones, tablet computers, personal digital assistants, and may also include terminals such as desktop computers.

[0159] The difference between this embodiment and the previous embodiment is S205. S201-S204 in this embodiment are exactly the same as S101-S104 in the previous embodiment. Please refer to the relevant description of S101-S104 in the previous embodiment for details, which will not be repeated here.

[0160] Furthermore, in order to facilitate the user to clean up the image, S205 may be included after S204, as follows:

[0161] S205: When an image cleaning request is received, cleaning the image based on the cleaning condition in the image cleaning request.

[0162] Specifically, when the terminal obtains an image cleaning request, it obtains the cleaning conditions in the image cleaning request, and cleans the corresponding images according to the cleaning conditions. For example, when the cleaning condition is to clean overexposed images, the terminal obtains the detected overexposed images and deletes these images; when the cleaning condition is to clean underexposed images, the terminal obtains the detected underexposed images and deletes these images; when the cleaning condition is to clean exposed images, the terminal obtains the detected overexposed images and underexposed images, and deletes these images; when the cleaning condition is to clean similar images, the terminal obtains the detected similar images, and checks the similar images to be deleted for deletion. The user can also choose which similar images to check. For example, if there are two images marked as similar images, the user can check one of them, and the terminal deletes the selected similar image; when the cleaning condition is to clean blurred images, the terminal obtains the detected blurred images and deletes them.

[0163] In the embodiment of the present application, the terminal obtains a set of images to be processed; uses the non-exposed images in the set of images to be processed as the first set of images; determines similar images in the first set of images according to the feature vectors of the images in the first set of images, and removes the similar images to obtain a second set of images; detects blurred images in the second set of images to obtain a set of non-blurred images. This detection method itself occupies a small amount of memory and runs fast, which speeds up the speed of image detection; and this detection method has a more comprehensive detection, and can quickly and accurately detect various types of images to meet the different needs of users; further, the terminal can also mark the exposed images, similar images, and blurred images in the set of images to be processed, and can clear these marked images with one click when cleaning the images, thereby improving the speed of cleaning the images; and this detection method can accurately detect the exposed images, similar images, and blurred images in the set of images to be processed, thereby improving the cleaning accuracy when cleaning the images; further, the image cleaning can be performed according to different cleaning conditions in the image cleaning request, and the corresponding images can be cleaned in a targeted manner, thereby meeting the different needs of users and speeding up the speed and accuracy of image cleaning.

[0164] See also Figure 4 , Figure 4 is a schematic diagram of an image processing terminal provided by an embodiment of the present application. The terminal includes units for executing Figure 1 , Figure 3 For details, please refer to the steps in the corresponding embodiment. Figure 1 , Figure 3 For the convenience of explanation, only the parts related to this embodiment are shown. Figure 4 ,include:

[0165] An acquisition unit 310 is used to acquire a set of images to be processed;

[0166] A first processing unit 320, configured to use the non-exposed images in the image set to be processed as a first image set;

[0167] A second processing unit 330 is configured to determine similar images in the first image set according to feature vectors of images in the first image set, and remove the similar images to obtain a second image set;

[0168] The third processing unit 340 is configured to detect blurred images in the second image set to obtain a non-blurred image set.

[0169] Furthermore, the first processing unit 320 includes:

[0170] A first reduction unit, configured to reduce each image in the to-be-processed image set based on a first preset ratio value to obtain a first target image corresponding to the image;

[0171] A calculation unit, used to calculate an average value of channel values ​​of a preset channel corresponding to the first target image;

[0172] A first determining unit, configured to determine a non-exposed image in the set of images to be processed based on the average value and a preset exposure threshold;

[0173] The first filtering unit is configured to generate the first image set based on all the non-exposed images in the image set to be processed.

[0174] Furthermore, the computing unit is specifically used for:

[0175] Convert the RGB color mode corresponding to the first target image into the HSV color mode;

[0176] An average value of channel values ​​of the V channel corresponding to the first target image is calculated based on the HSV color model.

[0177] Furthermore, the second processing unit 320 is specifically configured to:

[0178] For each image in the first image set, reducing the image based on a second preset ratio value to obtain a second target image corresponding to the image;

[0179] Inputting each of the second target images into a neural network model for feature extraction processing to obtain a feature vector corresponding to each of the second target images;

[0180] Calculate a similarity value between any two images in the first image set based on each of the feature vectors;

[0181] Determining similar images in the first image set based on the similarity value and a preset similarity threshold;

[0182] The similar images in the first image set are removed to obtain the second image set.

[0183] Further, the third processing unit 330 includes:

[0184] A second reduction unit, configured to reduce each image in the second image set based on a third preset ratio value to obtain a reduced image corresponding to the image;

[0185] A fuzzy value calculation unit, used to calculate a fuzzy value corresponding to the reduced image;

[0186] A second determining unit, configured to determine a blurred image in the second image set based on the blur value and a preset blur threshold;

[0187] The second filtering unit is used to filter all the blurred images in the second image set to obtain the non-blurred image set.

[0188] Furthermore, the fuzzy value calculation unit includes:

[0189] A convolution operation unit, configured to perform a convolution operation based on a preset matrix and a matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image;

[0190] A standard deviation calculation unit, used for calculating the standard deviation corresponding to the reduced image based on the target matrix;

[0191] A mapping unit is used to map the standard deviation based on a preset function to obtain a blur value corresponding to the reduced image.

[0192] Furthermore, the preset function includes a nonlinear activation function, and the mapping unit is specifically used to: input the standard deviation into the nonlinear activation function for calculation to obtain a blur value corresponding to the reduced image.

[0193] Furthermore, the terminal further includes:

[0194] The cleaning unit is used to clean the image based on the cleaning condition in the image cleaning request when receiving the image cleaning request.

[0195] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of an image processing terminal provided by another embodiment of the present application. Figure 5 As shown, the terminal 4 of this embodiment includes: a processor 40, a memory 41, and a computer-readable instruction 42 stored in the memory 41 and executable on the processor 40. When the processor 40 executes the computer-readable instruction 42, the steps in the above-mentioned various image processing method embodiments are implemented, for example Figure 1 Alternatively, when the processor 40 executes the computer-readable instructions 42, the functions of each unit in the above embodiments are realized, for example, Figure 4 The functions of units 310 to 340 are shown.

[0196] Exemplarily, the computer-readable instructions 42 may be divided into one or more units, which are stored in the memory 41 and executed by the processor 40 to complete the present application. The one or more units may be a series of computer-readable instruction segments capable of completing specific functions, which are used to describe the execution process of the computer-readable instructions 42 in the terminal 4. For example, the computer-readable instructions 42 may be an acquisition unit, a first processing unit, a second processing unit, and a third processing unit, and the specific functions of each unit are as described above.

[0197] The terminal may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 5 It is only an example of terminal 4 and does not constitute a limitation on terminal 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include an input and output terminal, a network access terminal, a bus, etc.

[0198] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0199] The memory 41 may be an internal storage unit of the terminal 4, such as a hard disk or memory of the terminal 4. The memory 41 may also be an external storage terminal of the terminal 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 4. Further, the memory 41 may also include both an internal storage unit of the terminal 4 and an external storage terminal. The memory 41 is used to store the computer-readable instructions and other programs and data required by the terminal. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0200] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for image processing, It is characterized in that include: Get the image set to be processed; The non-exposed images of the to be processed image set are used as the first image set; Determine a similar image in the first image set based on a feature vector of an image of the first image set, and remove the similar image to obtain a second image set; Detect the blurred images in the second image set to obtain a non-fuzzy image set; The detecting blurred images in the second image set to obtain the non-blurred image set includes: reducing each image in the second image set according to a preset ratio value to obtain a reduced image corresponding to each image; for each reduced image, performing a convolution operation based on a preset matrix and a matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image; calculating a standard deviation corresponding to the reduced image based on the target matrix; mapping the standard deviation based on a preset function to obtain a blur value corresponding to the reduced image; determining blurred images in the second image set based on the blur value corresponding to each reduced image and a preset blur threshold; filtering all blurred images in the second image set to obtain the non-blurred image set; When an image cleaning request is received, the image is cleaned based on a cleaning condition in the image cleaning request; the cleaning condition includes at least one of cleaning an exposed image, cleaning a similar image, and cleaning a blurred image, and the exposed image includes an overexposed image and an underexposed image.

2. The method according to claim 1, It is characterized in that The non-exposed images in the to be processed image set include as the first image set: For each image in the set of images to be processed, the image is reduced based on the first preset ratio value to obtain a first target image corresponding to the image; Calculate an average value of the channel values ​​of the preset channel corresponding to the first target image; Based on the average value and a preset exposure threshold, the non-exposed image in the to be processed image set is determined; The first image set is generated based on all the non-exposed images in the to be processed image set.

3. The method according to claim 2, It is characterized in that The average value of the channel value of the preset channel corresponding to the first target image includes: The RGB color mode corresponding to the first target image is converted to the HSV color mode; The average value of the channel values ​​of the V channel corresponding to the first target image is calculated based on the HSV color mode.

4. The method according to claim 1, It is characterized in that The determination of similar images in the first image set based on the feature vector of the images of the first image set, and removing the similar images to obtain a second image set includes: For each image in the first image set, the image is reduced based on the second preset ratio value to obtain a second target image corresponding to the image; Each of the second target image is input to the neural network model for feature extraction processing, and a feature vector corresponding to each of the second target image is obtained; A similarity value between any two images in the first image set is calculated based on each of the feature vectors; Based on the similarity value and a preset similarity threshold, a similar image in the first image set is determined; The similar images in the first image set are removed to obtain the second image set.

5. The method according to claim 1, It is characterized in that The preset function includes a nonlinear activation function, the mapping of the standard deviation based on the preset function, and obtaining the blur value corresponding to the reduced image includes: The standard deviation is input to the nonlinear activation function for calculation, and the blur value corresponding to the reduced image is obtained.

6. An image processing terminal, It is characterized in that include: An acquisition unit, used for acquiring a set of images to be processed; A first processing unit is configured to use the non-exposed images in the to-processed image set as the first image set; The second processing unit is configured to determine a similar image in the first image set based on a feature vector of the images of the first image set, and remove the similar image to obtain a second image set; The third processing unit is configured to detect the blurred image in the second image set and obtain the non-fuzzy image set; The detecting blurred images in the second image set to obtain the non-blurred image set includes: reducing each image in the second image set according to a preset ratio value to obtain a reduced image corresponding to each image; for each reduced image, performing a convolution operation based on a preset matrix and a matrix corresponding to the reduced image to obtain a target matrix corresponding to the reduced image; calculating a standard deviation corresponding to the reduced image based on the target matrix; mapping the standard deviation based on a preset function to obtain a blur value corresponding to the reduced image; determining blurred images in the second image set based on the blur value corresponding to each reduced image and a preset blur threshold; filtering all blurred images in the second image set to obtain the non-blurred image set; When an image cleaning request is received, the image is cleaned based on a cleaning condition in the image cleaning request; the cleaning condition includes at least one of cleaning an exposed image, cleaning a similar image, and cleaning a blurred image, and the exposed image includes an overexposed image and an underexposed image.

7. An image processing terminal, including a memory, a processor, and a computer program stored in the memory and can be run on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.

Images