Picture screening method and device

Abstract

The application relates to the technical field of image processing, and discloses a picture screening method and device, aiming to solve the problems of low efficiency, low accuracy and poor picture quality of the existing picture screening mode, and mainly comprising the following steps: determining picture similarities between pictures in a picture library, dividing the pictures into multiple similar picture groups according to the picture similarities; determining picture qualities of the pictures, determining representative pictures corresponding to each similar picture group according to the picture qualities; performing face recognition on the representative pictures, screening face pictures containing preset faces from the representative pictures according to the face recognition results; performing motion blur detection on the face pictures, and removing the face pictures with a motion blur confidence score greater than a second threshold; and performing expression detection on the remaining face pictures, and screening face pictures with expressions meeting preset conditions from the remaining face pictures. The application improves the efficiency, accuracy and quality of picture screening, and is suitable for batch picture screening.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to an image filtering method and apparatus. Background Technology

[0002] To select suitable facial images from an image library, existing technologies typically employ two methods: The first is manual selection from the image library, which is extremely time-consuming and labor-intensive, resulting in low efficiency. The second method utilizes computer vision and deep learning technologies to analyze and process images, automatically displaying similar or matching facial images from the library. However, this method has a relatively limited approach to image analysis and processing, failing to meet the diverse needs of facial image selection. For more demanding selection requirements, further manual screening is still necessary, which is also very time-consuming and labor-intensive, and results in lower accuracy and image quality. Summary of the Invention

[0003] This invention aims to address the shortcomings of existing image filtering methods, which cannot meet the diverse needs of face image filtering and suffer from low efficiency, inaccuracy, and poor image quality. It proposes an image filtering method and apparatus.

[0004] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0005] On the one hand, an image filtering method is proposed, the method comprising:

[0006] Determine the image similarity between images in the image library, and divide the images into multiple similar image groups based on the image similarity. In each similar image group, each image has an image similarity greater than a first threshold with at least one other image in the same similar image group.

[0007] Determine the image quality of each image, and based on the image quality, determine the representative image corresponding to each group of similar images;

[0008] Perform facial recognition on each representative image, and select facial images containing preset faces from the representative images based on the facial recognition results;

[0009] Motion blur detection is performed on each face image, and face images with a motion blur confidence score greater than the second threshold are removed;

[0010] The remaining face images are subjected to expression detection, and face images whose expressions meet the preset conditions are selected from the remaining face images.

[0011] Furthermore, determining the image similarity between images in the image library specifically includes:

[0012] Based on the image similarity detection algorithm, the structural similarity and color similarity between each image in the image library and each other image are determined. The image similarity is determined according to the structural similarity and color similarity. Images with an image similarity greater than a first threshold are added to the same similar image group.

[0013] Furthermore, the similarity detection algorithm is a structural similarity algorithm or a mean squared error algorithm.

[0014] Furthermore, the image similarity is obtained by weighting the structural similarity and color similarity.

[0015] Furthermore, determining the image quality of each image specifically includes:

[0016] Image quality is determined based on one or more of the following metrics: peak signal-to-noise ratio, structural similarity index, visual information fidelity, and image naturalness.

[0017] Furthermore, the facial recognition of each representative image specifically includes:

[0018] A face recognition model is created to detect whether a preset face is included. Representative images are input into the face recognition model to obtain the face recognition results of the representative images.

[0019] Furthermore, the creation of the face recognition model for detecting whether a preset face is included specifically includes:

[0020] Obtain a set of face images, determine the face information of each image in the set, and perform image enhancement, normalization, cropping, and rotation processing on each image in the set.

[0021] The processed set of face images and the face information corresponding to each image are used as training samples to train a deep learning model for recognizing face information, and a face recognition model is created based on the deep learning model.

[0022] Furthermore, the motion blur detection for each face image specifically includes:

[0023] Obtain a set of motion-blurred images and determine the motion blur confidence score of each image in the set.

[0024] The motion-blurred image set and the motion-blur confidence score corresponding to each image are used as training samples to train a motion-blurred detection model for motion-blurred detection. Each face image is input into the motion-blurred detection model to obtain the motion-blurred detection result of the face image.

[0025] Furthermore, the expression detection for the remaining facial images specifically includes:

[0026] Obtain a set of emoticon images and determine the emoticon type of each image in the set;

[0027] The set of facial expression images and the corresponding facial expression types for each image are used as training samples to train an facial expression recognition model for facial expression detection. The remaining facial images are then input into the facial expression recognition model to obtain the facial expression detection results.

[0028] On the other hand, an image filtering device is proposed, the device comprising:

[0029] The similarity detection unit is used to determine the image similarity between images in the image library, and to divide the images into multiple similar image groups based on the image similarity. Each image in each similar image group has an image similarity greater than a first threshold with at least one other image in the same similar image group.

[0030] A quality detection unit is used to determine the image quality of each image and, based on the image quality, determine the representative image corresponding to each group of similar images.

[0031] The face recognition unit is used to perform face recognition on each representative image and to filter out face images containing preset faces from the representative images based on the face recognition results.

[0032] The motion blur detection unit is used to perform motion blur detection on each face image and remove face images whose motion blur confidence score is greater than the second threshold.

[0033] The expression detection unit is used to detect expressions in the remaining face images and select face images whose expressions meet preset conditions from the remaining face images.

[0034] The beneficial effects of the present invention are as follows: The image screening method and apparatus of the present invention use a variety of image processing and deep learning models to automatically screen and filter out face images that meet the requirements, realize the automated and accurate screening and optimization processing of a large number of images, avoid the tediousness and time-consuming nature of manual screening, improve screening efficiency and quality, and can also perform customized processing for different image screening needs, with high flexibility and adaptability, and can meet diverse face image screening needs. Attached Figure Description

[0035] Figure 1 This is a schematic flowchart of the image filtering method described in an embodiment of the present invention;

[0036] Figure 2 This is another flowchart illustrating the image filtering method described in an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the image filtering device according to an embodiment of the present invention. Detailed Implementation

[0038] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] This invention aims to improve the efficiency, accuracy, and quality of image filtering. It proposes an image filtering method and apparatus, the main technical solutions of which include: determining the image similarity between images in an image library; dividing the images into multiple similar image groups based on the image similarity, wherein each image in each similar image group has an image similarity greater than a first threshold with at least one other image in the same similar image group; determining the image quality of each image; determining representative images corresponding to each similar image group based on the image quality; performing face recognition on each representative image; filtering out face images containing preset faces from the representative images based on the face recognition results; performing motion blur detection on each face image; removing face images with a motion blur confidence score greater than a second threshold; and performing expression detection on the remaining face images; filtering out face images whose expressions meet preset conditions from the remaining face images.

[0040] In practical applications, users pre-determine their requirements for filtering facial images. These requirements can include the faces and expressions to be filtered. Once the requirements are determined, high-quality facial images that meet the criteria can be automatically filtered from the image library. Specifically, this invention first divides all images in the image library into multiple similar image groups based on image similarity. Images in each similar image group are determined to be similar. Then, a representative image of the highest quality is selected from each similar image group. Facial recognition is performed on all representative images to filter out facial images that meet the filtering requirements. Next, for the filtered facial images, images with blurred faces due to motion are removed to further improve the quality of the facial images. Finally, expression detection is performed on the remaining facial images to filter out facial images that meet the filtering requirements.

[0041] Example

[0042] Please see Figure 1 The image filtering method described in this embodiment of the invention includes the following steps:

[0043] Step 1: Determine the image similarity between images in the image library, and divide the images into multiple similar image groups based on the image similarity. In each similar image group, each image has an image similarity greater than a first threshold with at least one other image in the same similar image group.

[0044] Please see Figure 2In practical applications, this embodiment requires the user to pre-determine the filtering requirements for face images. The filtering requirements may include the faces and expressions to be filtered. After the filtering requirements are determined, high-quality face images that meet the filtering requirements can be automatically filtered from the image library.

[0045] The images in the image library are pre-acquired, for example, by downloading a video to the local machine, extracting frames (screenshots) from the video locally to obtain a batch of images, and then adding the images to the image library.

[0046] Specifically, this embodiment uses an image similarity detection algorithm to determine the structural similarity and color similarity between each image in the image library and every other image. The weighted sum of the structural similarity and color similarity is used as the image similarity. Images with a similarity greater than a first threshold are added to the same similar image group. This allows the images to be divided into multiple similar image groups, and the images in each similar image group are determined to be similar images.

[0047] In this embodiment, open-source libraries such as OpenCV can be used for image processing and to calculate the similarity between images. Alternatively, structural similarity algorithms (SSIM) or mean squared error algorithms (MSE) can be used to calculate the similarity between two images. These algorithms can measure the structural and color similarity between two images, and the final image similarity is obtained by weighted summing of the structural and color similarities. A high image similarity indicates that the two images are very similar in content, possibly different versions of the same image, while a low similarity indicates that the two images are very different in content. When the similarity is greater than a preset first threshold, the images are classified as similar and placed in the same similar image group. By analogy, after traversing all images in the image library pairwise in the above manner, multiple similar image groups can be obtained.

[0048] The first threshold can be set according to the actual situation. This embodiment does not limit it. For example, the first threshold is 0.8. That is, when the image similarity of two images is greater than 0.8, it means that the two images are similar and are temporarily placed in a similar image group.

[0049] Step 2: Determine the image quality of each image, and determine the representative image corresponding to each group of similar images based on the image quality.

[0050] In this embodiment, the image quality of each image can be determined based on indicators such as peak signal-to-noise ratio, structural similarity index, visual information fidelity, and image naturalness. For each group of similar images, the image with the best quality is selected as the representative image of this group.

[0051] Among them, PSNR (Peak Signal-to-Noise Ratio) is a widely used method for evaluating image quality. It is calculated by comparing the signal-to-noise ratio between the original image and the compressed image to assess the image quality.

[0052] SSIM (Structural Similarity Index): The structural similarity index is an evaluation method based on image structural information. It assesses image quality by comparing the structural similarity between the original image and the compressed image.

[0053] VIF (Visual Information Fidelity) is a perceptual quality assessment method that evaluates image quality by comparing the visual information fidelity between the original image and the compressed image.

[0054] NIQE (Natural Image Quality Evaluator) is an image quality assessment method based on the naturalness of an image. It evaluates the quality of an image by comparing the naturalness between the original image and the compressed image.

[0055] In addition, there are other objective evaluation methods, such as FSIM (Feature Similarity), IFC (Image Fidelity Criterion), Laplacian gradient method, etc. Each evaluation method has its applicable scenarios and characteristics. In practical applications, one or more appropriate methods can be selected for evaluation according to actual needs. This embodiment does not impose any restrictions on this.

[0056] Step 3: Perform face recognition on each representative image, and select face images containing preset faces from the representative images based on the face recognition results;

[0057] This embodiment can create a face recognition model for detecting whether a preset face is included. By inputting representative images into the face recognition model, the face recognition results of the representative images can be obtained.

[0058] The steps involved in creating a face recognition model may include:

[0059] A set of face images is acquired, the facial information of each image in the set is determined, and image enhancement, normalization, cropping, and rotation are performed on each image in the set. The processed set of face images and the facial information corresponding to each image are used as training samples to train a deep learning model for recognizing facial information. A face recognition model is then created based on the deep learning model.

[0060] In practical applications, a set of labeled face image sets can be prepared. The images in the face image sets can be obtained from the Internet or collected manually. For each image in the face image sets, the face information is labeled. The face information includes the position of the face and the name of the face. These labels can be completed manually or using automated tools. Then, preprocessing is performed on each image in the face image sets, including operations such as image enhancement, normalization, cropping, and rotation. Then, the face image sets and their corresponding face information are divided into a training set and a test set. A deep learning model for identifying face information is trained through the training set, and the accuracy of the deep learning model is verified through the test set. When the accuracy meets the conditions, a face recognition model can be created using the deep learning model. The face recognition model can determine whether the input image contains a preset face.

[0061] For example, assume that a face recognition model has been pre-trained and the data information of multiple faces has been entered into the database. The face recognition model can identify the number of qualified faces (front faces) and compare with the data in the database to obtain the most similar face similarity (between 0 and 1). Assume that the screening requirement is to filter out all images containing "A". Then, the images screened in the previous step are input into the face recognition model one by one. The face recognition model will return whether "A" is contained in each image (if contained, it will return the probability of "A"). If 0.735 is set as the threshold, if the similarity exceeds this value and the number of faces ≤ 3 (of course, the number of faces can also be not restricted), the face image is retained.

[0062] Step 4: Perform motion blur detection on each face image, and remove the face images with a motion blur confidence score greater than the second threshold;

[0063] In this embodiment, a motion blur detection model can be created. By inputting each face image into the motion blur detection model, the motion blur detection results of the face images can be obtained, and then it can be judged which faces in the images are blurred due to motion of the human body. These face images are removed from the list, thereby further improving the quality of image screening.

[0064] Among them, the steps of creating a motion blur detection model can include:

[0065] Obtain a motion blur image set and determine the motion blur confidence scores of each image in the motion blur image set; use the motion blur image set and the motion blur confidence scores corresponding to each image as training samples to train a motion blur detection model for motion blur detection

[0066] In this embodiment, a deep learning framework such as TensorFlow or PyTorch can be used to train the model. The training data can come from labeled face and human motion-blurred images in a motion-blurred image set; or it can be implemented through a convolutional neural network (CNN) model, such as the "BlurDetection" model implemented based on TensorFlow, which can perform motion blur detection on the input image and return a motion blur confidence score.

[0067] The second threshold can be set according to the actual situation, and this embodiment does not impose any restrictions on it.

[0068] Step 5: Perform expression detection on the remaining face images and filter out face images whose expressions meet the preset conditions.

[0069] This embodiment can create an expression recognition model, input the remaining face images into the expression recognition model, obtain the expression detection results of the face images, and then filter out face images whose expressions meet the filtering requirements. At the same time, it can also filter out face images with exaggerated expressions, further improving the quality of image filtering.

[0070] In this embodiment, the facial expression recognition model can be implemented using the HaarCascade classifier model provided in OpenCV, which can detect faces and recognize facial expression features. Alternatively, a convolutional neural network (CNN) can be trained to achieve this function. The specific steps are as follows:

[0071] Obtain a set of facial expression images and determine the facial expression type of each image in the set; use the set of facial expression images and the corresponding facial expression types of each image as training samples to train an facial expression recognition model for facial expression detection.

[0072] Specifically, the first step is to search for publicly available facial expression image sets on the internet, such as FER2013. These image sets need to contain facial expression images of different people, along with labels for each image indicating the type of expression. Then, for each expression image, features need to be extracted for subsequent classification or recognition. Common feature extraction methods can be used, such as Local Binary Pattern Recognition (LBP), Histogram of Oriented Gradients (HOG), and facial landmarks. Next, the labeled facial expression image set can be divided into training and testing samples. The model is trained using the training samples and then evaluated using the testing samples, calculating metrics such as accuracy, recall, and F1 score. Once the evaluation metrics meet the requirements, the trained facial expression recognition model can be applied to real-world scenarios. This model can classify expressions in images, selecting those that meet the requirements while filtering out exaggerated expressions, such as "angry," "twitching," or "making faces."

[0073] In summary, the image filtering method and apparatus provided in this embodiment automatically filter and select facial images that meet the requirements using various image processing and deep learning models. This achieves automated and accurate filtering and optimization of large batches of images, avoiding the tediousness and time-consuming nature of manual filtering, and improving filtering efficiency and quality. At the same time, it can also be customized for different image filtering needs, exhibiting high flexibility and adaptability, and can meet diverse facial image filtering requirements.

[0074] Please see Figure 3 Based on the above technical solution, this embodiment also provides an image filtering device, the device comprising:

[0075] The similarity detection unit is used to determine the image similarity between images in the image library, and to divide the images into multiple similar image groups based on the image similarity. Each image in each similar image group has an image similarity greater than a first threshold with at least one other image in the same similar image group.

[0076] A quality detection unit is used to determine the image quality of each image and, based on the image quality, determine the representative image corresponding to each group of similar images.

[0077] The face recognition unit is used to perform face recognition on each representative image and to filter out face images containing preset faces from the representative images based on the face recognition results.

[0078] The motion blur detection unit is used to perform motion blur detection on each face image and remove face images whose motion blur confidence score is greater than the second threshold.

[0079] The expression detection unit is used to detect expressions in the remaining face images and select face images whose expressions meet preset conditions from the remaining face images.

[0080] It is understood that since the image filtering device described in the embodiments of the present invention is a device for implementing the image filtering method described in the embodiments, the device disclosed in the embodiments is described in a simpler way because it corresponds to the method disclosed in the embodiments. For relevant parts, please refer to the description of the method.

Claims

1. An image filtering method, characterized in that, The method includes: Determine the image similarity between images in the image library, and divide the images into multiple similar image groups based on the image similarity. In each similar image group, each image has an image similarity greater than a first threshold with at least one other image in the same similar image group. Determine the image quality of each image, and based on the image quality, determine the representative image corresponding to each group of similar images; Perform facial recognition on each representative image, and select facial images containing preset faces from the representative images based on the facial recognition results; Motion blur detection is performed on each face image, and face images with a motion blur confidence score greater than the second threshold are removed; The remaining face images are subjected to expression detection, and face images whose expressions meet the preset conditions are selected from the remaining face images.

2. The image filtering method as described in claim 1, characterized in that, Determining the image similarity between images in the image library specifically includes: Based on the image similarity detection algorithm, the structural similarity and color similarity between each image in the image library and each other image are determined. The image similarity is determined according to the structural similarity and color similarity. Images with an image similarity greater than a first threshold are added to the same similar image group.

3. The image filtering method as described in claim 2, characterized in that, The similarity detection algorithm is either a structural similarity algorithm or a mean squared error algorithm.

4. The image filtering method as described in claim 2, characterized in that, The image similarity is obtained by weighting the structural similarity and color similarity.

5. The image filtering method as described in claim 1, characterized in that, Determining the image quality of each image specifically includes: Image quality is determined based on one or more of the following metrics: peak signal-to-noise ratio, structural similarity index, visual information fidelity, and image naturalness.

6. The image filtering method as described in claim 1, characterized in that, The facial recognition process for each representative image specifically includes: A face recognition model is created to detect whether a preset face is included. Representative images are input into the face recognition model to obtain the face recognition results of the representative images.

7. The image filtering method as described in claim 6, characterized in that, The creation of the face recognition model for detecting whether a preset face is included specifically includes: Obtain a set of face images, determine the face information of each image in the set, and perform image enhancement, normalization, cropping, and rotation processing on each image in the set. The processed set of face images and the face information corresponding to each image are used as training samples to train a deep learning model for recognizing face information, and a face recognition model is created based on the deep learning model.

8. The image filtering method as described in claim 1, characterized in that, The motion blur detection for each face image specifically includes: Obtain a set of motion-blurred images and determine the motion blur confidence score of each image in the set. The motion-blurred image set and the motion-blur confidence score corresponding to each image are used as training samples to train a motion-blurred detection model for motion-blurred detection. Each face image is input into the motion-blurred detection model to obtain the motion-blurred detection result of the face image.

9. The image filtering method as described in claim 1, characterized in that, The expression detection for the remaining facial images specifically includes: Obtain a set of emoticon images and determine the emoticon type of each image in the set; The set of facial expression images and the corresponding facial expression types for each image are used as training samples to train an facial expression recognition model for facial expression detection. The remaining facial images are then input into the facial expression recognition model to obtain the facial expression detection results.

10. An image filtering device, characterized in that, The device includes: The similarity detection unit is used to determine the image similarity between images in the image library, and to divide the images into multiple similar image groups based on the image similarity. Each image in each similar image group has an image similarity greater than a first threshold with at least one other image in the same similar image group. A quality detection unit is used to determine the image quality of each image and, based on the image quality, determine the representative image corresponding to each group of similar images. The face recognition unit is used to perform face recognition on each representative image and to filter out face images containing preset faces from the representative images based on the face recognition results. The motion blur detection unit is used to perform motion blur detection on each face image and remove face images whose motion blur confidence score is greater than the second threshold. The expression detection unit is used to detect expressions in the remaining face images and select face images whose expressions meet preset conditions from the remaining face images.

Images