Image processing method, device and computer readable storage medium

Abstract

An image processing method, device and computer-readable storage medium, wherein the method comprises: obtaining a face image to be detected, extracting a target face area image from the face image to be detected; processing the target face area image using a first feature extraction network and a second feature extraction network included in a scoring model to obtain a scoring result for the target face area image; wherein the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process, the target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold. The above method can effectively improve the efficiency and accuracy of facial image scoring.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an image processing method, device and computer-readable storage medium. Background Art

[0002] In recent years, with the development of face recognition technology, face image scoring has received extensive attention and research. At present, face images are usually scored manually, such as in the scene of talent shows. However, the manual scoring method is inefficient, and due to human subjectivity, the accuracy of face scoring results is not high. Therefore, how to improve the efficiency and accuracy of face image scoring is a current research hotspot.

[0003] Application Contents

[0004] The embodiments of the present application provide an image processing method, device, and computer-readable storage medium, which can effectively improve the efficiency and accuracy of facial image scoring.

[0005] On the one hand, an embodiment of the present application discloses an image processing method, the method comprising:

[0006] Acquire a face image to be detected, and extract a target face area image from the face image to be detected;

[0007] Then, the target face region image is processed using the first feature extraction network and the second feature extraction network included in the scoring model to obtain a scoring result of the target face region image;

[0008] Among them, the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process. The target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process. The weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold.

[0009] In one embodiment, extracting the target face region image from the face image to be detected includes:

[0010] Extracting an initial face region image from the face image to be detected;

[0011] Filling the initial face region image to obtain a face region image with a target aspect ratio;

[0012] The face region image with the target aspect ratio is scaled, and the scaled face region image is used as the target face region image.

[0013] In one embodiment, the first feature extraction network and the second feature extraction network included in the scoring model are used to process the target face area image to obtain a scoring result of the target face area image, including:

[0014] Using the first feature extraction network to extract features from the target face region image to obtain a first feature vector;

[0015] Using the second feature extraction network to extract features from the target face region image to obtain a second feature vector;

[0016] The first feature vector and the second feature vector are processed to obtain a scoring result of the target face area image.

[0017] In one embodiment, the scoring model further includes a scoring network, and the processing of the first feature vector and the second feature vector to obtain a scoring result of the target face region image includes:

[0018] Fusing the first feature vector and the second feature vector to obtain a fused feature vector;

[0019] The fused feature vector is processed using the scoring network to obtain a scoring result of the target face area image.

[0020] In one embodiment, the image processing method further includes: acquiring a training data set, wherein the training data set includes a plurality of training images and a reference score value of each training image;

[0021] Processing the plurality of training images using the initial scoring model to obtain a predicted scoring value for each training image;

[0022] Determine an error parameter and a weighted loss parameter based on the reference score value and the predicted score value of each training image, and determine a target loss value based on the error parameter and the weighted loss parameter;

[0023] One or more of the first feature extraction network and the second feature extraction network included in the initial scoring model are trained based on the target loss value to obtain a trained scoring model.

[0024] In one embodiment, the error parameters are determined based on the reference score values ​​and predicted score values ​​of each training image, including: determining one or more of the mean absolute error and the root mean square error based on the reference score values ​​and predicted score values ​​of each training image, and using one or more of the mean absolute error and the root mean square error as the error parameters.

[0025] In one embodiment, determining the weighted loss parameter based on the reference score value and the predicted score value of each training image includes: determining the prediction error of each training image based on the reference score value and the predicted score value of each training image;

[0026] The proportion of training images whose prediction errors are greater than an error threshold in the training data set is determined, and a weighted loss parameter is determined based on the proportion.

[0027] In one embodiment, determining the target loss value based on the error parameter and the weighted loss parameter includes:

[0028] The mean absolute error, the root mean square error, and the weighted loss parameter are multiplied by the corresponding weights and then added to obtain a target loss value.

[0029] On the one hand, an embodiment of the present application discloses an image processing device, the device comprising:

[0030] An acquisition unit, used for acquiring a face image to be detected;

[0031] A processing unit, used for extracting a target face region image from the face image to be detected, and processing the target face region image using a first feature extraction network and a second feature extraction network included in the scoring model to obtain a scoring result of the target face region image;

[0032] The scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process, the target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on a proportion of training images in the training data set whose prediction errors are greater than an error threshold;

[0033] Training unit, used to train the scoring model.

[0034] In one embodiment, when the processing unit extracts the target face area image from the face image to be detected, it is specifically used to: extract the initial face area image from the face image to be detected, then fill the initial face area image to obtain a face area image with a target aspect ratio, and finally scale the face area image with the target aspect ratio, and use the scaled face area image as the target face area image.

[0035] In one embodiment, when the processing unit uses the first feature extraction network and the second feature extraction network included in the scoring model to process the target face area image and obtains the scoring result of the target face area image, it is specifically used to: use the first feature extraction network to extract features of the target face area image to obtain a first feature vector; use the second feature extraction network to extract features of the target face area image to obtain a second feature vector; process the first feature vector and the second feature vector to obtain the scoring result of the target face area image.

[0036] In one embodiment, the image processing device also includes a training unit, which is used to: obtain a training data set, the training data set including multiple training images and a reference score value for each training image; use an initial scoring model to process the multiple training images to obtain a predicted score value for each training image; determine an error parameter and a weighted loss parameter based on the reference score value and the predicted score value of each training image, and determine a target loss value based on the error parameter and the weighted loss parameter; train one or more of the first feature extraction network and the second feature extraction network included in the initial scoring model based on the target loss value to obtain a trained scoring model.

[0037] In one embodiment, when the training unit determines the error parameter based on the reference score value and the predicted score value of each training image, it is specifically used to: determine one or more of the mean absolute error and the root mean square error based on the reference score value and the predicted score value of each training image, and use one or more of the mean absolute error and the root mean square error as the error parameter.

[0038] In one embodiment, when the training unit determines the weighted loss parameter based on the reference score value and the predicted score value of each training image, it is specifically used to: determine the prediction error of each training image based on the reference score value and the predicted score value of each training image; determine the proportion of training images whose prediction errors are greater than the error threshold in the training data set, and determine the weighted loss parameter based on the proportion.

[0039] In one embodiment, when the training unit determines the target loss value based on the error parameter and the weighted loss parameter, it is also used to: multiply the mean absolute error, the root mean square error and the weighted loss parameter by the corresponding weights and then add them to obtain the target loss value.

[0040] The present application also discloses an image processing device, which includes:

[0041] A processor, suitable for implementing one or more instructions; and a computer-readable storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executing the image processing method provided in the embodiment of the present application.

[0042] The embodiment of the present application further discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the image processing method provided by the embodiment of the present application is executed.

[0043] The embodiment of the present application also discloses a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the image processing device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the image processing device performs the above-mentioned image processing method.

[0044] In an embodiment of the present application, an image processing device processes a target face area image using a trained scoring model to obtain a scoring result of the target face area image, wherein the target face area image is obtained from a face image to be detected, and a specific process is to first extract a bounding box area image of the face of the face image to be detected, i.e., an initial face area image, and then fill and scale the initial face area image to obtain the target face area image; a feature extraction network in the scoring model is trained in combination with an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on the proportion of training images in a training data set whose prediction errors are greater than an error threshold. Since the proportion loss of difficult samples (i.e., training images whose prediction errors are greater than the error threshold) is considered during the training process, the trained feature extraction network can extract better face features. When the scoring model trained in the above manner is used for face scoring, the efficiency and accuracy of face scoring are higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a schematic diagram of the architecture of an image processing system disclosed in an embodiment of the present application;

[0047] Figure 2 It is a flowchart of an image processing method disclosed in an embodiment of the present application;

[0048] Figure 3 The method of extracting and adjusting the face area image is shown;

[0049] Figure 4 shows the architecture of the scoring model;

[0050] Figure 5a A training method for the scoring model is shown;

[0051] Figure 5b Another way to train the scoring model is shown;

[0052] Figure 5c Another way to train the scoring model is shown;

[0053] Figure 6 It is a flowchart of another image processing method disclosed in an embodiment of the present application;

[0054] Figure 7 is a structural schematic diagram of an image processing device disclosed in an embodiment of the present application;

[0055] Figure 8 It is a structural schematic diagram of an image processing device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] In one embodiment, in order to improve the efficiency and accuracy of facial image scoring, the embodiment of the present application provides the following two facial image scoring methods: one is to extract the directional gradient histogram (HOG) features, Gabor features (a feature used to describe image texture information), LBP features (a feature used to describe local texture of an image) of the facial image through an image processing device as the appearance features, and then extract the key points of the facial image as geometric features, and then perform feature fusion on the extracted appearance features and geometric features, and then send them to the regressor to regress the facial image. The regression methods commonly used by the regressor include linear regression, Gaussian regression, support vector machine regression, etc. The other is to use a convolutional neural network to extract the depth features of the facial region image, and then directly use the absolute error MAE or the root mean square error RMSE as the objective function for regression. The above two facial image scoring methods realize the automatic scoring of facial images, which can effectively improve the efficiency of facial image scoring, and the objectivity of the scoring based on the computer can improve the accuracy of facial image scoring to a certain extent.

[0058] However, the above two methods do not take into account the impact of face recognition features (including different features of different areas of the face) on face image scoring during the face image scoring process, and use a single task for model training during the training of the scoring model. Therefore, the accuracy of the above two methods of face image scoring is not high enough and needs to be further improved.

[0059] Based on this, the embodiment of the present application provides an image processing method to further improve the accuracy of facial image scoring. Specifically, the image processing method is implemented based on a scoring model, which takes into account facial recognition features, wherein the facial recognition features are effective expressions of different features of different areas of the face, and may include relevant features of the facial features, such as the size of the eyes, the distance between the eyes and eyebrows, the size of the mouth, the distance between the mouth and the nose, the size and shape of the ears, etc. Therefore, the facial recognition features have a more detailed representation capability for the face, which is conducive to the scoring model to extract the detailed features of the face, so that a more reasonable score prediction can be given. At the same time, in the training process of the scoring model, a difficult sample ratio loss is also proposed. The function of the difficult sample ratio loss is to optimize the sample ratio whose absolute error of the predicted score is greater than a preset threshold during the training process; finally, a multi-task model training method is formed in the process of scoring model training, that is, a training method formed by multiple loss functions, and the scoring accuracy of the scoring model trained by the above method is higher.

[0060] The image processing method provided in the embodiment of the present application can be implemented based on artificial intelligence technology AI. AI refers to the theory, method, technology and application system of using digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. AI technology is a comprehensive discipline, and the fields involved are relatively wide; and the data processing method provided in the embodiment of the present application mainly involves machine learning (ML) technology in AI technology. Machine learning technology is a multi-field cross-disciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or implement human learning behavior to acquire new knowledge or skills, reorganize existing knowledge structures and continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all fields of artificial intelligence. Machine learning generally includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and teaching learning.

[0061] The image processing method provided in the embodiment of the present application can also be implemented based on computer vision technology (Computer Vision, CV) in artificial intelligence technology. Computer vision technology (Computer Vision, CV) is a science that studies how to make machines "see". To put it more concretely, it refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further do graphic processing to make computer processing more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology generally includes image processing, image recognition, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, synchronous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0062] The image processing method provided in the embodiment of the present application can be applied to Figure 1The network architecture shown includes a database 101 and an image processing device 102. The database 101 may be a local database of the image processing device 102 or a database of other devices in the cloud. The image processing method can be executed by the image processing device 102. Specifically, the image processing device 102 obtains a face image to be detected from the database 101 and determines whether the face image to be detected includes a face area image. If there is no face area image in the face image to be detected, prompt information indicating that there is no face area image in the face image to be detected is returned to re-acquire the face image to be detected; if there is a face area image in the face image to be detected, a target face area image is extracted from it, and then the target face area image is processed using a first feature extraction network and a second feature extraction network included in a scoring model to obtain a scoring result of the target face area image; wherein the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined in the training process, and the target loss value is calculated based on an error parameter determined in the training process and a weighted loss parameter, and the weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold. Since the scoring model takes into account the weighted loss parameter during the training process (i.e., the proportion of training images in the training data set whose prediction errors are greater than the error threshold), the trained feature extraction network can extract better facial features. Therefore, when the trained scoring model is used for face scoring, the efficiency and accuracy of face scoring are higher.

[0063] It should be noted that the image processing device 102 can be a terminal device or a server. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart car, etc., but is not limited to this; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0064] The above briefly introduces the image processing method provided in the embodiment of the present application. The following is a detailed description of the specific implementation of the image processing method.

[0065] See also Figure 2 , is a flow chart of an image processing method disclosed in an embodiment of the present application. The image processing method can be Figure 1 The image processing device shown performs, including but not limited to, the following steps:

[0066] S201, obtaining a face image to be detected, and extracting a target face region image from the face image to be detected.

[0067] In an embodiment of the present application, the facial image to be detected can be a frame of image extracted from a video, including a frame of image extracted from a live video (for example, a live singing video); the facial image to be detected can also be a photo taken by a shooting device. For example, a user uses the shooting device of a client to take a photo of a person, and sends the photo of the person taken to an image processing device. The image processing device uses the photo of the person as the facial image to be detected.

[0068] After acquiring the face image to be detected, the image processing device performs face image detection on the face image to be detected. If there is no face area image in the face image to be detected, prompt information indicating that there is no face area image in the face image to be detected is returned to re-acquire the face image to be detected; if there is a face area image in the face image to be detected, the target face area image is extracted therefrom.

[0069] In one embodiment, an image processing device determines the position of a face area image from a face image to be detected, captures an image at the position, and obtains an initial face area image; then, the aspect ratio of the initial face area image is determined. If the aspect ratio of the initial face area image is not the target aspect ratio (for example, 1:1), the initial face area image is padded to obtain a face area image of the target aspect ratio; the padded process may be padding the peripheral area of ​​the initial face area image with zeros so that the aspect ratio of the initial face area image after the padded process is the target aspect ratio. For example, assuming that the width of the face detection frame is w and the height is h, the side lengths of the sample face area images of the target aspect ratio obtained after the padded process are all max(w,h); further, the face area image of the target aspect ratio is scaled so that the size of the scaled face area image is a preset size (for example, 350 pixels×350 pixels). It should be noted that during the scaling process, the aspect ratio of the image needs to be kept unchanged; finally, the scaled face region image is used as the target face region image. Among them, filling the face region image with an image of the target aspect ratio and scaling its size to a preset size is beneficial to the processing of the scoring model.

[0070] In a feasible implementation, if Figure 3 As shown, a dual shot face detector (DSFD) can be used to process the face image to be detected to extract the initial face area image from the face image to be detected; then, the face image can be processed by using a dual shot face detector (DSFD). Figure 3The adjustment module in fills and scales the initial face area image to obtain the target face area image.

[0071] S202. Process the target face area image using the first feature extraction network and the second feature extraction network included in the scoring model to obtain a scoring result for the target face area image.

[0072] In the embodiment of the present application, the scoring result of the target face area image can be a scoring value of the face appearance of the target face area image.

[0073] After extracting the target face region image, the image processing device inputs the target face region image into the scoring model, and the scoring model extracts features from the target face region image, determines the scoring result of the target face region image based on the extracted features, and finally outputs the scoring result. Among them, the features extracted by the scoring model may include face recognition features, and may also include one or more of directional gradient histogram (HOG) features, Gabor features (a feature used to describe image texture information), LBP features (a feature used to describe local texture of an image), and geometric features of the face image. Among them, the face recognition features are effective expressions of different features of different regions of the face, and may include relevant features of the facial features, such as the size of the eyes, the distance between the eyes and eyebrows, the size of the mouth, the distance between the mouth and the nose, the size and shape of the ears, etc. Therefore, the face recognition features have a more detailed representation capability for the face, which is conducive to the scoring model extracting detailed features of the face, so as to give a more reasonable score prediction.

[0074] In one embodiment, if Figure 4 As shown, the scoring model includes a first feature extraction network and a second feature extraction network. The scoring model uses the first feature extraction network to extract features from the target face area image to obtain a first feature vector, and uses the second feature extraction network to extract features from the target face area image to obtain a second feature vector. The scoring model then fuses the first feature vector and the second feature vector to obtain a fused feature vector, and processes the fused feature vector to obtain a scoring result for the target face area image. In one embodiment, the scoring model also includes a scoring network (such as Figure 4 The scoring model utilizes the scoring network to fuse the first feature vector and the second feature vector, and utilizes the scoring network to process the fused feature vector to obtain a scoring result of the target face area image; or the scoring model utilizes the scoring network to process only the fused feature vector to obtain a scoring result of the target face area image.

[0075] Among them, the features included in the first feature vector include face recognition features, and may also include one or more of the above features (histogram of oriented gradients (HOG) features, Gabor features, LBP features, geometric features of face images); the features included in the second feature vector include face recognition features, and may also include one or more of the above features (histogram of oriented gradients (HOG) features, Gabor features, LBP features, geometric features of face images), and the dimensions of the first feature vector and the second feature vector may be different. By using multiple feature networks for feature extraction, more abundant face feature information can be obtained compared to using a single feature network for feature extraction, which is conducive to the scoring model to obtain more accurate prediction results.

[0076] In a feasible implementation, the first feature extraction network can be a face recognition network FaceNet whose backbone network is inception-resnet (a residual network), and the dimension of the first feature vector can be 512 dimensions; the second feature extraction network can be a feature extraction network constructed based on the squeeze and excitation residual network se-resnet, and the dimension of the second feature vector can be 2048 dimensions.

[0077] In a feasible implementation, the first feature vector and the second feature vector may be fused by concatenation. For example, assuming that the dimension of the first feature vector is 512 dimensions and the dimension of the second feature vector is 2048 dimensions, the 512-dimensional first feature vector and the 2048-dimensional second feature vector are concatenated to obtain a concatenated 2560-dimensional feature vector.

[0078] In an embodiment of the present application, the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network included in the scoring model are trained in combination with a target loss value determined during the training process. The target loss value is calculated based on the error parameter and weighted loss parameter determined during the training process. The weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than the error threshold.

[0079] The above introduces the process of using the scoring model to score the face area image. The following introduces the training method of the scoring model.

[0080] See also Figure 6 , is a flow chart of a training method for a scoring model disclosed in an embodiment of the present application. The training method can be performed by a training device, which can be the same as the image processing device that performs the above-mentioned image processing method, or can be different from the image processing device. The training method includes but is not limited to the following steps:

[0081] S601: Obtain a training data set, where the training data set includes a plurality of training images and a reference score value of each training image.

[0082] In an embodiment of the present application, the training images in the training data set are face area images extracted from sample face images; the reference score values ​​of the training images in the training data set can be determined artificially. For example, the reference score value of the training image can be that 5 people are asked to score each training image separately at the same time, and then the average of the scores of the 5 people is taken, and the average value is used as the reference score value of the training image.

[0083] In one possible implementation, the sample facial image may be a frame of image extracted from a video, including a frame of image extracted from a live video (e.g., a live singing video); the sample facial image may also be a photo taken by a camera, for example, a user uses a camera on a client to take a photo of a person, and sends the photo of the person to an image processing device, which uses the photo of the person as a sample facial image.

[0084] After acquiring the sample face image, the image processing device performs face detection on the sample face image. If there is no face area image in the sample face image, a prompt message indicating that there is no face area image in the sample face image is returned to reacquire the sample face image; if there is a face area image in the sample face image, a training image is extracted from it.

[0085] In one possible implementation, an image processing device determines the position of a face region image from a sample face image, performs image capture at the position, and obtains a sample face region image; then determines the aspect ratio of the sample face region image, and if the aspect ratio of the sample face region image is not the target aspect ratio (e.g., 1:1), performs padding processing on the sample face region image to obtain a sample face region image of the target aspect ratio; the padding processing may refer to padding a peripheral area of ​​the sample face region image with zeros so that the aspect ratio of the sample face region image after padding is the target aspect ratio, for example, assuming that the face detection frame width is w and the height is h, the side lengths of the sample face region image of the target aspect ratio obtained after padding are all max(w,h); further, the sample face region image of the target aspect ratio is scaled so that the size of the scaled face region image is a preset size (e.g., 350 pixels×350 pixels). It should be noted that during the scaling process, the aspect ratio of the image is kept unchanged; finally, the scaled sample face area image is used as the training image. Among them, the sample face area image is filled with the target aspect ratio and its size is scaled to the preset size, which is more conducive to the processing of the scoring model.

[0086] In a feasible implementation, if Figure 5a As shown, a dual shot face detector (DSFD) can be used to process the sample face image to extract a sample face region image from the sample face image; then Figure 5a The adjustment module in fills and scales the sample face area image.

[0087] S602. Process the plurality of training images using an initial scoring model to obtain a predicted scoring value of each training image, wherein the initial scoring model includes a first feature extraction network and a second feature extraction network.

[0088] In an embodiment of the present application, the image processing device uses multiple training images to perform preliminary training on the initial scoring model to obtain a predicted scoring value for each training image, and the predicted scoring value can be a scoring value for the appearance of the face in the training image.

[0089] After extracting the training image, the image processing device inputs the training image into the initial scoring model, and the initial scoring model extracts features from the training image, and determines the predicted score value of the training image based on the extracted features. Among them, the features extracted by the initial scoring model may include face recognition features, and may also include one or more of directional gradient histogram (HOG) features, Gabor features, LBP features, and geometric features of the face image. Among them, the face recognition features are effective expressions of different features of different areas of the face, and may include relevant features of the facial features, such as the size of the eyes, the distance between the eyes and eyebrows, the size of the mouth, the distance between the mouth and the nose, the size and shape of the ears, etc. Therefore, the face recognition features have a more detailed representation capability for the face, which is conducive to the initial scoring model to extract detailed features of the face, so as to give a more reasonable score prediction.

[0090] In one embodiment, if Figure 4 As shown, the initial scoring model includes a first feature extraction network and a second feature extraction network. The initial scoring model uses the first feature extraction network to extract features from the training image to obtain a first feature vector, and uses the second feature extraction network to extract features from the training image to obtain a second feature vector. The initial scoring model then fuses the first feature vector and the second feature vector to obtain a fused feature vector, and processes the fused feature vector to obtain a predicted score value for the training image. In one embodiment, the initial scoring model also includes a scoring network (such as Figure 4The initial scoring model uses the scoring network to fuse the first eigenvector and the second eigenvector, and uses the scoring network to process the fused eigenvector to obtain a predicted scoring value of the training image; or the initial scoring model uses the scoring network to process only the fused eigenvector to obtain a predicted scoring value of the training image.

[0091] Among them, the features included in the first feature vector include face recognition features, and may also include one or more of the above features (histogram of oriented gradients (HOG) features, Gabor features, LBP features, geometric features of face images); the features included in the second feature vector include face recognition features, and may also include one or more of the above features (histogram of oriented gradients (HOG) features, Gabor features, LBP features, geometric features of face images), wherein the dimensions of the first feature vector and the second feature vector may be different. By using multiple feature networks for feature extraction, more abundant face feature information can be obtained compared to using a single feature network for feature extraction, which is conducive to the initial scoring model to obtain more accurate prediction results.

[0092] In a feasible implementation, the first feature extraction network can be a face recognition network FaceNet whose backbone network is inception-resnet (a residual network), and the face recognition network FaceNet has the following advantages: a deep convolutional neural network (CNN) is used to map the training image to the Euclidean space, and before training or in online learning, the image that is least similar to the training image is constantly searched, and the image that is most similar to the training image is searched, and the gap between all training images is continuously shortened through the random gradient descent method, while the gap with other images is widened as much as possible, and finally an optimum is reached. Through such an embedding learning, the original feature extraction network output layer can be further learned to improve the expression of features, and then the face recognition network FaceNet is also combined with inception-resnet (a residual network), wherein the residual connections in inception-resnet (a residual network) allow shortcuts in the model, allowing researchers to successfully train deeper neural networks. Therefore, the constructed training data set can be input into the above-mentioned face recognition network FaceNet for preliminary training to obtain a first feature vector, wherein the features included in the first feature vector include face recognition features, and may also include one or more of oriented gradient histogram (HOG) features, Gabor features, LBP features, and geometric features of face images.

[0093] In a feasible implementation, the second feature extraction network can be a feature extraction network constructed based on a squeeze and excitation residual network se-resnet, which can extract more complex features. Therefore, the constructed training data set can be input into the squeeze and excitation residual network se-resnet for preliminary training to obtain a second feature vector. The features included in the second feature vector include face recognition features, and can also include one or more of directional gradient histogram (HOG) features, Gabor features, LBP features, and geometric features of face images. The dimensions of the first feature vector and the second feature vector can be different. For example, the dimension of the first feature vector can be 512 dimensions, and the dimension of the second feature vector can be 2048 dimensions.

[0094] In a feasible implementation, the first feature vector and the second feature vector may be fused by concatenation. For example, assuming that the dimension of the first feature vector is 512 dimensions and the dimension of the second feature vector is 2048 dimensions, the 512-dimensional first feature vector and the 2048-dimensional second feature vector are concatenated to obtain a concatenated 2560-dimensional feature vector.

[0095] S603: Determine an error parameter and a weighted loss parameter based on the reference score value and the predicted score value of each training image, and determine a target loss value based on the error parameter and the weighted loss parameter.

[0096] In the embodiment of the present application, one or more of the mean absolute error and the root mean square error of the training image can be determined based on the reference score value and the predicted score value of each training image, and one or more of the mean absolute error and the root mean square error can be used as error parameters. For example, the calculation of the mean absolute error and the root mean square error can adopt the following formulas (1) and (2):

[0097]

[0098]

[0099] Among them, Loss MAE represents the mean absolute error, Loss RMSE represents the root mean square error, N is the number of training images, abs is the absolute value function, x i With y i They represent the predicted score and reference score of the i-th training image respectively.

[0100] In an embodiment of the present application, the prediction error of each training image can be determined based on the reference score value and the predicted score value of each training image; the proportion of training images with prediction errors greater than an error threshold in the training data set is determined, and a weighted loss parameter is determined based on the proportion, wherein the error threshold can be set to 0.5, for example.

[0101] In one embodiment, the proportion of training images whose prediction errors are greater than the error threshold in the training data set can be used as a weighted loss parameter. The function of the weighted loss parameter is to directly optimize the proportion of samples whose prediction errors are greater than the error threshold. In this case, the weighted loss parameter can be calculated using the following formula (3):

[0102]

[0103] Among them, the error threshold is set to 0.5, Loss ours represents the weighted loss parameter, N is the number of training images, I is the indicator function, abs is the absolute value function, x i With y i They represent the predicted score and reference score of the i-th training image respectively.

[0104] In an embodiment of the present application, the above-determined mean absolute error, root mean square error, and weighted loss parameter can be multiplied by the corresponding weights and then added to obtain the target loss value.

[0105] In one embodiment, the target loss value may be calculated using the following formula (4):

[0106] Loss = a*Loss ours +b*Loss RMSE +c*Loss MAE (4)

[0107] Among them, Loss is the target loss value, Loss ours is a weighted loss parameter, and the weight coefficient a can be, for example, 0.1. RMSE Root mean square error, the weight coefficient b can be, for example, 0.3, Loss MAE is the mean absolute error, and the weight coefficient c may be, for example, 0.6, wherein the sum of the weight coefficients a, b, c may be 1.

[0108] S604: Train one or more of the first feature extraction network and the second feature extraction network included in the initial scoring model based on the target loss value to obtain a trained scoring model.

[0109] In the embodiment of the present application, only the second feature extraction network included in the initial scoring model can be trained based on the target loss value, such as Figure 5aAs shown, you can follow Figure 5a The direction indicated by the dotted arrow returns the determined target loss value to the second feature extraction network, and the second feature extraction network adjusts its own network architecture or network parameters based on the target loss value to better extract facial features. In this case, the first feature extraction network can be trained in advance, so there is no need to train it based on the target loss value.

[0110] In the embodiment of the present application, only the first feature extraction network included in the initial scoring model can be trained based on the target loss value, such as Figure 5b As shown, you can follow Figure 5b The direction indicated by the dotted arrow returns the determined target loss value to the first feature extraction network, and the first feature extraction network adjusts its own network architecture or network parameters based on the target loss value to enable better extraction of facial features. In this case, the second feature extraction network can be trained in advance, so there is no need to train based on the target loss value.

[0111] In the embodiment of the present application, the first feature extraction network and the second feature extraction network included in the initial scoring model can also be trained based on the target loss value, such as Figure 5c As shown, you can follow Figure 5c The direction indicated by the dotted arrow returns the determined target loss value to the first feature extraction network and the second feature extraction network. The first feature extraction network and the second feature extraction network adjust their own network architecture or network parameters based on the target loss value to better extract facial features.

[0112] Since the target loss value is determined based on the weighted loss parameter, and the function of the weighted loss parameter is to optimize the proportion of samples whose absolute error between the reference score value and the predicted score value is greater than a preset threshold, one or both of the first feature extraction network and the second feature extraction network included in the initial scoring model are trained based on the target loss value, so that the trained feature extraction network can extract better facial features. When the scoring model trained in the above manner is used for face scoring, the efficiency and accuracy of face scoring are higher.

[0113] In one embodiment, the scoring model preliminarily trained in step S602 is adjusted according to the target loss value determined in step S603 to obtain a scoring model. The adjustment may be performed in the following manner:

[0114] Determine the proportion of training images in the training data set whose prediction errors are greater than the error threshold. If the weighted loss parameter is greater than the set threshold, adjust the scoring model after preliminary training according to the target loss value. Specifically, first feed back the obtained target loss value to the fully connected layer of the scoring model after preliminary training, and then feed back to one or both of the first feature extraction network and the second feature extraction network, so as to continuously optimize the scoring model after preliminary training, and continuously reduce the proportion of training images with prediction errors greater than the error threshold during the optimization process, and finally obtain the adjusted model as the scoring model, wherein the feedback method can be based on the back propagation algorithm.

[0115] The above steps S601 to S604 may be specifically performed as follows: Figure 5a , 5b, 5c, include scoring module, preprocessing module and adjustment module, wherein the scoring module is mainly used to score the extracted features, the preprocessing module is mainly used to obtain the training data set from the sample face image, and the adjustment module is mainly used to fill and scale the sample face area image.

[0116] In an embodiment of the present application, an image processing device obtains a sample face image, extracts a face bounding box area image, i.e., a sample face area image, from the sample face image, and then fills and scales the sample face area image to obtain a training image; then the training image is imported into a first feature extraction network and a second feature extraction network as an input of a scoring model, and a first feature vector and a second feature vector are extracted respectively, wherein the features included in the feature vector include face recognition features, and may also include one or more of directional gradient histogram (HOG) features, Gabor features, LBP features, and geometric features of the face image, and then The first feature vector and the second feature vector are fused to obtain a fused feature vector, and finally the fused feature vector is processed by the scoring model to obtain a prediction result of the training image. The scoring model is trained by this method, and the amount of data during model training can be increased. At the same time, the feature information of the face image is enriched by fusing features. In addition, since the proportional loss of difficult samples (i.e., training images with prediction errors greater than an error threshold) is considered during the training process, the feature extraction network obtained by training can extract better face features. When the scoring model trained by the above method is used for face scoring, the efficiency and accuracy of face scoring are higher.

[0117] Based on the above method embodiment, the present application embodiment also provides a structural diagram of an image processing device. Figure 7 , is a structural schematic diagram of an image processing device provided in an embodiment of the present application. Figure 7 The image processing device 1000 shown can run the following units:

[0118] An acquisition unit 1001 is used to acquire a face image to be detected;

[0119] The processing unit 1002 is used to extract a target face region image from the face image to be detected, and process the target face region image using the first feature extraction network and the second feature extraction network included in the scoring model to obtain a scoring result of the target face region image;

[0120] Among them, the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process. The target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process. The weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold in the training data set.

[0121] In one embodiment, when the processing unit 1002 extracts the target face area image from the face image to be detected, it is specifically used to: extract the initial face area image from the face image to be detected, then fill the initial face area image to obtain a face area image with a target aspect ratio, and finally scale the face area image with the target aspect ratio, and use the scaled face area image as the target face area image.

[0122] In one embodiment, when the processing unit 1002 extracts the target face area image from the face image to be detected, it is specifically used to: extract an initial face area image from the face image to be detected; fill the initial face area image to obtain a face area image with a target aspect ratio; scale the face area image with the target aspect ratio, and use the scaled face area image as the target face area image.

[0123] In one embodiment, the processing unit 1002 uses the first feature extraction network and the second feature extraction network included in the scoring model to process the target facial area image to obtain the scoring result of the target facial area image, which is specifically used to: use the first feature extraction network to extract features of the target facial area image to obtain a first feature vector; use the second feature extraction network to extract features of the target facial area image to obtain a second feature vector; process the first feature vector and the second feature vector to obtain the scoring result of the target facial area image.

[0124] In one embodiment, the image processing device also includes a training unit 1003, which is used to: obtain a training data set, wherein the training data set includes multiple training images and a reference score value for each training image; use an initial scoring model to process the multiple training images to obtain a predicted score value for each training image; determine an error parameter and a weighted loss parameter based on the reference score value and the predicted score value of each training image, and determine a target loss value based on the error parameter and the weighted loss parameter; train one or more of the first feature extraction network and the second feature extraction network included in the initial scoring model based on the target loss value to obtain a trained scoring model.

[0125] In one embodiment, when the training unit 1003 determines the error parameter based on the reference score value and the predicted score value of each training image, it is specifically used to: determine one or more of the mean absolute error and the root mean square error based on the reference score value and the predicted score value of each training image, and use one or more of the mean absolute error and the root mean square error as the error parameter.

[0126] In one embodiment, when the training unit 1003 determines the weighted loss parameter based on the reference score value and the predicted score value of each training image, it is specifically used to: determine the prediction error of each training image based on the reference score value and the predicted score value of each training image; determine the proportion of training images whose prediction errors are greater than the error threshold in the training data set, and determine the weighted loss parameter based on the proportion.

[0127] In one embodiment, when the training unit 1003 determines the target loss value based on the error parameter and the weighted loss parameter, it is specifically used to: multiply the mean absolute error, the root mean square error and the weighted loss parameter by the corresponding weights respectively and then add them to obtain the target loss value.

[0128] According to one embodiment of the present application, Figure 2 as well as Figure 6 The various steps involved in the image processing method shown can be Figure 7 The image processing device shown in FIG. 1 is executed by each unit in the image processing device shown in FIG. Figure 2 In the image processing method shown in FIG. 1 , step S201 can be performed by Figure 7 The acquisition unit 1001 in the image processing device shown in FIG. 1 is used to perform step S202, which can be performed by Figure 7 The processing unit 1002 in the image processing device shown in the figure is used for execution; for example, Figure 6 In the image processing method shown in FIG. 1 , step S601 can be performed by Figure 7 The acquisition unit 1001 in the image processing device shown in FIG. 1 is used to perform the steps S602 to S604. Figure 7The training unit 1003 in the image processing device shown is used for execution.

[0129] According to another embodiment of the present application, Figure 7 The various units in the image processing device shown can be separately or completely combined into one or several other units to constitute, or one (some) of the units can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, other units can also be included based on the image processing device. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0130] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory medium (RAM), a read-only memory medium (ROM), and other processing elements and storage elements. Figure 2 as well as Figure 6 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Figure 7 The image processing apparatus shown in and the image processing method of the embodiment of the present application are implemented. The computer program can be recorded on, for example, a computer-readable storage medium, and loaded into the above-mentioned computing device through the computer-readable storage medium, and run therein.

[0131] In an embodiment of the present application, the acquisition unit 1001 acquires a face image to be detected, and the processing unit 1002 uses a dual shot face detector (DSFD) to extract a bounding box area image of the face, i.e., an initial face area image, from the face image to be detected, and then fills and scales the initial face area image of the face to obtain a target face area image, and uses a feature extraction network included in a scoring model to predict the target face area image to obtain a scoring result of the target face area image; wherein the feature extraction network is trained in combination with an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold. Since the proportion loss of difficult samples (i.e., training images whose prediction errors are greater than the error threshold) is considered during the training process, the trained feature extraction network can extract better face features. When the scoring model trained in the above manner is used to score faces, the efficiency and accuracy of face scoring are higher.

[0132] Based on the above method and device embodiments, the present application embodiment provides an image processing device. Figure 8 , is a structural schematic diagram of an image processing device provided in an embodiment of the present application. Figure 8 The image processing device 1100 shown includes at least a processor 1101, an input interface 1102, an output interface 1103, a computer storage medium 1104, and a memory 1105. The processor 1101, the input interface 1102, the output interface 1103, the computer storage medium 1104, and the memory 1105 may be connected via a bus or other means.

[0133] The computer storage medium 1104 may be stored in the memory 1105 of the image processing device 1100, and the computer storage medium 1104 is used to store a computer program, and the computer program includes program instructions, and the processor 1101 is used to execute the program instructions stored in the computer storage medium 1104. The processor 1101 (or CPU (Central Processing Unit)) is the computing core and control core of the image processing device 1100, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0134] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in the image processing device 1100 for storing programs and data. It is understandable that the computer storage medium here can include both the built-in storage medium in the image processing device 1100 and the extended storage medium supported by the image processing device 1100. The computer storage medium provides a storage space, which stores the operating system of the image processing device 1100. In addition, one or more instructions suitable for being loaded and executed by the processor 1101 are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0135] In one embodiment, the computer storage medium can be loaded by the processor 1101 and execute one or more instructions stored in the computer storage medium to implement the above-mentioned Figure 2 as well as Figure 3 In a specific implementation, one or more instructions in the computer storage medium are loaded by the processor 1101 and execute the following steps:

[0136] Acquire a face image to be detected, and extract a target face area image from the face image to be detected;

[0137] Processing the target face region image using a first feature extraction network and a second feature extraction network included in the scoring model to obtain a scoring result of the target face region image;

[0138] Wherein, the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process, and the target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold.

[0139] In a possible implementation, when the processor 1101 extracts the target face region image from the face image to be detected, it is specifically configured to:

[0140] Extracting an initial face region image from the face image to be detected;

[0141] Filling the initial face region image to obtain a face region image with a target aspect ratio;

[0142] The face region image with the target aspect ratio is scaled, and the scaled face region image is used as the target face region image.

[0143] In a possible implementation, the processor 1101 processes the target face region image using the first feature extraction network and the second feature extraction network included in the scoring model to obtain a scoring result of the target face region image, including:

[0144] Using the first feature extraction network to extract features from the target face region image to obtain a first feature vector;

[0145] Using the second feature extraction network to extract features from the target face region image to obtain a second feature vector;

[0146] The first feature vector and the second feature vector are processed to obtain a scoring result of the target face area image.

[0147] In a possible implementation, the processor 1101 processes the first feature vector and the second feature vector according to a scoring model including a scoring network to obtain a scoring result of the target face region image, including:

[0148] Fusing the first feature vector and the second feature vector to obtain a fused feature vector;

[0149] The fused feature vector is processed using the scoring network to obtain a scoring result of the target face area image.

[0150] In a possible implementation, the processor 1101 is further configured to obtain a training data set, where the training data set includes a plurality of training images and a reference score value of each training image;

[0151] Processing the plurality of training images using the initial scoring model to obtain a predicted scoring value for each training image;

[0152] Determine an error parameter and a weighted loss parameter based on the reference score value and the predicted score value of each training image, and determine a target loss value based on the error parameter and the weighted loss parameter;

[0153] One or more of the first feature extraction network and the second feature extraction network included in the initial scoring model are trained based on the target loss value to obtain a trained scoring model.

[0154] In one possible implementation, the processor 1101 determines an error parameter based on a reference score value and a predicted score value of each training image, including: determining one or more of a mean absolute error and a root mean square error based on the reference score value and the predicted score value of each training image, and using one or more of the mean absolute error and the root mean square error as an error parameter.

[0155] In one possible implementation, the processor 1101 determines a weighted loss parameter based on a reference score value and a predicted score value of each training image, including: determining a prediction error of each training image based on the reference score value and the predicted score value of each training image; determining a proportion of training images having a prediction error greater than an error threshold in the training data set, and determining a weighted loss parameter based on the proportion.

[0156] In the implementation of the present application, the processor 1101 obtains a face image to be detected, and extracts a target face area image from the face image to be detected; the target face area image is processed using a first feature extraction network and a second feature extraction network included in a scoring model to obtain a scoring result for the target face area image; wherein the scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process, and the target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold, and the feature vectors extracted by the first feature extraction network and the second feature extraction network include a first feature vector and a second feature vector, and the first feature vector and the second feature vector are respectively The features included in the two feature vectors include face recognition features, and may also include one or more of directional gradient histogram (HOG) features, Gabor features, LBP features, and geometric features of face images. The first feature vector and the second feature vector are fused in a preset manner to obtain a fused feature vector. Finally, the fused feature vector is processed using a scoring model to obtain a scoring result of the target face area image. The scoring model is trained by this method, and the amount of data during model training can be increased. At the same time, the feature information of the face image is enriched by fusing features. In addition, since the proportional loss of difficult samples (i.e., training images with prediction errors greater than an error threshold) is considered during the training process, the feature extraction network obtained by training can extract better face features. When the scoring model trained in the above manner is used for face scoring, the efficiency and accuracy of face scoring are higher.

[0157] According to one aspect of the present application, the embodiment of the present application further provides a computer product or a computer program, the computer product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. The processor 1101 reads the computer instruction from the computer-readable storage medium, and the processor 1101 executes the computer instruction, so that the image processing device 1100 performs Figure 2 as well as Figure 8 The image processing method shown.

[0158] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0159] In the several embodiments provided in this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, and the division of the modules described above is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0160] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An image processing method, It is characterized in that The method comprises: Acquire a face image to be detected, and extract a target face area image from the face image to be detected; Processing the target face region image using a first feature extraction network and a second feature extraction network included in the scoring model to obtain a scoring result of the target face region image; The scoring model is trained using a training data set containing training images, and one or more of the first feature extraction network and the second feature extraction network are trained in combination with a target loss value determined during the training process, the target loss value is calculated based on an error parameter and a weighted loss parameter determined during the training process, and the weighted loss parameter is determined based on the proportion of training images in the training data set whose prediction errors are greater than an error threshold; The method further comprises: Acquire a training data set, wherein the training data set includes a plurality of training images and a reference score value of each training image; Processing the plurality of training images using the initial scoring model to obtain a predicted scoring value for each training image; Based on the reference score value and the predicted score value of each training image, an error parameter and a weighted loss parameter are determined, and a target loss value is determined based on the error parameter and the weighted loss parameter, wherein the weighted loss parameter is determined based on the reference score value and the predicted score value of each training image, including: determining the prediction error of each training image based on the reference score value and the predicted score value of each training image; determining the proportion of training images with prediction errors greater than an error threshold in the training data set, and determining the weighted loss parameter based on the proportion, wherein the weighted loss parameter is used to optimize the proportion of samples with prediction errors greater than the error threshold; One or more of the first feature extraction network and the second feature extraction network included in the initial scoring model are trained based on the target loss value to obtain a trained scoring model.

2. The method according to claim 1, It is characterized in that The step of extracting a target face region image from the face image to be detected comprises: Extracting an initial face region image from the face image to be detected; Filling the initial face region image to obtain a face region image with a target aspect ratio; The face region image with the target aspect ratio is scaled, and the scaled face region image is used as the target face region image.

3. The method according to claim 1, It is characterized in that The step of processing the target face region image using the first feature extraction network and the second feature extraction network included in the scoring model to obtain a scoring result of the target face region image includes: Using the first feature extraction network to extract features from the target face region image to obtain a first feature vector; Using the second feature extraction network to extract features from the target face region image to obtain a second feature vector; The first feature vector and the second feature vector are processed to obtain a scoring result of the target face area image.

4. The method according to claim 3, It is characterized in that The scoring model further includes a scoring network, and the first feature vector and the second feature vector are processed to obtain a scoring result of the target face region image, including: Fusing the first feature vector and the second feature vector to obtain a fused feature vector; The fused feature vector is processed using the scoring network to obtain a scoring result of the target face area image.

5. The method according to claim 1, It is characterized in that The step of determining the error parameter based on the reference score value and the predicted score value of each training image includes: Based on the reference score value and the predicted score value of each training image, one or more of the mean absolute error and the root mean square error are determined, and one or more of the mean absolute error and the root mean square error are used as error parameters.

6. The method according to claim 5, It is characterized in that The determining of the target loss value based on the error parameter and the weighted loss parameter comprises: The mean absolute error, the root mean square error, and the weighted loss parameter are multiplied by the corresponding weights and then added to obtain a target loss value.

7. An image processing device, It is characterized in that The image processing device comprises: a processor adapted to implement one or more instructions; and, A computer-readable storage medium, wherein the computer-readable storage medium stores one or more instructions, wherein the one or more instructions are suitable for being loaded by the processor and executing the image processing method according to any one of claims 1 to 6.

8. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the image processing method according to any one of claims 1 to 6.

Images