Mobile phone intelligent selfie optimization method

Abstract

The invention relates to the technical field of selfie image optimization, and provides a mobile phone intelligent selfie optimization method comprising the following steps: obtaining original image data of a selfie scene and carrying out feature extraction to obtain a feature map; obtaining corrected key point coordinates according to the feature map; extracting skin color distribution information and illumination intensity information to obtain balanced skin color and illumination parameters; acquiring an ambient light change trend, and generating an illumination compensation coefficient; real-time face tracking and neural network feature extraction are combined to obtain an optimized version of an illumination compensation coefficient; fusing the adjusted image data to obtain a fused image with enhanced details; and extracting and matching the face feature vectors, and outputting an optimized selfie image according to a matching result. Through the cooperation of deep learning and computational photography, the definition, naturalness and individuation effect of the selfie image in a complex light environment are remarkably improved, and the high-quality selfie requirement is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of selfie image optimization, and particularly relates to a method for optimizing intelligent selfie of a mobile phone. BACKGROUND

[0002] Mobile photography, as one of the core functions of modern communication and social media, has become an important part of user experience. With the growing demand for high-quality selfies, mobile selfie technology not only concerns image quality, but also involves the improvement of user personalized expression and real-time interaction experience.

[0003] However, the limitations of existing mobile selfie technology mainly lie in the insufficient adaptability to complex environments and user needs. Many devices rely on preset image processing algorithms, which perform well in specific light or scenes, but often fail in dynamic real-world environments. Due to the limitations of mobile computing resources and the interference of complex lighting conditions, existing technologies often lose or distort facial details when processing high dynamic range scenes.

[0004] Therefore, how to achieve efficient and accurate real-time facial feature analysis on resource-constrained mobile devices, and on this basis, complete dynamic lighting compensation to adapt to complex and changing selfie scenes, has become a key problem in mobile selfie enhancement technology. SUMMARY

[0005] The present application provides a method for optimizing intelligent selfie of a mobile phone, which aims to solve the problem of loss or distortion of facial details when existing mobile devices process high dynamic range scenes.

[0006] The present application is implemented as follows: a method for optimizing intelligent selfie of a mobile phone, the method comprising:

[0007] Obtaining original image data of a selfie scene, using an image processing network to extract features from the original image data, and obtaining a feature map containing a face region;

[0008] According to the feature map, extracting the position information of the facial contour and expression key points, and using a smoothing filter method to process the position information to obtain corrected key point coordinates;

[0009] From the corrected key point coordinates, extracting skin color distribution information and lighting intensity information, using an image enhancement method to normalize the skin color distribution information and lighting intensity information, and obtaining balanced skin color and lighting parameters;

[0010] According to the balanced skin color and lighting parameters, obtaining the trend of environmental light changes, analyzing the trend of environmental light changes through an image prediction network, and generating a lighting compensation coefficient;

[0011] By real-time face tracking and neural network feature extraction, the light compensation coefficient and the equalized skin color and light parameters are processed to obtain an optimized version of the light compensation coefficient;

[0012] According to the optimized version of the light compensation coefficient, the brightness and color balance of the original image data are adjusted, and pixel interpolation method is used to fuse the adjusted image data to obtain a detail-enhanced fused image.

[0013] From the fused image, a face feature vector is extracted, the face feature vector is matched through a classification model, and an optimized selfie image is output according to the matching result.

[0014] Preferably, the feature map of the face region includes the boundary and texture information of the face.

[0015] Preferably, according to the feature map, the position information of the face contour and expression key points is extracted, and the position information is processed by a smoothing filtering method to obtain corrected key point coordinates, specifically including:

[0016] The coordinates of the boundary points of the face contour and the expression key points are identified from the feature map, the deviation between the coordinates of the expression key points and the reference position is calculated, the coordinates whose deviation exceeds a preset range are adjusted by a smoothing filtering method to generate corrected key point coordinates.

[0017]

[0018] wherein, represents the corrected key point coordinates, represents the original key point coordinates, represents the key point position, represents the reference position, represents the preset threshold, represents the number of neighborhood points, represents the smoothing filtering weight coefficient, represents the coordinates of the neighborhood points.

[0019] Preferably, the coordinates whose deviation exceeds a preset range are adjusted by a smoothing filtering method, specifically including:

[0020] For the coordinates whose deviation exceeds a preset range, the coordinate data of the neighborhood points is obtained;

[0021] According to the coordinate data of the neighborhood points and the preset weight coefficient, the corrected key point coordinates are generated.

[0022] Preferably, the trend of the ambient light is obtained according to the equalized skin color and illumination parameters, the trend of the ambient light is analyzed by an image prediction network, and an illumination compensation coefficient is generated. Specifically, the method comprises the following steps:

[0023] The change data of the ambient light is extracted from the equalized skin color and illumination parameters, the change data is processed by an image prediction network, an illumination mutation state is determined, and an illumination compensation coefficient is generated according to the illumination mutation state:

[0024]

[0025] Among them, The illumination compensation coefficient is represented by I, The target illumination intensity is represented by I target, The current illumination intensity is represented by I current, The trend of the ambient light is represented by I trend, The illumination difference weight coefficient is represented by w, The illumination change trend weight coefficient is represented by w trend.

[0026] Preferably, the illumination compensation coefficient is processed by real-time face tracking and neural network feature extraction to obtain an optimized version of the illumination compensation coefficient. Specifically, the method comprises the following steps:

[0027] The dynamic features of the face region are extracted by real-time face tracking;

[0028] The dynamic features and the illumination compensation coefficient are fused by a neural network to generate an optimized version of the illumination compensation coefficient.

[0029] Preferably, the brightness and color balance of the original image data are adjusted according to the optimized version of the illumination compensation coefficient, and the adjusted image data is fused by a pixel interpolation method to obtain a detail-enhanced fused image. Specifically, the method comprises the following steps:

[0030] The adjusted image data is processed by multi-layer pixel interpolation;

[0031] The fusion image is generated by fusing the interpolation results of each layer according to a preset interpolation weight.

[0032] Preferably, the face feature vector is matched by a classification model, and an optimized selfie image is output according to the matching result. Specifically, the method comprises the following steps:

[0033] The face feature vector is classified by a classification model. If the classification result matches a preset user preference label, an optimized selfie image is output. Otherwise, the illumination compensation coefficient is iteratively adjusted to obtain a final selfie image:

[0034]

[0035] wherein, represents the final selfie image pixel value, represents the original image pixel value, represents the optimized version of the light compensation coefficient, represents the brightness color adjustment weight, represents the interpolation layer number, represents the first layer interpolation weight, represents the first layer pixel interpolation result.

[0036] Preferably, the preset user preference label is generated based on user historical selfie optimization records and actively set preference information.

[0037] Compared with the prior art, the beneficial effects of the present application are:

[0038] The present application first extracts a face feature map through an image processing network, obtains boundary, texture and key point information, performs smoothing filter correction on key points with large deviations, then extracts skin color and light information and normalizes to generate equalization parameters. For sudden changes in ambient light, the present application uses an image prediction network to analyze light trends, generates and optimizes light compensation coefficients, adjusts image brightness and color balance in combination with real-time face tracking, fuses detail enhancement images through pixel interpolation, and finally uses a classification model to match user preferences to output an optimized image. Therefore, through the synergy of deep learning and computational photography, the present application significantly improves the clarity, naturalness and personalized effect of selfie images in complex light environments, meeting the user's high-quality selfie needs. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments, it should be understood that the following drawings only show some embodiments of the present application, therefore should not be regarded as a limitation on the scope, for those skilled in the art, on the premise of not paying creative labor, other related drawings can also be obtained according to these drawings.

[0040] Figure 1 is the overall flowchart of the embodiments of the present application. DETAILED DESCRIPTION

[0041] To better understand the technical content of this invention, the technical solutions of this invention are further described and explained below with reference to specific embodiments, but are not limited thereto. The technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0042] refer to Figure 1 The mobile phone's intelligent selfie optimization method includes:

[0043] S101, acquire the original image data of the selfie scene, and use an image processing network (such as a convolutional neural network CNN or a recurrent neural network RNN) to extract features from the original image data to obtain a feature map containing the face region. The feature map of the face region includes the boundary and texture information of the face.

[0044] S102, Based on the feature map, extract the position information of the facial contour and expression key points, and use a smoothing filter method to process the position information to obtain the corrected key point coordinates.

[0045] The coordinates of boundary points and expression key points of the facial contour are identified from the feature map. For the coordinates of the expression key points, the deviation between the coordinates and the reference position is calculated. Coordinates with deviations exceeding a preset range are adjusted using a smoothing filter method. For coordinates with deviations exceeding the preset range, the coordinate data of neighboring points are obtained. Based on the coordinate data of the neighboring points and a preset weight coefficient, the corrected key point coordinates are generated.

[0046]

[0047] in, This indicates the corrected coordinates of the key points. Represents the original keypoint coordinates. Indicates the first Key locations Indicates a reference position. Indicates the preset threshold. Indicates the number of neighboring points. Represents the smoothing filter weight coefficients. This represents the coordinates of neighboring points. The reference position... To determine the standard facial expression key point locations based on statistics from a large number of facial samples, a preset threshold is set. The setting is based on the allowable range of facial pose changes (e.g., 5-10 pixels).

[0048] S103, extract skin color distribution information and light intensity information from the corrected key point coordinates, and use image enhancement methods to normalize the skin color distribution information and light intensity information to obtain balanced skin color and light parameters.

[0049] S104. Based on the equalized skin color and illumination parameters, the ambient light change trend is obtained, and the ambient light change trend is analyzed by an image prediction network to generate an illumination compensation coefficient.

[0050] Ambient light variation data is extracted from the equalized skin tone and illumination parameters. This variation data is then processed by an image prediction network to determine illumination abrupt changes. Illumination compensation coefficients are then generated based on these abrupt changes.

[0051]

[0052] in, Indicates the illumination compensation coefficient. Indicates the target light intensity. Indicates the current light intensity. Indicates the trend of changes in ambient light. This represents the weighting coefficient for differences in illumination. This represents a weighting coefficient indicating the trend of light intensity change. The target light intensity... The weighting coefficient is set to the preset suitable lighting intensity for selfies (e.g., 100-300 lux). , Determined through model training and optimization ( The value range is 0.6-0.8. The value range is 0.2-0.4.

[0053] S105, by combining real-time face tracking with neural network feature extraction, the illumination compensation coefficient and the equalized skin color and illumination parameters are processed to obtain an optimized version of the illumination compensation coefficient.

[0054] The dynamic features of the face region are extracted by real-time face tracking in smartphone selfies. The dynamic features are then fused with the illumination compensation coefficient using a computational photography neural network (such as an attention mechanism fusion network). The illumination compensation coefficient is then processed in conjunction with the equalized skin color and illumination parameters to obtain an optimized version of the illumination compensation coefficient.

[0055] S106, according to the optimized version of the illumination compensation coefficient, adjust the brightness and color balance of the original image data, and use pixel interpolation method to fuse the adjusted image data to obtain a fused image with enhanced details.

[0056] The adjusted image data is subjected to multi-layer pixel interpolation, and the interpolation results of each layer are fused according to the preset interpolation weights to generate a fused image with enhanced details.

[0057] S107. Extract facial feature vectors from the fused image, match the facial feature vectors using a classification model (such as Support Vector Machine (SVM) or Lightweight CNN), and output an optimized selfie image based on the matching results.

[0058] A classification model is used to classify the facial feature vector. If the classification result matches a preset user preference label, an optimized selfie image is output; otherwise, the illumination compensation coefficient is iteratively adjusted to obtain the final selfie image.

[0059]

[0060] in, This represents the pixel value of the final selfie image. Represents the pixel values ​​of the original image. This indicates the lighting compensation coefficient for the optimized version. Indicates the brightness and color adjustment weights. Indicates the number of interpolation layers. Indicates the first Layer interpolation weights, Indicates the first Layer pixel interpolation results. Weights , Determined through experimental debugging, and satisfying ( The value range is 0.5-0.7. (Assigned based on the interpolation layer detail retention capability).

[0061] The preset user preference tags are generated based on the user's historical selfie optimization records and actively set preference information.

[0062] The above-described embodiments are only some embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for optimizing intelligent selfies on mobile phones, characterized in that, The mobile phone's intelligent selfie optimization methods include: The original image data of the selfie scene is acquired, and the original image data is used to extract features from the original image data to obtain a feature map containing the face region. Based on the feature map, the positional information of the facial contour and expression key points is extracted, and the positional information is processed by a smoothing filter method to obtain the corrected key point coordinates; Skin color distribution information and light intensity information are extracted from the corrected key point coordinates. Image enhancement methods are then used to normalize the skin color distribution information and light intensity information to obtain balanced skin color and light parameters. Based on the equalized skin color and illumination parameters, the trend of ambient light change is obtained, and the trend of ambient light change is analyzed by an image prediction network to generate an illumination compensation coefficient. By combining real-time face tracking with neural network feature extraction, the illumination compensation coefficient is processed along with the equalized skin color and illumination parameters to obtain an optimized version of the illumination compensation coefficient. Based on the optimized version of the illumination compensation coefficient, the brightness and color balance of the original image data are adjusted, and the adjusted image data is fused using a pixel interpolation method to obtain a fused image with enhanced details. Facial feature vectors are extracted from the fused image, and the facial feature vectors are matched using a classification model. An optimized selfie image is then output based on the matching results.

2. The mobile phone intelligent selfie optimization method as described in claim 1, characterized in that, The feature map of the face region includes the boundary and texture information of the face.

3. The mobile phone intelligent selfie optimization method as described in claim 2, characterized in that, The step of extracting the positional information of facial contours and expression key points based on the feature map, and processing the positional information using a smoothing filter method to obtain the corrected key point coordinates, specifically includes: The coordinates of the boundary points of the facial contour and key expression points are identified from the feature map. For the coordinates of the key expression points, the deviation between the coordinates and the reference position is calculated. The coordinates with deviations exceeding a preset range are adjusted using a smoothing filter method to generate corrected key point coordinates. Among them, C corrected (x,y) represents the corrected keypoint coordinates, C original (x,y) represents the original keypoint coordinates, P i P represents the position of the i-th key point. ref θ represents the reference position, θ represents the preset threshold, K represents the number of neighboring points, and ω represents the reference position. k C represents the smoothing filter weight coefficients. neighbor (x k ,y k ) represents the coordinates of a neighboring point.

4. The mobile phone intelligent selfie optimization method as described in claim 3, characterized in that, The adjustment of coordinates with deviations exceeding a preset range using a smoothing filtering method specifically includes: For coordinates whose deviations exceed a preset range, obtain the coordinate data of neighboring points; Based on the coordinate data of the neighboring points and the preset weight coefficients, the corrected coordinates of the key points are generated.

5. The mobile phone intelligent selfie optimization method as described in claim 1, characterized in that, The step of obtaining the ambient light change trend based on the equalized skin color and illumination parameters, analyzing the ambient light change trend through an image prediction network, and generating an illumination compensation coefficient specifically includes: Ambient light variation data is extracted from the equalized skin tone and illumination parameters. This variation data is then processed by an image prediction network to determine illumination abrupt changes. Illumination compensation coefficients are then generated based on these abrupt changes. Among them, L comp L represents the illumination compensation coefficient. target L represents the target illumination intensity. current Indicates the current light intensity. This represents the trend of ambient light change, where α represents the weighting coefficient for light difference and β represents the weighting coefficient for the trend of light change.

6. The mobile phone intelligent selfie optimization method as described in claim 1, characterized in that, The process combines real-time face tracking with neural network feature extraction to process the illumination compensation coefficient and the equalized skin color and illumination parameters, resulting in an optimized version of the illumination compensation coefficient. Specifically, this includes: Dynamic features of facial regions are extracted through real-time face tracking; The dynamic features and the illumination compensation coefficient are fused together using a neural network to generate an optimized version of the illumination compensation coefficient.

7. The mobile phone intelligent selfie optimization method as described in claim 1, characterized in that, The step of adjusting the brightness and color balance of the original image data according to the optimized version of the illumination compensation coefficient, and fusing the adjusted image data using a pixel interpolation method to obtain a fused image with enhanced details, specifically includes: The adjusted image data is subjected to multi-layer pixel interpolation, with the number of layers determined based on the image resolution. The interpolation results of each layer are fused according to the preset interpolation weights to generate a fused image with enhanced details.

8. The mobile phone intelligent selfie optimization method as described in claim 1, characterized in that, The step of matching the facial feature vector using a classification model and outputting an optimized selfie image based on the matching results specifically includes: A classification model is used to classify the facial feature vector. If the classification result matches a preset user preference label, an optimized selfie image is output; otherwise, the illumination compensation coefficient is iteratively adjusted to obtain the final selfie image. Among them, I final (x,y) represents the final pixel values ​​of the selfie image, I original (x,y) represents the pixel values ​​of the original image, L opt γ represents the lighting compensation coefficient of the optimized version, M represents the brightness and color adjustment weights, and λ represents the number of interpolation layers. n This represents the interpolation weights for the nth layer. This represents the interpolation result of the nth pixel.

9. The mobile phone intelligent selfie optimization method as described in claim 8, characterized in that, The preset user preference tags are generated based on the user's historical selfie optimization records and actively set preference information.

Images