Image processing method and system

Abstract

The present invention provides an image processing method and system, wherein the method comprises: S1 receiving the face template information of a game character selected by a user, wherein the face template information comprises a standard face image of the game character, wherein the standard face image of the game character comprises face contour information and skin color information; S2 receiving the user face image uploaded by the user, pre-processing the received user face image, and obtaining a target face image; S3 performing feature extraction processing according to the obtained target face image, obtaining user face feature information, and obtaining a corrected user face image according to the obtained user face feature information; S4 fusing the corrected target face image into the standard face image of the game character, and obtaining a user-defined face image of the game character. The present invention is helpful to improve the personalization level of game character creation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image processing method and system. Background Art

[0002] Electronic games have become an indispensable part of people's entertainment. At present, most electronic games require users to create game characters when playing for the first time. When creating game characters in most current games, the game characters are role templates designed in advance by the developers. Users can only choose one from the ready-made game character templates to play the game, which reduces the flexibility of game character creation and the level of personalization of game character creation. It cannot meet the user's gaming experience requirements. Summary of the invention

[0003] In view of the fact that the level of personalization is insufficient when creating game characters based on game character templates in the above-mentioned electronic games, the present invention aims to provide an image processing method and system.

[0004] The purpose of the present invention is achieved by the following technical solutions:

[0005] In a first aspect, the present invention provides an image processing method, comprising:

[0006] S1 receives the face template information of the game character selected by the user, wherein the face template information includes a standard face image of the game character, wherein the standard face image of the game character includes face contour information and skin color information;

[0007] S2 receives the user face image uploaded by the user, pre-processes the received user face image, and obtains the target face image;

[0008] S3 performs feature extraction processing based on the obtained target face image to obtain user face feature information, and obtains a corrected user face image based on the obtained user face feature information;

[0009] S4 fuses the corrected target face image into the standard face image of the game character to obtain a user-defined face image of the game character.

[0010] In one embodiment, the method further comprises:

[0011] S5 applies the facial image of the user-defined game character to the standard game character to generate the user game character.

[0012] In one embodiment, the method further comprises:

[0013] SB1 receives the character custom face instruction fed back by the user, and returns the selectable game character face template information to the user according to the received instruction.

[0014] In one embodiment, step S1 specifically includes: providing multiple game character facial templates for user selection, wherein each facial template is provided with a corresponding game character standard facial image, different game character standard facial images are provided with different facial contour information and skin color information, wherein the facial contour information and skin color information are adjusted and set by the user, and when the user completes the setting, receiving the game character standard facial image selected by the user and its corresponding facial contour information and skin color information.

[0015] In one implementation, in step S2, the received user face image is preprocessed, including:

[0016] S21 performs dimensionality reduction and denoising processing on the received user face image to obtain a first user face image;

[0017] S22 performs enhancement and adjustment processing on the first user face image to obtain a second user face image;

[0018] S23 performs edge detection and segmentation processing according to the second user's facial image, and adjusts the size of the segmented facial image to obtain a target facial image.

[0019] In one implementation, step S3 includes:

[0020] S31 performs feature point detection processing based on the target user's facial image to obtain key point information of the target user's facial image;

[0021] S32 calculates the angle features of the user's face according to the obtained key point information, wherein the calculated user angle features include the user's left and right head turning angles and left and right head tilting angles;

[0022] S33 corrects the target user face image according to the angle feature of the user face to obtain a corrected user face image, including:

[0023] When it is detected that the user's left and right head turning angle is greater than the set head turning threshold, the user's face image is further corrected for head turning;

[0024] When it is detected that the left and right head tilt angles of the user are greater than the set head tilt threshold, the user's facial image is further corrected for head tilt.

[0025] In one implementation, in step S33, performing head turn correction on the user's facial image includes:

[0026] Performing head turn correction on the target user's face image based on the trained head turn posture correction model, wherein the input of the head turn posture correction model is the target user's face image, wherein the head turn posture correction model includes a shape correction model, a texture correction model and an image transformation model;

[0027] The shape correction model performs face shape extraction processing based on the target user's face image to obtain the user's face posture features, and performs posture feature correction based on the obtained user's face posture features to obtain the corrected posture features;

[0028] The texture correction model performs texture feature extraction processing based on the target user's face image to obtain the user's face texture features, and performs texture feature correction based on the obtained user's face texture features to obtain the corrected texture features;

[0029] The image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image.

[0030] In one implementation, in step S33, performing head tilt correction on the user's face image includes:

[0031] The target user's face image is rotated accordingly according to the obtained head tilt angle to obtain a corrected user's face image.

[0032] In one implementation, step S4 includes:

[0033] S41 adjusts the skin color of the corrected user face image to a skin color corresponding to the skin color information of the standard face image of the game character, thereby obtaining the user face image after skin color adjustment;

[0034] S42 performs Poisson fusion on the user face image after skin color adjustment and the standard face image of the game character to obtain a user-defined face image of the game character.

[0035] In a second aspect, the present invention provides an image processing system, comprising:

[0036] The template information acquisition module is used to receive the face template information of the game character selected by the user, wherein the face template information includes the standard face image of the game character, wherein the standard face image of the game character includes the face contour information and the skin color information;

[0037] The face image processing module is used to receive the user face image uploaded by the user, pre-process the received user face image, and obtain the target face image;

[0038] A feature processing module is used to perform feature extraction processing based on the obtained target face image to obtain user face feature information, and obtain a corrected user face image based on the obtained user face feature information;

[0039] The image fusion module is used to fuse the corrected target face image into the standard face image of the game character to obtain the user-defined face image of the game character.

[0040] In one embodiment, the system further comprises:

[0041] The character generation module is used to apply the facial image of the user-defined game character to the standard game character to generate the user game character.

[0042] In one embodiment, the method further comprises:

[0043] The character selection module is used to receive the character custom face instructions fed back by the user, and return the selectable game character face template information to the user according to the received instructions.

[0044] In one embodiment, the template information acquisition module includes: providing multiple game character face templates for user selection, wherein each face template is provided with a corresponding game character standard face image, different game character standard face images are provided with different face contour information and skin color information, wherein the face contour information and skin color information are adjusted and set by the user, and when the user completes the setting, receiving the game character standard face image selected by the user and its corresponding face contour information and skin color information.

[0045] In one implementation, in the face image processing module, preprocessing the received user face image includes:

[0046] Performing dimensionality reduction and denoising processing on the received user face image to obtain a first user face image;

[0047] Performing enhancement and adjustment processing on the first user's facial image to obtain a second user's facial image;

[0048] Edge detection and segmentation processing are performed on the second user's facial image, and the size of the segmented facial image is adjusted to obtain a target facial image.

[0049] In one implementation, the feature processing module includes:

[0050] Perform feature point detection processing based on the target user's facial image to obtain key point information of the target user's facial image;

[0051] and calculating the angle features of the user's face based on the obtained key point information, wherein the calculated user angle features include the user's left and right head turning angles and left and right head tilting angles;

[0052] Correcting the target user's face image according to the angle feature of the user's face to obtain a corrected user's face image includes:

[0053] When it is detected that the user's left and right head turning angle is greater than the set head turning threshold, the user's face image is further corrected for head turning;

[0054] When it is detected that the left and right head tilt angles of the user are greater than the set head tilt threshold, the user's facial image is further corrected for head tilt.

[0055] In one implementation, in the feature processing module, performing head turn correction on the user's face image includes:

[0056] Performing head turn correction on the target user's face image based on the trained head turn posture correction model, wherein the input of the head turn posture correction model is the target user's face image, wherein the head turn posture correction model includes a shape correction model, a texture correction model and an image transformation model;

[0057] The shape correction model performs face shape extraction processing based on the target user's face image to obtain the user's face posture features, and performs posture feature correction based on the obtained user's face posture features to obtain the corrected posture features;

[0058] The texture correction model performs texture feature extraction processing based on the target user's face image to obtain the user's face texture features, and performs texture feature correction based on the obtained user's face texture features to obtain the corrected texture features;

[0059] The image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image.

[0060] In one implementation, in the feature processing module, performing head tilt correction on the user's face image includes:

[0061] The target user's face image is rotated accordingly according to the obtained head tilt angle to obtain a corrected user face image.

[0062] In one implementation, the image fusion module includes:

[0063] Adjusting the skin color of the corrected user face image to a skin color corresponding to the skin color information of the standard face image of the game character, to obtain a user face image after skin color adjustment;

[0064] The user's face image after skin color adjustment is Poisson fused with the standard face image of the game character to obtain the user-defined face image of the game character.

[0065] The beneficial effects of the present invention are as follows: an image processing method and system proposed by the present invention can fuse the acquired user face image into the game character face template selected by the user to obtain the user's personalized game character face image, and generate a corresponding game character based on the obtained game character face image, which helps to improve the personalization level of game character creation.

[0066] At the same time, in the process of fusion based on user facial images and game character facial templates, a technical solution for correcting facial images is proposed. It can correct a corrected user facial image that meets the requirements based on the acquired single user facial image, and apply the corrected user facial image to the fusion of game character facial templates, which helps to improve the robustness and adaptability of game character facial image fusion and improve the quality and effect of game character creation. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0068] Figure 1 A schematic diagram of a method flow of an image processing method according to an embodiment of the present invention;

[0069] Figure 2 for Figure 1 A schematic flow chart of a method for preprocessing a received user face image in an embodiment;

[0070] Figure 3 for Figure 1 A schematic flow chart of a method for obtaining a corrected user face image based on the obtained user face feature information in an embodiment;

[0071] Figure 4 The figure is a framework diagram of an image processing system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The present invention is further described in conjunction with the following application scenarios.

[0073] join Figure 1 An image processing method is shown in the embodiment, wherein the method can be applied to a server, and the method includes:

[0074] The SB1 server receives the character custom face instruction fed back by the user, and returns the selectable game character face template information to the user according to the received instruction;

[0075] In one scenario, a user selects a customized game character through a character creation interface and sends a character customized face instruction to a server. After receiving the instruction, the server pushes preset game character face template information to the user for selection; wherein the preset game character face template includes multiple game character face templates, including different standard face images, different face contours and different skin colors.

[0076] S1: The server receives the face template information of the game character selected by the user, wherein the face template information includes a standard face image of the game character, wherein the standard face image of the game character includes face contour information and skin color information;

[0077] In one embodiment, the server provides multiple game character facial templates for users to choose from, wherein each facial template is provided with a corresponding game character standard facial image, and different game character standard facial images are provided with different facial contour information and skin color information, wherein the facial contour information and skin color information can also be adjusted and set by the user. When the user completes the setting, the server receives the game character standard facial image selected by the user and its corresponding facial contour information and skin color information.

[0078] In one scenario, a user selects a suitable facial template from a received game character facial template as the facial basis of the game character, wherein the game character facial template includes different preset game character standard facial images, wherein the facial contours and / or skin colors of the different standard facial images are different, wherein the facial area in the facial image does not carry texture information, and only retains a preset facial feature map (or none), so as to facilitate subsequent creation of a new custom game character based on the user's facial image.

[0079] The S2 server receives the user face image uploaded by the user, pre-processes the received user face image, and obtains the target face image;

[0080] In one implementation, after receiving the face template information of the game character selected by the user, the server further initiates a face image acquisition request to the user, so that the user acquires the user face image through the user terminal and then transmits the user face image to the server.

[0081] In one embodiment, see Figure 2 In step S2, the received user face image is preprocessed, including:

[0082] S21 performs dimensionality reduction and denoising processing on the received user face image to obtain a first user face image;

[0083] S22 performs enhancement and adjustment processing on the first user face image to obtain a second user face image;

[0084] S23 performs edge detection and segmentation processing according to the second user's facial image, and adjusts the size of the segmented facial image to obtain a target facial image.

[0085] Considering that the image quality and parameters of the user face images received by the server vary, the acquired user face images are first preprocessed, including dimension reduction, denoising and enhancement adjustment processing, which can improve the image quality. At the same time, face segmentation is further performed based on the user face image to extract the user's face area and adjust the image size corresponding to the game character face image, making necessary preparations for the subsequent fusion of the user face image with the game character face image.

[0086] In one implementation, in step S22, enhancing and adjusting the facial image of the first user includes:

[0087] Convert the first user's face image from the RGB color space to the LAB color space, and obtain a brightness component L, a color component A, and a color component B respectively;

[0088] For the acquired brightness component L, the brightness component value of each pixel is statistically histogramed to obtain the brightness component statistical histogram Q1, where the abscissa of the brightness component statistical histogram Q1 represents the brightness component value, and its range is x l ∈[0,100], the ordinate represents the number of pixels corresponding to the brightness component value n(x l );

[0089] The brightness component value of each pixel is adjusted according to the acquired brightness component statistical histogram Q1, wherein the standard adjustment function used is:

[0090]

[0091] In the formula, Indicates the brightness component value after standard adjustment, x l Indicates the brightness component value before standard adjustment. represents the average value of the brightness component of the first user's face image, Indicates the set standard brightness value, where

[0092] Brightness component value adjusted according to the standard A brightness component statistical histogram Q2 is obtained, wherein the abscissa of the brightness component statistical histogram Q2 represents the brightness component value, and its range is x2∈[0,100], and the ordinate represents the number of pixel points corresponding to the brightness component value n(x2);

[0093] The brightness component value of each pixel is enhanced and adjusted according to the acquired brightness component statistical histogram Q2, including:

[0094] According to the obtained brightness component statistical histogram Q2, conditional judgment is performed to obtain the set enhancement threshold, wherein the conditional judgment function used is:

[0095]

[0096] Wherein, △1 ​​represents the conditional function, n(t1) and n(t2) represent the number of pixels corresponding to the brightness component values ​​t1 and t2 respectively; N is the total number of pixels;

[0097] According to the above judgment function △1, t1 and t2 that meet the conditions are obtained as the first enhanced brightness component value t1 and the second enhanced brightness component value t2 respectively, and the brightness component value of each pixel is enhanced and adjusted according to the obtained first enhanced brightness component value t1 and the second enhanced brightness component value t2, wherein the brightness enhancement function used is:

[0098]

[0099] In the formula, △2 represents the brightness enhancement condition function, x2 represents the brightness component value of the corresponding brightness component statistical histogram Q2 before brightness enhancement, Represents the brightness component value after brightness enhancement;

[0100] According to the brightness enhancement condition function △2, the corresponding values ​​of each brightness value that meet the condition are obtained respectively. And adjust the corresponding pixel value x2 to the corresponding Obtain the brightness component L′ after brightness adjustment;

[0101] Reconstruction is performed based on the brightness component L′, color component A, and color component B after brightness adjustment, and the LAB color space is reconverted into the RGB color space to obtain a second user face image.

[0102] In one implementation, in the case where the above calculation process for the brightness component and the number of pixels is not an integer, an integer result can be obtained by rounding up, rounding down or rounding off according to the actual situation. This application does not make specific limitations here.

[0103] Considering that the received user face image has uneven brightness levels, it is easy to affect the accuracy of texture feature and posture feature extraction in the subsequent feature extraction process based on the user face image. Therefore, the present application also proposes a technical solution for further brightness enhancement processing for the first user face image after dimensionality reduction and denoising, wherein the user face image is first converted to the LAB color space, and enhancement processing is performed based on the obtained brightness component, including standard adjustment of the brightness component, and preliminary adjustment of the overall brightness of the image to a suitable level; at the same time, on this basis, in order to avoid the situation where the linear adjustment in the traditional standard adjustment causes image distortion, the image after the standard adjustment is further subjected to brightness component value histogram statistics, and based on the statistical results, a judgment function is proposed to first judge the brightness concentration area, and further propose a brightness enhancement function to flatten the brightness concentration area, so that the concentrated brightness area in the image is relaxed, which helps to improve the overall brightness distribution level of the image and the representation of the feature information in the image. At the same time, the clarity and perception of the user face image are improved, laying a foundation for further integration into the game character face template based on the user face image in the future.

[0104] The S3 server performs feature extraction processing based on the obtained target face image to obtain user face feature information, and obtains a corrected user face image based on the obtained user face feature information;

[0105] In one embodiment, see Figure 3 , step S3 comprises:

[0106] S31 performs feature point detection processing based on the target user's facial image to obtain key point information of the target user's facial image;

[0107] S32 calculates the angle features of the user's face according to the obtained key point information, wherein the calculated user angle features include the user's left and right head turning angles and left and right head tilting angles;

[0108] S33 corrects the target user face image according to the angle feature of the user face to obtain a corrected user face image, including:

[0109] When it is detected that the user's left and right head turning angle is greater than the set head turning threshold, the user's face image is further corrected for head turning;

[0110] When it is detected that the left and right head tilt angles of the user are greater than the set head tilt threshold, the user's facial image is further corrected for head tilt.

[0111] In one scenario, the head turning threshold is set at 2-5 degrees; the head tilting threshold is set at 2-5 degrees.

[0112] In one scenario, when a user needs to use a frontal image to create a facial image of a game character, in the target facial image obtained, since the user's head posture cannot be accurately controlled during the shooting process, it is easy to cause the user's head angle in the target facial image to deviate. In the prior art, there is also a technical solution that detects the user's head angle based on the obtained user facial image to determine whether the user's head angle meets the requirements (for example, it must be a frontal face). When it does not meet the requirements (the user's head angle is slightly deviated, such as slightly turning the head or tilting the head), the user is sent back to re-acquire the user image until a frontal facial image of the user that meets the requirements is obtained. However, when the judgment sensitivity of the above scheme is too high, it is difficult to obtain a user facial image that meets the requirements, and the user needs to repeatedly collect and upload facial images many times, affecting the user experience; when the judgment sensitivity is too low, it is easy for the final game character facial image to have inconsistent facial features, affecting the quality of game character creation. In response to the above problems, the present invention proposes a technical solution that can perform adaptive angle correction based on the acquired user face image, perform angle correction on the target face image, and obtain a corrected user face image with high robustness. At the same time, it helps to improve the effect of subsequent game character face image fusion based on the corrected user face image.

[0113] In one implementation, feature point detection processing is performed based on the target user's facial image, wherein an 83-point feature method is used to extract corresponding feature point information from the target user's facial image (wherein the facial contour contains 19 feature points; the eyes contain 20 feature points; the eyebrows contain 16 feature points; the lips contain 18 feature points; and the nose contains 10 feature points); based on the coordinate information corresponding to the obtained feature points, the angular features of the user's face are calculated.

[0114] In one implementation, in step S33, performing head turn correction on the user's facial image includes:

[0115] Performing head turn correction on the target user's face image based on the trained head turn posture correction model, wherein the input of the head turn posture correction model is the target user's face image, wherein the head turn posture correction model includes a posture correction model, a texture correction model and an image transformation model;

[0116] The posture correction model extracts the face posture of the target user according to the face image to obtain the face posture features of the user, and corrects the face posture features according to the obtained face posture features of the user to obtain the corrected posture features;

[0117] The texture correction model performs texture feature extraction processing based on the target user's face image to obtain the user's face texture features, and performs texture feature correction based on the obtained user's face texture features to obtain the corrected texture features;

[0118] The image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image.

[0119] In the above implementation, a technical solution is proposed for performing image correction based on the posture features and texture features of the user's face image when the user's face image turns its head, wherein correction is performed based on the posture features extracted from the current face image to obtain the corrected posture features, correction is performed based on the texture features extracted from the current face image to obtain the corrected texture features, and the corrected posture features and texture features obtained are used as target quantities to transform the current target user's face image to obtain the corrected user's face image, wherein the corrected face image is presented as a standard frontal image. Using feature correction to transform the face image can avoid the image distortion caused by traditional head turning correction and effectively improve the image quality.

[0120] In one implementation, in step S33, the posture correction model performs facial posture extraction processing based on the target user's facial image to obtain the user's facial posture features, and performs posture feature correction based on the obtained user's facial posture features to obtain the corrected posture features; including:

[0121] Extract the posture features of the target user’s face image: q A ={(x A1 ,y A1 ),(x A2 ,y A2 ),…,(x AN ,y AN )}, where q A represents the posture features of the target user’s face image, (x An ,y An ) represents the coordinate value of the nth feature point of the target user's face image, n = 1, 2, ..., N, N represents the total number of feature points;

[0122] Calculate the posture offset based on the extracted posture features: q dA =q A -q meA ; where q dA represents the pose offset of the target user’s face image, q meA represents the average posture feature of the target user’s face image, where q meA ={(x meA ,y meA ),(x meA ,y meA ),…,(x meA ,y meA )},(xmeA ,y meA ) represents the center point coordinates of each feature point in the target user’s face image, where q dA The size is N×1;

[0123] Calculate the head turning posture feature projection of the target user's face image: where δ A represents the head turning posture feature projection of the target user's face image, where Contains posture features of m standard head-turning face images The size is N×m; represents the posture feature of the mth standard head-turning face image, in The size of is N×m, then δ J The size is m×1;

[0124] The standard posture feature projection of the target user's face image is calculated based on the head turning posture feature projection of the target user's face image: where δ K represents the standard pose feature projection of the target user’s face image, λ J represents the standard head turning posture feature projection, where Where D J Contains the posture offset of m standard head-turning face images D J The size is N×m, represents the posture offset of the mth standard head-turning face image, represents the average posture feature of the mth standard head-turning face image; K represents the standard head posture feature projection, where in Contains the posture features of m standard frontal face images The size is N×m, represents the posture feature of the mth standard head-turning face image, D K Contains the pose offsets of m standard frontal face images D K The size is N×m, represents the posture offset of the mth standard frontal face image, represents the average posture feature of the mth standard frontal face image;

[0125] The corrected posture features are calculated based on the standard posture feature projection of the target user's face image: where qZ represents the corrected posture features of the target user’s face image, represents the average posture feature of the standard frontal face image,

[0126] In one embodiment, in step S33, the texture correction model performs texture feature extraction processing based on the target user face image to obtain the user face texture feature, and performs texture feature correction based on the obtained user face texture feature to obtain the corrected texture feature; including:

[0127] Extract the texture features of the target user’s face image: H A ={h A1 ,h A2 ,…,h AI}, where h Ai represents the texture feature value of the i-th pixel of the target user's face image, i = 1, 2, ... I, I represents the total number of pixels;

[0128] Calculate the texture offset based on the acquired texture features: H dA =H A -H meA ; where H dA The texture offset of the target user’s face image, H me represents the average texture feature value of the target user’s face image, where

[0129] The corrected texture features of the target user's face image are calculated based on the obtained texture feature offset: Among them, H z represents the corrected texture features of the target user’s face image, H meK represents the average standard positive head texture feature, where H meK is obtained based on the average value of the texture features of m standard frontal face images, represents the texture features of the cth standard frontal face image, c = 1, 2, ... m; μ K Contains the texture offsets of m standard frontal face images, represents the texture offset of the mth standard frontal face image, represents the texture features of the mth standard frontal face image, represents the average texture feature value of the mth standard frontal face image; μ J Contains the texture offsets of m standard head-turning face images, represents the texture offset of the mth standard head-turning face image, represents the texture features of the mth standard head-turning face image, Represents the average texture feature value of the mth standard head-turning face image.

[0130] In one implementation, in step S33, the image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image, including:

[0131] According to the corrected posture feature q of the target user's face image Z and the rectified texture feature H of the target user's face image z , using the image fitting model, the target user's face image is iteratively transformed so that the corresponding posture features and texture features of the target user's face image are close to the correction posture features q Z and the corrected texture feature H z , and finally the corrected user face image is obtained according to the result of iterative transformation.

[0132] In one scenario, based on the corrected posture feature q of the target user's face image Z and the rectified texture feature H of the target user's face image z Set the objective function Tar = |q Z -q t | 2 +|H z -H t | 2 ,q t represents the posture features of the target user’s face image in the current iteration, H t Represents the texture features of the target user face image in the current iteration process; transforms the target user face image in the current iteration process through the set model parameters, and recalculates the target function Tar′ based on the transformed target user face image. When Tar′ is less than Tar, continue to use the current model parameters to continue transforming the target user face image; otherwise, re-use the new model parameters to transform the target user face image until the target function converges or the target function is less than the set standard range, end the image transformation, and output the user face image corresponding to the current target function as the corrected user face image.

[0133] In view of the image distortion caused by the traditional head turning correction, the above embodiment of the present invention proposes an improved head turning correction scheme, which transforms the target face image based on the correction posture feature and the correction texture feature, thereby correcting the head turning situation in the target face image and obtaining a front face image that meets the standard. In the process of obtaining the correction posture feature, the posture feature is first constructed according to the distribution information of the key feature points (such as 5 or 83 feature points) of the face image, and the posture offset of each feature point is calculated according to the posture feature, and the posture offset is projected with the feature set corresponding to the standard head turning face image to obtain the head turning posture feature projection, and further the head turning posture feature projection is mapped with the feature corresponding to the standard front face image to obtain the standard posture feature projection, and the posture feature of the user face image is corrected according to the obtained standard posture feature projection to obtain the corrected posture feature of the target user face image, wherein the corrected posture feature of the target user face image is obtained based on the feature of the original target face image, which retains the personalized posture feature information corresponding to the user, and at the same time corrects to obtain the texture feature information that meets the front standard. At the same time, in the process of correcting texture features, similar to the process of posture feature correction, according to the texture features and texture offsets of the extracted face image, combined with the texture features corresponding to the standard head-turning face image and the standard head-up face image, projection and mapping are performed, and the corrected texture features finally obtained retain the personalized texture feature information corresponding to the user, and at the same time, the texture feature information that meets the front standard is corrected. Finally, the objective function is established according to the corrected posture features and corrected texture features obtained, and the target user face image is fitted and transformed through iterative training, and finally the corrected user face image is obtained, wherein the corrected user face image obtained can meet the front standard required for the game character face fusion, and at the same time, the personalized feature information of the user in this application is well retained, which helps to avoid the distortion of the image or the user's features caused by the head-turning correction, while improving the robustness of the game character face fusion, and helps to improve the quality and effect of the game character face image fusion.

[0134] In one implementation, in step S33, performing head tilt correction on the user's face image includes:

[0135] The target user's face image is rotated accordingly according to the obtained head tilt angle to obtain a corrected user's face image.

[0136] Wherein, in case of left or right head tilt in the user's face image, the image is rotated according to the angle of the head tilt to obtain a corrected user's face image.

[0137] The S4 server fuses the corrected target face image into the standard face image of the game character to obtain the user-defined face image of the game character;

[0138] In one implementation, step S4 includes:

[0139] S41 adjusts the skin color of the corrected user face image to a skin color corresponding to the skin color information of the standard face image of the game character, thereby obtaining the user face image after skin color adjustment;

[0140] S42 performs Poisson fusion on the user face image after skin color adjustment and the standard face image of the game character to obtain a user-defined face image of the game character.

[0141] In one scenario, the skin color of the corrected user face image is adjusted according to the standard skin color of the standard face image of the game character, and the skin color of the corrected user face image is linearly transformed to the standard skin color of the standard face image of the game character according to the skin color distribution information of the corrected user face image. At the same time, the user face image with the adjusted skin color is Poisson fused with the standard face image of the game character to obtain the user-defined face image of the game character.

[0142] In one scenario, the color-adjusted user face image can also be texture-fused with the standard face image of a standard game character, and the color-adjusted user face image can be fused into the blank portion corresponding to the standard face image of the standard game character to obtain a user-defined game character face image.

[0143] The S5 server applies the facial image of the user-defined game character to the standard game character to generate the user game character.

[0144] The above-mentioned implementation method proposes an image processing method, which can fuse the acquired user face image into the game character face template selected by the user to obtain the user's personalized game character face image, and generate a corresponding game character based on the obtained game character face image, which helps to improve the personalization level of game character creation.

[0145] Corresponding to the above-mentioned image processing method, see Figure 4 An embodiment shows an image processing system, comprising:

[0146] The template information acquisition module is used to receive the face template information of the game character selected by the user, wherein the face template information includes the standard face image of the game character, wherein the standard face image of the game character includes the face contour information and the skin color information;

[0147] The face image processing module is used to receive the user face image uploaded by the user, pre-process the received user face image, and obtain the target face image;

[0148] A feature processing module is used to perform feature extraction processing based on the obtained target face image to obtain user face feature information, and obtain a corrected user face image based on the obtained user face feature information;

[0149] The image fusion module is used to fuse the corrected target face image into the standard face image of the game character to obtain the user-defined face image of the game character.

[0150] In one embodiment, the system further comprises:

[0151] The character generation module is used to apply the facial image of the user-defined game character to the standard game character to generate the user game character.

[0152] In one embodiment, the method further comprises:

[0153] The character selection module is used to receive the character custom face instructions fed back by the user, and return the selectable game character face template information to the user according to the received instructions.

[0154] In one embodiment, the template information acquisition module includes: providing multiple game character face templates for user selection, wherein each face template is provided with a corresponding game character standard face image, different game character standard face images are provided with different face contour information and skin color information, wherein the face contour information and skin color information are adjusted and set by the user, and when the user completes the setting, receiving the game character standard face image selected by the user and its corresponding face contour information and skin color information.

[0155] In one implementation, in the face image processing module, preprocessing the received user face image includes:

[0156] Performing dimensionality reduction and denoising processing on the received user face image to obtain a first user face image;

[0157] Performing enhancement and adjustment processing on the first user's facial image to obtain a second user's facial image;

[0158] Edge detection and segmentation processing are performed on the second user's facial image, and the size of the segmented facial image is adjusted to obtain a target facial image.

[0159] In one implementation, the feature processing module includes:

[0160] Perform feature point detection processing based on the target user's facial image to obtain key point information of the target user's facial image;

[0161] and calculating the angle features of the user's face based on the obtained key point information, wherein the calculated user angle features include the user's left and right head turning angles and left and right head tilting angles;

[0162] Correcting the target user's face image according to the angle feature of the user's face to obtain a corrected user's face image includes:

[0163] When it is detected that the user's left and right head turning angle is greater than the set head turning threshold, the user's face image is further corrected for head turning;

[0164] When it is detected that the left and right head tilt angles of the user are greater than the set head tilt threshold, the user's facial image is further corrected for head tilt.

[0165] In one implementation, in the feature processing module, performing head turn correction on the user's face image includes:

[0166] Performing head turn correction on the target user's face image based on the trained head turn posture correction model, wherein the input of the head turn posture correction model is the target user's face image, wherein the head turn posture correction model includes a posture correction model, a texture correction model and an image transformation model;

[0167] The posture correction model extracts the face posture of the target user according to the face image to obtain the face posture features of the user, and corrects the face posture features according to the obtained face posture features of the user to obtain the corrected posture features;

[0168] The texture correction model performs texture feature extraction processing based on the target user's face image to obtain the user's face texture features, and performs texture feature correction based on the obtained user's face texture features to obtain the corrected texture features;

[0169] The image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image.

[0170] In one implementation, in the feature processing module, performing head tilt correction on the user's face image includes:

[0171] The target user's face image is rotated accordingly according to the obtained head tilt angle to obtain a corrected user's face image.

[0172] In one implementation, the image fusion module includes:

[0173] Adjusting the skin color of the corrected user face image to a skin color corresponding to the skin color information of the standard face image of the game character, to obtain a user face image after skin color adjustment;

[0174] The user's face image after skin color adjustment is Poisson fused with the standard face image of the game character to obtain the user-defined face image of the game character.

[0175] It should be noted that the functional modules in the image processing system proposed in the above embodiment are used to implement the above Figure 1-3 The specific embodiments corresponding to the steps in the corresponding method in the image processing method are not repeated in this application.

[0176] It should be noted that each functional unit / module in each embodiment of the present invention may be integrated into one processing unit / module, or each unit / module may exist physically separately, or two or more units / modules may be integrated into one unit / module. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of software functional unit / module.

[0177] Through the description of the above implementation mode, it can be clearly understood by those skilled in the art that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the process of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication medium includes any medium that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that can be accessed by a computer. Computer-readable media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should analyze that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. An image processing method, characterized in that: include: S1 receives the face template information of the game character selected by the user, wherein the face template information includes a standard face image of the game character, wherein the standard face image of the game character includes face contour information and skin color information; S2 receives the user face image uploaded by the user, pre-processes the received user face image, and obtains the target face image; S3 performs feature extraction processing based on the obtained target face image to obtain user face feature information, and obtains a corrected user face image based on the obtained user face feature information; S4 fuses the corrected target face image into the standard face image of the game character to obtain a user-defined face image of the game character; Wherein, step S3 comprises: S31 performs feature point detection processing based on the target user's facial image to obtain key point information of the target user's facial image; S32 calculates the angle features of the user's face according to the obtained key point information, wherein the calculated user angle features include the user's left and right head turning angles and left and right head tilting angles; S33 corrects the target user face image according to the angle feature of the user face to obtain a corrected user face image, including: When it is detected that the user's left and right head turning angle is greater than the set head turning threshold, the user's face image is further corrected for head turning; When it is detected that the left and right head tilt angles of the user are greater than the set head tilt threshold, the user's face image is further corrected for head tilt; Among them, in step S33, the head turning correction is performed on the user's face image, including: Performing head turn correction on the target user's face image based on the trained head turn posture correction model, wherein the input of the head turn posture correction model is the target user's face image, wherein the head turn posture correction model includes a shape correction model, a texture correction model and an image transformation model; The shape correction model extracts the face shape of the target user according to the face image to obtain the face posture features of the user, and corrects the posture features according to the obtained face posture features of the user to obtain the corrected posture features, which specifically includes: Extract the posture features of the target user’s face image: q A ={(x A1 ,y A1 ),(x A2 ,y A2 ),…,(x AN ,y AN )}, where q A represents the posture features of the target user’s face image, (x An ,y An ) represents the coordinate value of the nth feature point of the target user's face image, n = 1, 2, ..., N, N represents the total number of feature points; Calculate the posture offset based on the extracted posture features: q dA =q A -q meA ; where q dA represents the pose offset of the target user’s face image, q meA represents the average posture feature of the target user’s face image, where q meA ={(x meA ,y meA ),(x meA ,y meA ),…,(x meA ,y meA )},(x meA ,y meA ) represents the center point coordinates of each feature point in the target user’s face image, where q dA The size is N×1; Calculate the head turning posture feature projection of the target user's face image: where δ A represents the head turning posture feature projection of the target user's face image, where Contains posture features of m standard head-turning face images The size is N×m; represents the posture feature of the mth standard head-turning face image, in The size of is N×m, then δ J The size is m×1; The standard posture feature projection of the target user's face image is calculated based on the head turning posture feature projection of the target user's face image: where δ K represents the standard pose feature projection of the target user’s face image, λ J represents the standard head turning posture feature projection, where Where D J Contains the posture offset of m standard head-turning face images D J The size is N×m, represents the posture offset of the mth standard head-turning face image, represents the average posture feature of the mth standard head-turning face image; K represents the standard head posture feature projection, where in Contains the posture features of m standard frontal face images The size is N×m, represents the posture feature of the mth standard head-turning face image, D K Contains the pose offsets of m standard frontal face images D K The size is N×m, represents the posture offset of the mth standard frontal face image, represents the average posture feature of the mth standard frontal face image; The corrected posture features are calculated based on the standard posture feature projection of the target user's face image: where q Z represents the corrected posture features of the target user’s face image, represents the average posture feature of the standard frontal face image, The texture correction model performs texture feature extraction processing based on the target user's face image to obtain the user's face texture features, and performs texture feature correction based on the obtained user's face texture features to obtain the corrected texture features, specifically including: Extract the texture features of the target user’s face image: H A ={h A1 ,h A2 ,…,h AI }, where h Ai represents the texture feature value of the i-th pixel of the target user's face image, i = 1, 2, ... I, I represents the total number of pixels; Calculate the texture offset based on the acquired texture features: H dA =H A -H meA ; where H dA The texture offset of the target user’s face image, H meA represents the average texture feature value of the target user’s face image, where The corrected texture features of the target user's face image are calculated based on the obtained texture feature offset: Among them, H z represents the corrected texture features of the target user’s face image, H meK represents the average standard positive head texture feature, where H meK is obtained based on the average value of the texture features of m standard frontal face images, represents the texture features of the cth standard frontal face image, c = 1, 2, ... m; μ K Contains the texture offsets of m standard frontal face images, represents the texture offset of the mth standard frontal face image, represents the texture features of the mth standard frontal face image, represents the average texture feature value of the mth standard frontal face image; μ J Contains the texture offsets of m standard head-turning face images, represents the texture offset of the mth standard head-turning face image, represents the texture features of the mth standard head-turning face image, represents the average texture feature value of the mth standard head-turning face image; The image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image, specifically including: According to the corrected posture feature q of the target user's face image Z and the rectified texture feature H of the target user's face image z , using the image fitting model, the target user's face image is iteratively transformed so that the corresponding posture features and texture features of the target user's face image are close to the correction posture features q Z and the corrected texture feature H z , and finally the corrected user face image is obtained according to the result of iterative transformation.

2. An image processing method according to claim 1, characterized in that: Also includes: S5 applies the facial image of the user-defined game character to the standard game character to generate the user game character.

3. An image processing method according to claim 1, characterized in that: Also includes: SB1 receives the character custom face instruction fed back by the user, and returns the selectable game character face template information to the user according to the received instruction.

4. The image processing method according to claim 1, characterized in that: Step S1 specifically includes: providing multiple game character face templates for user selection, wherein each face template is provided with a corresponding game character standard face image, different game character standard face images are provided with different face contour information and skin color information, wherein the face contour information and skin color information are adjusted and set by the user, and when the user completes the setting, receiving the game character standard face image selected by the user and its corresponding face contour information and skin color information.

5. An image processing method according to claim 4, characterized in that: In step S2, the received user face image is preprocessed, including: S21 performs dimensionality reduction and denoising processing on the received user face image to obtain a first user face image; S22 performs enhancement and adjustment processing on the first user face image to obtain a second user face image; S23 performs edge detection and segmentation processing according to the second user's facial image, and adjusts the size of the segmented facial image to obtain a target facial image.

6. An image processing method according to claim 1, characterized in that: In step S33, the head tilt correction is performed on the user's face image, including: The target user's face image is rotated accordingly according to the obtained head tilt angle to obtain a corrected user face image.

7. An image processing method according to claim 1, characterized in that: Step S4 includes: S41 adjusts the skin color of the corrected user face image to a skin color corresponding to the skin color information of the standard face image of the game character, thereby obtaining the user face image after skin color adjustment; S42 performs Poisson fusion on the user face image after skin color adjustment and the standard face image of the game character to obtain a user-defined face image of the game character.

8. An image processing system, characterized in that: include: The template information acquisition module is used to receive the face template information of the game character selected by the user, wherein the face template information includes the standard face image of the game character, wherein the standard face image of the game character includes the face contour information and the skin color information; The face image processing module is used to receive the user face image uploaded by the user, pre-process the received user face image, and obtain the target face image; A feature processing module is used to perform feature extraction processing based on the obtained target face image to obtain user face feature information, and obtain a corrected user face image based on the obtained user face feature information; An image fusion module is used to fuse the corrected target face image into the standard face image of the game character to obtain a user-defined face image of the game character; Among them, the feature processing module includes: Perform feature point detection processing based on the target user's facial image to obtain key point information of the target user's facial image; and calculating the angle features of the user's face based on the obtained key point information, wherein the calculated user angle features include the user's left and right head turning angles and left and right head tilting angles; Correcting the target user's face image according to the angle feature of the user's face to obtain a corrected user's face image includes: When it is detected that the user's left and right head turning angle is greater than the set head turning threshold, the user's face image is further corrected for head turning; When it is detected that the left and right head tilt angles of the user are greater than the set head tilt threshold, the user's face image is further corrected for head tilt; Among them, in the feature processing module, the user's face image is corrected for head turning, including: Performing head turn correction on the target user's face image based on the trained head turn posture correction model, wherein the input of the head turn posture correction model is the target user's face image, wherein the head turn posture correction model includes a shape correction model, a texture correction model and an image transformation model; The shape correction model extracts the face shape of the target user according to the face image to obtain the face posture features of the user, and corrects the posture features according to the obtained face posture features of the user to obtain the corrected posture features, which specifically includes: Extract the posture features of the target user’s face image: q A ={(x A1 ,y A1 ),(x A2 ,y A2 ),…,(x AN ,y AN )}, where q A represents the posture features of the target user’s face image, (x An ,y An ) represents the coordinate value of the nth feature point of the target user's face image, n = 1, 2, ..., N, N represents the total number of feature points; Calculate the posture offset based on the extracted posture features: q dA =q A -q meA ; where q dA represents the pose offset of the target user’s face image, q meA represents the average posture feature of the target user’s face image, where q meA ={(x meA ,y meA ),(x meA ,y meA ),…,(x meA ,y meA )},(x meA ,y meA ) represents the center point coordinates of each feature point in the target user’s face image, where q dA The size is N×1; Calculate the head turning posture feature projection of the target user's face image: where δ A represents the head turning posture feature projection of the target user's face image, where Contains posture features of m standard head-turning face images The size is N×m; represents the posture feature of the mth standard head-turning face image, in The size of is N×m, then δ J The size is m×1; The standard posture feature projection of the target user's face image is calculated based on the head turning posture feature projection of the target user's face image: where δ K represents the standard pose feature projection of the target user’s face image, λ J represents the standard head turning posture feature projection, where Where D J Contains the posture offset of m standard head-turning face images D J The size is N×m, represents the posture offset of the mth standard head-turning face image, represents the average posture feature of the mth standard head-turning face image; K represents the standard head posture feature projection, where in Contains the posture features of m standard frontal face images The size is N×m, represents the posture feature of the mth standard head-turning face image, D K Contains the pose offsets of m standard frontal face images D K The size is N×m, represents the posture offset of the mth standard frontal face image, represents the average posture feature of the mth standard frontal face image; The corrected posture features are calculated based on the standard posture feature projection of the target user's face image: where q Z represents the corrected posture features of the target user’s face image, represents the average posture feature of the standard frontal face image, The texture correction model performs texture feature extraction processing based on the target user's face image to obtain the user's face texture features, and performs texture feature correction based on the obtained user's face texture features to obtain the corrected texture features, specifically including: Extract the texture features of the target user’s face image: H A ={h A1 ,h A2 ,…,h AI }, where h Ai represents the texture feature value of the i-th pixel of the target user's face image, i = 1, 2, ... I, I represents the total number of pixels; Calculate the texture offset based on the acquired texture features: H dA =H A -H meA ; where H dA The texture offset of the target user’s face image, H meA represents the average texture feature value of the target user’s face image, where The corrected texture features of the target user's face image are calculated based on the obtained texture feature offset: Among them, H z represents the corrected texture features of the target user’s face image, H meK represents the average standard positive head texture feature, where H meK is obtained based on the average value of the texture features of m standard frontal face images, represents the texture features of the cth standard frontal face image, c = 1, 2, ... m; μ K Contains the texture offsets of m standard frontal face images, represents the texture offset of the mth standard frontal face image, represents the texture features of the mth standard frontal face image, represents the average texture feature value of the mth standard frontal face image; μ J Contains the texture offsets of m standard head-turning face images, represents the texture offset of the mth standard head-turning face image, represents the texture features of the mth standard head-turning face image, represents the average texture feature value of the mth standard head-turning face image; The image transformation model transforms the target face image according to the obtained corrected posture features and corrected texture features to obtain a corrected user face image, specifically including: According to the corrected posture feature q of the target user's face image Z and the rectified texture feature H of the target user's face image z , using the image fitting model, the target user's face image is iteratively transformed so that the corresponding posture features and texture features of the target user's face image are close to the correction posture features q Z and the corrected texture feature H z , and finally the corrected user face image is obtained according to the result of iterative transformation.

Images