Artificial intelligence-based expression package generation method, device, equipment and storage medium
Through artificial intelligence technology, the face detection model and image segmentation model are used, combined with emoticon package templates are used for processing, which solves the problems of cumbersome operation and poor results of the existing emoticon package generation software, and achieves the rapid and accurate generation of high-quality emoticon packages. .
Patent Information
- Application Number
- CN202010724846.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-24
- Publication Date
- 2024-11-19
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing emoticon package generation software is cumbersome to operate and has poor generation results, and has a single background, making it difficult to meet the diverse needs of users.
Using an artificial intelligence-based method, the key points of the original portrait image are obtained through the face detection model, and screenshot, correction and adaptation are combined with the target emoji package template, and the pre-trained image segmentation model is input to generate high-quality emoji packages.
It realizes the rapid, automated and accurate generation of high-quality emoticon packages, reduces operation difficulty, improves generation efficiency and effect, and meets users' needs for diversified emoticon packages.
Smart Images

Figure CN111860372B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing, and in particular to an artificial intelligence-based expression package generation method, device, equipment and storage medium. Background Art
[0002] Nowadays, more and more people like to use emoticons when chatting. Although there are many emoticon generation software on the market, the existing emoticon generation software often requires a long time of focusing and cutting out when generating emoticons, which is cumbersome to operate, or the background of the existing emoticon generation process is single, and the effect of generating emoticons is not good. Summary of the invention
[0003] The embodiments of the present invention provide an artificial intelligence-based emoticon package generation method, device, equipment and storage medium to solve the problems of complicated operation and poor effect of generating emoticon packages.
[0004] An artificial intelligence-based expression package generation method, comprising:
[0005] Get the original portrait image including the head portrait;
[0006] Using a face detection model to detect the original portrait image to obtain an original key point map;
[0007] Obtain a target expression package template, and perform screenshot processing on the original key point map based on the target expression package template to obtain a target key point map;
[0008] Performing correction processing on the target key point map based on the target expression package template to obtain a target correction map;
[0009] According to the target expression package template, the target correction image is adapted to obtain a standard portrait image;
[0010] Inputting the standard portrait image into a pre-trained image segmentation model to obtain a target portrait image;
[0011] The target expression package template and the target portrait image are combined to generate a target expression package corresponding to the target expression package template.
[0012] An artificial intelligence-based emoticon package generation device, comprising:
[0013] The original portrait image acquisition module is used to acquire the original portrait image including the head portrait;
[0014] An original key point map acquisition module is used to detect the original portrait image using a face detection model to obtain an original key point map;
[0015] A target key point map acquisition module is used to acquire a target expression package template, and to perform screenshot processing on the original key point map based on the target expression package template to acquire a target key point map;
[0016] A target correction image acquisition module, used for performing correction processing on the target key point image based on the target expression package template to obtain a target correction image;
[0017] A standard portrait image acquisition module is used to adapt the target correction image according to the target expression package template to obtain a standard portrait image;
[0018] A target portrait image acquisition module is used to input the standard portrait image into a pre-trained image segmentation model to obtain a target portrait image;
[0019] The target expression package generation module is used to combine the target expression package template and the target portrait image to generate a target expression package corresponding to the target expression package template.
[0020] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned artificial intelligence-based emoticon package generation method when executing the computer program.
[0021] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned artificial intelligence-based emoticon package generation method.
[0022] The above-mentioned artificial intelligence-based expression package generation method, device, computer equipment and storage medium can quickly determine the facial features and facial contours of the original portrait image by using a face detection model to detect the original portrait image and obtain the original key point map, providing a basis for the subsequent generation of the target expression package. Obtain the target expression package template, and take a screenshot of the original key point map based on the target expression package template to obtain the target key point map to cut off the background and reduce noise, thereby reducing the difficulty of subsequent segmentation, ensuring the accuracy of subsequent segmentation, and avoiding the occurrence of mis-segmentation. Correct the target key point map based on the target expression package template to obtain the target correction map, ensuring that the angle of the target correction map is the same as the angle of the portrait in the target expression package template, which is conducive to speeding up the subsequent target artificial intelligence-based expression package generation speed and ensuring the accuracy of the automatically generated target expression package template. The target correction image is adapted according to the target expression package template to obtain a standard portrait image. The portrait in the standard portrait image is consistent with the size of the portrait to be replaced in the expression package template, which can ensure that the target expression package generated subsequently is better and more realistic, and realize seamless generation of the target expression package. The standard portrait image is input into a pre-trained image segmentation model to obtain a target portrait image to improve the image segmentation efficiency and realize accurate segmentation of the standard portrait image. The target expression package template and the target portrait image are combined to generate a target expression package corresponding to the target expression package template, thereby realizing intelligent generation of the target expression package. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0024] Figure 1 It is a schematic diagram of an application environment of an emoticon package generation method based on artificial intelligence in one embodiment of the present invention;
[0025] Figure 2 is a flow chart of a method for generating an emoticon package based on artificial intelligence in one embodiment of the present invention;
[0026] Figure 3 is another flow chart of a method for generating an emoticon package based on artificial intelligence in one embodiment of the present invention;
[0027] Figure 4 is another flow chart of a method for generating an emoticon package based on artificial intelligence in one embodiment of the present invention;
[0028] Figure 5is another flow chart of a method for generating an emoticon package based on artificial intelligence in one embodiment of the present invention;
[0029] Figure 6 is another flow chart of a method for generating an emoticon package based on artificial intelligence in one embodiment of the present invention;
[0030] Figure 7 is another flow chart of a method for generating an emoticon package based on artificial intelligence in one embodiment of the present invention;
[0031] Figure 8 is a schematic diagram of an artificial intelligence-based emoticon package generating device in one embodiment of the present invention;
[0032] Fig. 9 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION
[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0034] The artificial intelligence-based expression package generation method provided in the embodiment of the present invention can be applied as follows: Figure 1 Specifically, the artificial intelligence-based expression package generation method is applied in an artificial intelligence-based expression package generation system, and the artificial intelligence-based expression package generation system includes: Figure 1 The client and server shown in the figure communicate with each other through the network to realize the intelligent generation of target emoticon packages. The client, also known as the user end, refers to the program corresponding to the server and providing local services to customers. The client can be installed on, but not limited to, various personal computers, laptops, smart phones, tablets, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.
[0035] In one embodiment, if Figure 2 As shown, a method for generating emoticons based on artificial intelligence is provided, and the method is applied in Figure 1 The server in the example is used as an example to illustrate the process, including the following steps:
[0036] S201: Obtain an original portrait image including an avatar.
[0037] The original portrait image is an image containing a head portrait collected by a collection module on a computer device. The original portrait image includes a head portrait and a background, and the background is the noise in the original portrait image except the head portrait. Noise is a factor that interferes with the head portrait, for example, noise is the natural environment in the original portrait image. In this embodiment, the client can collect the original portrait image containing the head portrait through the collection module on the computer device and upload it to the server so that the server obtains the original portrait image; or the server can crawl the original portrait image containing the head portrait from a picture website. The collection module includes but is not limited to camera shooting and local uploading.
[0038] S202: Use a face detection model to detect the original portrait image to obtain an original key point map.
[0039] The face detection model refers to a pre-trained model for detecting facial features and facial contours in an original portrait image. Facial features include but are not limited to eyebrows, eyes, nose, and mouth.
[0040] In this embodiment, the face detection model is a model obtained by training using the dlib detection algorithm, which is used to quickly detect and obtain the original key point map. The dlib detection algorithm specifically includes HOG features and a regression tree algorithm. HOG is the abbreviation of histogram of oriented gradient, which means gradient direction histogram. The gradient direction histogram is a description operator based on shape edge features that can detect objects. Its basic idea is to use gradient information to well reflect the edge information of image targets and characterize the local appearance and shape of the image through the size of local gradients. The regression tree algorithm refers to the ERT (ensemble of regression trees) cascade regression tree algorithm, that is, a regression tree method based on gradient boosting learning. The cascade regression tree algorithm uses cascade regression factors and requires a series of calibrated face images as a training set, and then generates a model.
[0041] The original key point graph is a graph in which the facial features and facial contours in the original portrait image are detected using a face detection model, and the facial features and facial contours are marked using original key points. It should be noted that each original key point includes point coordinates for subsequent correction processing. In the present embodiment, the original portrait image is detected using a face detection model, and the facial features and facial contours of the original portrait image can be quickly detected, providing a basis for the subsequent generation of a target expression package.
[0042] S203: Obtain a target expression package template, and take a screenshot of the original key point map based on the target expression package template to obtain a target key point map.
[0043] Among them, the target expression package template refers to a template containing portrait expressions that is pre-stored in the data platform for users to choose. In this example, the user can operate the client to independently select the target expression package template. The target key point map refers to the picture obtained after removing the background from the original key point map. Screenshot processing refers to a method of taking a screenshot of the original key point map and removing part of the background. In this embodiment, the original key point map is screenshoted based on the target expression package template to cut off the background and reduce noise, thereby reducing the difficulty of subsequent segmentation, ensuring the accuracy of subsequent segmentation, and avoiding the occurrence of mis-segmentation.
[0044] As an example, when the acquisition module is used to collect the original portrait image, the user's expression is recognized, the user's expression recognition result is obtained, and according to the user's expression recognition result, a target expression package template corresponding to the expression recognition result is recommended to the user, so as to realize the intelligent generation of the target expression package. For example, when the expression recognition result is happy, a relatively happy target expression package template is recommended to the user.
[0045] S204: Correct the target key point image based on the target expression package template to obtain a target correction image.
[0046] The target correction image refers to the image obtained by processing the portrait in the target key point image according to the angle of the portrait to be replaced in the target emoji template. Correction processing refers to the process of adjusting the target key point image according to the angle of the portrait in the target emoji template. The portrait to be replaced refers to the portrait that needs to be replaced in the target emoji template.
[0047] Specifically, the offset angle between the portrait in the target key point image and the portrait to be replaced is calculated according to the target emoji package template, and the portrait in the target key point image is corrected according to the offset angle to ensure that the angle of the target correction image is the same as the angle of the portrait in the target emoji package template, which is conducive to speeding up the subsequent target artificial intelligence-based emoji package generation speed and ensuring the accuracy of the automatically generated target emoji package template. As an example, the portrait to be replaced refers to the face in the target emoji package template. As another example, the portrait to be replaced refers to the face and head in the target emoji package template.
[0048] S205: Adapt the target correction image according to the target expression package template to obtain a standard portrait image.
[0049] Among them, the adaptation process refers to a method of enlarging or reducing the target correction image according to the size of the portrait to be replaced in the target expression package template, so that the portrait in the target correction image is the same size as the portrait to be replaced. The standard portrait image refers to a picture obtained after the target correction image is adapted. In this embodiment, the portrait in the standard portrait image is consistent with the size of the portrait to be replaced in the target expression package template, which can ensure that the target expression package generated subsequently has better effect and is more realistic, and the target expression package can be seamlessly generated.
[0050] S206: Input the standard portrait image into a pre-trained image segmentation model to obtain a target portrait image.
[0051] Among them, the target portrait image is obtained by processing the standard portrait image using an image segmentation model to obtain an image containing only the target portrait. The target portrait image has eliminated the interference of the background and can be accurately synthesized with the target emoticon package template to achieve automatic generation of emoticons.
[0052] The image segmentation model is an image segmentation model for the pixel level. In this embodiment, the image segmentation model uses the deeplabv3 plus algorithm to improve the efficiency of image segmentation and accurately segment the standard portrait image. The deeplabv3plus algorithm can accurately segment the boundary between the portrait and the background of the standard portrait image by adding a simple and effective decoding module to refine the segmentation results. Since the original key point image has been screenshot processed in step S203, the proportion of the target portrait in the obtained target key point image is much larger than the background. Therefore, this embodiment simplifies the deeplabv3 plus algorithm, specifically replacing the xception backbone network of the deeplabv3 plus algorithm with a target shallow network composed of a convolutional layer and a downsampling layer to speed up the operation.
[0053] S207: Combine the target expression package template and the target portrait image to generate a target expression package corresponding to the target expression package template.
[0054] The target emoticon package refers to an emoticon package formed by replacing the portrait to be replaced in the target emoticon package template with the target portrait.
[0055] In this embodiment, Alpha blending is used to superimpose the target expression package template and the target portrait image together to form a target expression package. In this embodiment, the parameter formula x=Alpha / 255, that is, the pixel value of the alpha mask corresponding to the target portrait image is divided by 255, and the image synthesis formula: new=image*x+(1-x)*bg are used to process the target portrait image and the target expression package template to automatically synthesize the target expression package and realize intelligent generation of the target expression package. Among them, bg refers to the color corresponding to the target expression package template, and image is the color corresponding to the target portrait image. It can be understood that in order to make the display effect of the synthesized target expression package more effective, it is necessary to improve the data accuracy during actual calculation, for example, x=double(Alpha) / 255.00.
[0056] The method for generating an emoticon package based on artificial intelligence provided in this embodiment uses a face detection model to detect the original portrait image, obtains the original key point map, can quickly determine the facial features and facial contours of the original portrait image, and provides a basis for the subsequent generation of a target emoticon package. Obtain a target emoticon package template, and perform screenshot processing on the original key point map based on the target emoticon package template to obtain a target key point map to cut off the background and reduce noise, thereby reducing the difficulty of subsequent segmentation, ensuring the accuracy of subsequent segmentation, and avoiding the occurrence of mis-segmentation. Correct the target key point map based on the target emoticon package template to obtain a target correction map, ensure that the angle of the target correction map is the same as the angle of the portrait in the target emoticon package template, which is conducive to speeding up the subsequent target emoticon package generation speed based on artificial intelligence and ensuring the accuracy of the automatic generation of the target emoticon package template. According to the target emoticon package template, the target correction map is adapted to obtain a standard portrait map, and the portrait in the standard portrait map is consistent with the size of the portrait to be replaced in the emoticon package template, which can ensure that the subsequent generated target emoticon package has a better effect and is more realistic, and the target emoticon package is seamlessly generated. Input the standard portrait image into the pre-trained image segmentation model to obtain the target portrait image to improve the image segmentation efficiency and accurately segment the standard portrait image. Combine the target expression package template and the target portrait image to generate a target expression package corresponding to the target expression package template, and realize intelligent generation of the target expression package.
[0057] In one embodiment, if Figure 3 As shown, step S203, i.e., obtaining a target expression package template, taking a screenshot of the original key point graph based on the target expression package template to obtain the target key point graph, includes:
[0058] S301: If the target expression package template is a face expression package template, generate a target key point frame according to the original key points of the original key point map, and based on the target key point frame, use a screenshot tool to intercept the original key point map to obtain the target key point map.
[0059] The face expression package template refers to a target expression package template that only contains a face. Therefore, when generating the target expression package, it is only necessary to intercept the face of the original key point map. The original key point refers to the point corresponding to the face features and the face contour in the original key point map.
[0060] The target key point frame refers to the key area of the face located in the original key point map, including eyebrows, eyes, nose and mouth, etc. It should be noted that in this example, the target key point frame is matched with the face expression package template.
[0061] Specifically, the target expression package template is selected by the customer independently, so as to flexibly generate the corresponding target portrait template according to different customers. When the target expression package template is a face expression package template, it is only necessary to intercept the face area of the original key point map. Therefore, the target key point frame is generated according to the original portrait key points on the original key point map, and the screenshot tool is called to intercept the target key point frame to obtain the target key point map, so that the target key point map only contains the face, so that the subsequent segmentation processing is simple and accurate, and the original key point map is intercepted to reduce the background, that is, to reduce the noise, so that the subsequent segmentation processing is simple and accurate.
[0062] S302: If the target emoticon package template is a head emoticon package template, generate a key point frame to be processed according to the original key points of the original key point map, enlarge the key point frame according to a preset ratio, obtain a target key point frame including the head, and based on the target key point frame, use a screenshot tool to capture the original key point map for screenshot processing to obtain the target key point map.
[0063] Among them, the head expression package template refers to the target expression package template including the face and head. Therefore, when generating the target expression package, only the face and head of the original key point image need to be intercepted.
[0064] The key point box to be processed is a box generated according to the original key points in the original key point map.
[0065] The target key point frame refers to the frame that includes the face and head. Since the target expression package in this example is a head expression package template, and the original key points detected by the face detection model do not include the head, it is also necessary to enlarge the key point frame to be processed according to the preset ratio so that the target key point frame includes the face and head, so that the target key point map is adapted to the head expression package template.
[0066] The preset scale is a preset value used to enlarge the key point box to be processed.
[0067] Specifically, when the target expression package template is a head expression package template, since the original key points detected by the face detection model do not include the head, it is also necessary to enlarge the key point frame to be processed according to a preset ratio, obtain the target key point frame including the face and the head, and call the screenshot tool to intercept the target key point frame to obtain the target portrait key map, so that the target key point map includes the face and the head. In this example, the original key point map is intercepted to reduce the background, that is, to reduce the noise, so that the subsequent segmentation process is simple and accurate.
[0068] The artificial intelligence-based expression package generation method provided in this embodiment, when the target expression package template is a face expression package template, generates a target key point frame according to the original key points of the original key point map, and based on the target key point frame, uses a screenshot tool to intercept and process the original key point map to obtain the target key point map. When the target expression package template is a head expression package template, generates a key point frame to be processed according to the original key points of the original key point map, enlarges the key point frame according to a preset ratio, obtains a target key point frame including the head, and based on the target key point frame, uses a screenshot tool to intercept and process the original key point map to obtain the target key point map, and intercepts and processes the original key point map to reduce the background, that is, reduce the noise, so that the subsequent segmentation process is simple and accurate.
[0069] In one embodiment, if Figure 4 As shown, step S204, i.e., correcting the target key point map based on the target expression package template to obtain the target correction map, includes:
[0070] S401: Obtain the point coordinates of the template key points of the portrait to be replaced in the target expression package template.
[0071] Among them, the template key points refer to the facial features or facial contour points corresponding to the portrait to be replaced.
[0072] Specifically, the target expression package template is input into the face detection model, and the template key point map corresponding to the target expression package template is output, and the coordinates of the two template key points are obtained in the template key point map. In this example, the coordinates of the template key points are obtained so as to subsequently determine the face offset angle of the target expression package and the target portrait key point map. Among them, the face offset angle refers to the offset angle of the portrait in the target key point map relative to the replacement portrait in the target expression package template.
[0073] S402: Obtain the point coordinates of the target key point in the target key point map, and determine the face offset angle of the target key point map according to the coordinates of the template key point and the point coordinates of the target key point.
[0074] The target key points refer to the points that mark the facial features and facial contours in the target key point map. It should be noted that the target key points are the same as the original key points. In order to ensure accuracy, the template key points and the target key points are the points corresponding to the same facial features. For example, if the template key points are the points corresponding to the eyes of the portrait to be replaced, then the target key points are the points corresponding to the eyes of the portrait in the target portrait key points.
[0075] Specifically, the point coordinates of the two template key points and the point coordinates corresponding to the two target key points are obtained, and the first straight line is determined according to the coordinates of the two template key points, and the second straight line is determined according to the point coordinates of the two target key points, and the angle between the first straight line and the second straight line is determined as the face offset angle. It should be noted that the target expression package template and the target key point map use the same face detection model, and the template key points and the target key points are determined in the same coordinate system to ensure the accuracy of the correction process.
[0076] S403: Correct the target key point map according to the face offset angle to obtain a target correction map.
[0077] The correction process refers to the process of rotating the target key point map. The target correction map refers to the map obtained after the target key point map is processed. In this example, the target key point map is corrected to ensure that the portrait in the target correction map is consistent with the angle of the portrait to be replaced in the target emoji template, so that the target emoji can be accurately generated later.
[0078] The artificial intelligence-based emoji generation method provided in this embodiment obtains the coordinates of the template key points of the portrait to be replaced in the target emoji template, so as to subsequently determine the face offset angle of the target emoji and the target portrait key point map. The point coordinates corresponding to the target key point in the target key point map are obtained, and the face offset angle of the target key point map is determined based on the coordinates of the template key point and the point coordinates of the target key point to ensure the accuracy of the correction processing. According to the face offset angle, the target key point map is corrected to obtain a target correction map to ensure that the portrait in the target correction map is consistent with the angle of the portrait to be replaced in the target emoji template.
[0079] In one embodiment, if Figure 5 As shown, step S205, i.e., performing adaptation processing on the target correction image based on the target expression package template to obtain the target key point image, includes:
[0080] S501: Obtain the length and height of the portrait to be replaced in the target emoticon template.
[0081] In this example, when the target emoticon package template is a face emoticon package template, the length and height of the portrait to be replaced refer to the length and height of the face; when the target emoticon package template is a head emoticon package template, the length and height of the portrait to be replaced refer to the length and height of the face and head.
[0082] Specifically, a plurality of original emoticon templates are pre-stored in the system, and the original emoticon templates carry template information. The user selects the target emoticon template from the original emoticon template according to his or her own preferences, and quickly determines the length and height of the portrait to be replaced according to the template information corresponding to the target emoticon template, so that the target emoticon package can be quickly generated later. The template information includes template labels, the length and height of the portrait to be replaced in the original emoticon template, and other information. The template label refers to the label corresponding to the original emoticon template, and the template label refers to a face label or a head label. The face label indicates that the original emoticon template is a face emoticon template, and the head label indicates that the original emoticon template is a head emoticon template.
[0083] S502: Based on the length and height of the portrait to be replaced, the target correction image is adapted using an image processing model to obtain a target key point image.
[0084] Among them, the image processing model is a model for enlarging or reducing the target correction image. The target key point map refers to the image obtained after the target correction image is enlarged or reduced. In this example, after obtaining the length and height of the portrait to be replaced, the length and height of the portrait in the target correction image are determined according to the target key point frame determined in step S301 or S302, and then the relative proportion of the portrait in the target correction image and the portrait to be replaced is calculated. According to the relative proportion, the target correction image is enlarged or reduced using the image processing model to obtain the target key point map, so that the portrait in the target key point map is consistent with the length and height of the portrait to be replaced, ensuring that the target portrait image that is consistent with the size of the portrait to be replaced is subsequently segmented, providing technical support for the subsequent generation of the target emoticon package.
[0085] The artificial intelligence-based expression package generation method provided in this embodiment obtains the length and height of the portrait to be replaced in the target expression package template, so that the target expression package can be quickly generated later. Based on the length and height of the portrait to be replaced, the target correction image is adapted by using the image processing model to obtain the target key point image, ensuring that the target portrait image with the same size as the portrait to be replaced is subsequently segmented, providing technical support for the subsequent generation of the target expression package.
[0086] In one embodiment, if Figure 6 As shown, step S206, that is, inputting the standard portrait image into the pre-trained image segmentation model to obtain the target portrait image, includes:
[0087] S601: Input the standard portrait image into a pre-trained target shallow network, and process the standard portrait image based on the convolution layer and downsampling layer in the target shallow network to obtain a thumbnail.
[0088] The thumbnail refers to the image obtained after the standard portrait image is processed by the target shallow network. The convolution layer and the downsampling layer are used to process the standard portrait image to extract features from the standard portrait image and compress the image to speed up the processing.
[0089] Specifically, the standard portrait image is input into the target shallow network, and the convolutional layer in the target shallow network is used to collect the convolutional features of the standard portrait image to obtain a convolutional feature map. The convolutional feature map is processed by a downsampling layer to obtain a thumbnail corresponding to the convolutional feature map, and the convolutional feature map is compressed by dimensionality reduction to speed up subsequent calculations.
[0090] Since the original key point map has been screenshotted, the proportion of the target portrait in the obtained target key point map is much larger than the background. Therefore, this embodiment simplifies the deeplabv3 plus algorithm, specifically replacing the xception backbone network of the deeplabv3plus algorithm with a target shallow network composed of a convolutional layer and a downsampling layer to speed up the calculation.
[0091] S602: Process the thumbnail using dilated convolution and global pooling to obtain facial feature information, and obtain a target portrait image from the standard portrait image based on the facial feature information.
[0092] The facial feature information includes boundary information between the face and the background, and the face and the background can be segmented according to the facial feature information.
[0093] Since S601 uses a downsampling layer, the pixel rate of the thumbnail is reduced and local information is lost. In order to extract more context information and ensure accurate image segmentation in the future, it is necessary to use dilated convolution and global pooling to process the thumbnail to expand the field of view to obtain more facial feature information, to obtain more image context information, and to accurately segment the portrait and background. The decoding module is used to finely segment the portrait and background to obtain the target portrait image.
[0094] The artificial intelligence-based expression package generation method provided in this embodiment inputs the standard portrait image into a pre-trained target shallow network, processes the standard portrait image based on the convolution layer and downsampling layer in the target shallow network, and obtains a thumbnail. The thumbnail is processed using dilated convolution and global pooling to obtain facial feature information, and the target portrait image is obtained from the standard portrait image based on the facial feature information.
[0095] In one embodiment, if Figure 7 As shown, step S201, i.e. obtaining an original portrait image including a head portrait, includes:
[0096] S701: Obtain an initial image including the avatar, and determine whether a pixel value of the initial image is not less than a preset pixel threshold.
[0097] The initial image is a picture containing a headshot uploaded by the customer, and the preset pixel threshold is a pre-set pixel value used to determine whether the initial image can be used.
[0098] Specifically, the camera module carried by the client is used to shoot the initial image, and the pixel value corresponding to the initial image is compared with the preset pixel threshold to determine whether the pixel value corresponding to the initial image is not less than the preset pixel value. The initial image with a pixel value not less than the preset pixel threshold is used as the original portrait image, and the initial image with a pixel value less than the preset pixel threshold is excluded to ensure that the target portrait image generated subsequently is clear. As another example, the initial image can also be obtained by the user by screening pictures stored locally.
[0099] S702: If the pixel value of the initial image is not less than the preset pixel threshold, the initial image is determined as the original portrait image.
[0100] In this example, when the pixel value of the initial image is not less than the preset pixel threshold, the initial image is determined as the original portrait image to ensure that the original portrait image is a clear image to ensure that subsequent processing of the original portrait image can obtain a clear target emoticon package.
[0101] S703: If the pixel value of the initial image is less than the preset pixel threshold, a reminder message is generated and sent to the client, and the process is repeated to obtain the initial image including the avatar.
[0102] The reminder information is used to remind the client that the initial image uploaded does not have enough pixels and needs to be re-uploaded. It can be understood that when the pixel value of the initial image is less than the preset pixel threshold, the initial image is not clear enough. Therefore, a reminder message is generated and sent to the client to enable the user client to re-upload the initial image to ensure that a sufficiently clear target emoticon package can be generated at one time.
[0103] The artificial intelligence-based emoticon package generation method provided in this embodiment obtains an initial image containing an avatar, and determines whether the pixel value of the initial image is not less than a preset pixel threshold to ensure that the target portrait image generated subsequently is clear. When the pixel value of the initial image is not less than the preset pixel threshold, the initial image is determined as the original portrait image to ensure that the original portrait image is a clear image to ensure that a clear target emoticon package can be obtained by subsequent processing of the original portrait image. When the pixel value of the initial image is less than the preset pixel threshold, a reminder message is generated and sent to the client, and the initial image containing the avatar is obtained repeatedly to enable the user client to re-upload the initial image to ensure that a sufficiently clear target portrait image can be generated at one time.
[0104] In one embodiment, step S201, that is, after obtaining the original portrait image containing the avatar, the artificial intelligence-based emoticon package generation method also includes: obtaining the number of original portrait images, if the number of original portrait images is greater than a preset number threshold, executing a multi-threaded processing mechanism, calling at least two threads to respectively execute the use of a face detection model to detect the original portrait image, and obtaining an original key point map.
[0105] The preset number threshold is a preset threshold of the original portrait images, and the preset number threshold is used to determine whether to start the multi-thread processing mechanism. The multi-thread processing mechanism refers to a mechanism that uses at least two threads to process the original portrait images. This example uses a corresponding number of threads to process the original portrait images according to the number of original portrait images to speed up the processing speed of the original image. For example, if the number of original portrait images is less than the preset number threshold, 1 thread is used if the number of original portrait images is greater than the preset number threshold; if the number of original portrait images is twice the preset number threshold, 2 threads are used if the number of original portrait images is greater than the preset number threshold.
[0106] Specifically, when the number of original portrait images is greater than a preset threshold, in order to speed up the processing, the required number of threads is determined according to the number of original portrait images, and a corresponding number of threads are called to respectively execute the detection of the original portrait images using the face detection model to obtain the original key point map.
[0107] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0108] In one embodiment, an artificial intelligence-based emoticon package generation device is provided, and the artificial intelligence-based emoticon package generation device corresponds one-to-one to the artificial intelligence-based emoticon package generation method in the above embodiment. Figure 8As shown, the artificial intelligence-based expression package generation device includes an original portrait image acquisition module 801, an original key point image acquisition module 802, a target key point image acquisition module 803, a target correction image acquisition module 804, a standard portrait image acquisition module 805, a target portrait image acquisition module 806 and a target expression package generation module 807. The functional modules are described in detail as follows:
[0109] The original portrait image acquisition module 801 is used to acquire an original portrait image including a head portrait.
[0110] The original key point map acquisition module 802 is used to detect the original portrait image using a face detection model to obtain an original key point map.
[0111] The target key point map acquisition module 803 is used to acquire a target expression package template, and to perform screenshot processing on the original key point map based on the target expression package template to acquire the target key point map.
[0112] The target correction image acquisition module 804 is used to perform correction processing on the target key point image based on the target expression package template to obtain the target correction image.
[0113] The standard portrait image acquisition module 805 is used to adapt the target correction image according to the target expression package template to obtain a standard portrait image.
[0114] The target portrait image acquisition module 806 is used to input the standard portrait image into a pre-trained image segmentation model to obtain a target portrait image.
[0115] The target expression package generation module 807 is used to combine the target expression package template and the target portrait image to generate a target expression package corresponding to the target expression package template.
[0116] Preferably, the target key point map acquisition module 803 includes: a first unit and a second unit.
[0117] Unit 1: If the target expression package template is a face expression package template, a target key point frame is generated according to the original key points of the original key point map, and based on the target key point frame, a screenshot tool is used to intercept the original key point map to obtain the target key point map.
[0118] The second unit is used to generate a key point frame to be processed according to the original key points of the original key point map if the target emoji package template is a head emoji package template, enlarge the key point frame according to a preset ratio, obtain a target key point frame including the head, and based on the target key point frame, use a screenshot tool to capture the original key point map for screenshot processing to obtain the target key point map.
[0119] Preferably, the target correction image acquisition module 804 includes: a template key point point coordinate acquisition unit, a face offset angle acquisition unit and a correction processing unit.
[0120] The point coordinate acquisition unit of the template key point is used to obtain the point coordinates of the template key point of the portrait to be replaced in the target expression package template.
[0121] The face offset angle acquisition unit is used to obtain the point coordinates of the target key point in the target key point map, and determine the face offset angle of the target key point map according to the coordinates of the template key point and the point coordinates of the target key point.
[0122] The correction processing unit is used to perform correction processing on the target key point map according to the face offset angle to obtain the target correction map.
[0123] Preferably, the standard portrait image acquisition module 805 includes: a length and height acquisition unit of the portrait to be replaced and an adaptation processing unit.
[0124] The length and height acquisition unit of the portrait to be replaced is used to acquire the length and height of the portrait to be replaced in the target emoticon package template.
[0125] The adaptation processing unit is used to adapt the target correction image based on the length and height of the portrait to be replaced by using the image processing model to obtain a standard portrait image.
[0126] Preferably, the target expression package generating module 807 includes:
[0127] The thumbnail acquisition unit is used to input the standard portrait image into a pre-trained target shallow network, process the standard portrait image based on the convolution layer and the downsampling layer in the target shallow network, and obtain a thumbnail.
[0128] The facial feature information acquisition unit is used to process the thumbnail using dilated convolution and global pooling to obtain facial feature information, and obtain a target portrait image from the standard portrait image based on the facial feature information.
[0129] Preferably, the original portrait image acquisition module 801 includes: a judgment unit, an original portrait image determination unit and a reminder information generation unit.
[0130] The judging unit is used to obtain an initial image including the avatar and judge whether the pixel value of the initial image is not less than a preset pixel threshold.
[0131] The original portrait image determining unit is used to determine the initial image as the original portrait image if the pixel value of the initial image is not less than a preset pixel threshold.
[0132] The reminder information generating unit is used to generate reminder information and send it to the client if the pixel value of the initial image is less than a preset pixel threshold, and repeatedly execute to obtain the initial image containing the avatar.
[0133] Preferably, after the original portrait image acquisition module 801, the emoticon package generation device based on artificial intelligence further includes: an original portrait image quantity acquisition unit.
[0134] The number acquisition unit of the original portrait images is used to obtain the number of original portrait images. If the number of original portrait images is greater than a preset number threshold, a multi-thread processing mechanism is executed to call at least two threads to respectively execute detection of the original portrait images using a face detection model to obtain an original key point map.
[0135] For the specific definition of the emoticon package generation device based on artificial intelligence, please refer to the definition of the emoticon package generation method based on artificial intelligence above, which will not be repeated here. Each module in the above-mentioned emoticon package generation device based on artificial intelligence can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0136] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store original expression package templates. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an expression package generation method based on artificial intelligence is implemented.
[0137] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for generating an emoticon package based on artificial intelligence in the above embodiment are implemented, for example: Figure 2 Steps S201-S207 shown, or Figures 3 to 7 Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the emoticon package generation device based on artificial intelligence are realized, for example Figure 8 The functions of the original portrait image acquisition module 801, the original key point image acquisition module 802, the target key point image acquisition module 803, the target correction image acquisition module 804, the standard portrait image acquisition module 805, the target portrait image acquisition module 806 and the target expression package generation module 807 are not repeated here to avoid repetition.
[0138] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for generating an emoticon package based on artificial intelligence in the above embodiment are implemented, for example: Figure 2 Steps S201-S207 shown, or Figures 3 to 7 To avoid repetition, the steps shown in are not described here. Alternatively, when the processor executes the computer program, the functions of each module / unit in the embodiment of the emoticon package generation device based on artificial intelligence are realized, for example Figure 8 The functions of the original portrait image acquisition module 801, the original key point image acquisition module 802, the target key point image acquisition module 803, the target correction image acquisition module 804, the standard portrait image acquisition module 805, the target portrait image acquisition module 806 and the target expression package generation module 807 are not repeated here to avoid repetition.
[0139] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0140] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.
[0141] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.
Claims
1. A method for generating emoticons based on artificial intelligence, characterized in that: include: Get the original portrait image including the head portrait; Using a face detection model to detect the original portrait image to obtain an original key point map; Obtain a target expression package template, and perform screenshot processing on the original key point map based on the target expression package template to obtain a target key point map; Obtaining the point coordinates of the template key points of the portrait to be replaced in the target expression package template; Obtaining point coordinates of a target key point in the target key point map, and determining a face offset angle of the target key point map according to the coordinates of the template key point and the point coordinates of the target key point; According to the face offset angle, the target key point map is corrected to obtain a target correction map; Obtain the length and height of the portrait to be replaced in the target emoticon template; Based on the length and height of the portrait to be replaced, the target correction image is adapted by using an image processing model to obtain a standard portrait image; Inputting the standard portrait image into a pre-trained target shallow network, processing the standard portrait image based on the convolution layer and the downsampling layer in the target shallow network to obtain a thumbnail; Processing the thumbnail image by using dilated convolution and global pooling to obtain facial feature information, and obtaining a target portrait image from a standard portrait image according to the facial feature information; The target expression package template and the target portrait image are combined to generate a target expression package corresponding to the target expression package template.
2. The artificial intelligence-based expression package generation method according to claim 1, wherein: Obtaining a target expression package template, and taking a screenshot of the original key point map based on the target expression package template to obtain a target key point map, including: If the target expression package template is a human face expression package template, a target key point frame is generated according to the original key points of the original key point map, and based on the target key point frame, a screenshot tool is used to perform a screenshot process on the original key point map to obtain a target key point map; If the target expression package template is a head expression package template, a key point frame to be processed is generated according to the original key points of the original key point map, the key point frame is enlarged according to a preset ratio, and a target key point frame including the head is obtained. Based on the target key point frame, a screenshot tool is used to capture the original key point map for screenshot processing to obtain the target key point map.
3. The artificial intelligence-based expression package generation method according to claim 1, characterized in that: The obtaining of the original portrait image including the head portrait comprises: Obtaining an initial image containing the avatar, and determining whether a pixel value of the initial image is not less than a preset pixel threshold; If the pixel value of the initial image is not less than the preset pixel threshold, the initial image is determined as the original portrait image; If the pixel value of the initial image is less than the preset pixel threshold, a reminder message is generated and sent to the client, and the process of obtaining the initial image including the avatar is repeated.
4. The artificial intelligence-based expression package generation method according to claim 1, wherein: After obtaining the original portrait image including the avatar, the artificial intelligence-based expression package generation method further includes: The number of the original portrait images is obtained. If the number of the original portrait images is greater than a preset number threshold, a multi-thread processing mechanism is executed to call at least two threads to respectively execute the detection of the original portrait images using a face detection model to obtain an original key point map.
5. An artificial intelligence-based expression package generation device, characterized in that: include: The original portrait image acquisition module is used to acquire the original portrait image including the head portrait; An original key point map acquisition module is used to detect the original portrait image using a face detection model to obtain an original key point map; A target key point map acquisition module is used to acquire a target expression package template, and to perform screenshot processing on the original key point map based on the target expression package template to acquire a target key point map; A target correction image acquisition module is used to obtain the point coordinates of the template key points of the portrait to be replaced in the target expression package template; Obtaining point coordinates of a target key point in the target key point map, and determining a face offset angle of the target key point map according to the coordinates of the template key point and the point coordinates of the target key point; According to the face offset angle, the target key point map is corrected to obtain a target correction map; A standard portrait image acquisition module is used to obtain the length and height of the portrait to be replaced in the target expression package template; Based on the length and height of the portrait to be replaced, the target correction image is adapted by using an image processing model to obtain a standard portrait image; A target portrait image acquisition module is used to input the standard portrait image into a pre-trained target shallow network, process the standard portrait image based on the convolution layer and the downsampling layer in the target shallow network, and obtain a thumbnail; Processing the thumbnail image by using dilated convolution and global pooling to obtain facial feature information, and obtaining a target portrait image from a standard portrait image according to the facial feature information; The target expression package generation module is used to combine the target expression package template and the target portrait image to generate a target expression package corresponding to the target expression package template.
6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the emoticon package generation method based on artificial intelligence as described in any one of claims 1 to 4 are implemented.
7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the emoticon package generation method based on artificial intelligence as described in any one of claims 1 to 4 are implemented.
