Special effects display method, device, electronic device and storage medium

Abstract

The embodiments of the present disclosure provide a special effect display method, device, electronic device and storage medium, which receive a special effect trigger operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed containing hairstyle information in the image display area. Then, when it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area to display a hairstyle transformation image that is at least partially different from the hairstyle information in the image to be transformed, thereby realizing automatic replacement of the hairstyle in the image. The method can replace the hairstyle in the image in real time in response to the user's special effect trigger operation without the need for subsequent manual operation by the user, thereby improving the efficiency and real-time performance of replacing the hairstyle in the image, and solving the technical problems of complex and inefficient operation of replacing the hairstyle in the image in the prior art. In addition, the method is not limited to professional users, has universal applicability, is simple to operate, and improves the user experience.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to image processing technology, and more particularly to a special effects display method, device, electronic device, and storage medium. Background Art

[0002] Currently, users who want to change the hairstyle in an image can only do so later using image processing software. This requires manually importing the captured image into the software, matching the input image with various hairstyle templates, and manually adjusting the input image to ensure it aligns with the templates, thus enabling the image-based hairstyle change.

[0003] However, this manual post-processing method requires professional users to constantly interact with the processing software, which is inefficient and requires user professionalism. The operation steps are complicated and inconvenient to use. Moreover, once the image or video changes, it needs to be manually processed again, and the speed of hairstyle replacement is relatively slow, which affects the user experience. Summary of the Invention

[0004] The present disclosure provides a special effects display method, device, electronic device and storage medium to realize automatic replacement of hairstyles in images, improve the efficiency of hairstyle replacement in images, and solve the technical problems of complex operation and low efficiency of image hairstyle replacement in the prior art.

[0005] In a first aspect, an embodiment of the present disclosure provides a special effects display method, comprising:

[0006] receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area;

[0007] When it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area;

[0008] The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image.

[0009] In a second aspect, the embodiments of the present disclosure further provide a special effects display device, comprising:

[0010] A first display module is configured to receive a special effect triggering operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed including hairstyle information in the image display area;

[0011] A second display module is configured to display a hairstyle transformation image corresponding to the image to be transformed in the image display area when it is detected that a preset transformation effect display condition is met;

[0012] The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image.

[0013] In a third aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0014] one or more processors;

[0015] a storage device for storing one or more programs,

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects display method provided in any embodiment of the present disclosure.

[0017] In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to execute a special effects display method as provided in any embodiment of the present disclosure.

[0018] The disclosed embodiment receives a special effect trigger operation for a hairstyle transformation special effect, displays an image display area, and displays an image to be transformed containing hairstyle information in the image display area. Then, when it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area to display a hairstyle transformation image that is at least partially different from the hairstyle information in the image to be transformed, thereby realizing automatic replacement of the hairstyle in the image. This method can replace the hairstyle in the image in real time in response to the user's special effect trigger operation without the need for subsequent manual operation by the user, thereby improving the efficiency and real-time performance of replacing the hairstyle in the image, and solving the technical problems of complex and inefficient operation of replacing the hairstyle in the image in the prior art. In addition, the method is not limited to professional users, has universal applicability, is simple to operate, and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A flowchart of a special effects display method provided by an embodiment of the present disclosure;

[0021] Figure 2 A flowchart of a special effects display method provided by an embodiment of the present disclosure;

[0022] Figure 3A flowchart of a special effects display method provided by an embodiment of the present disclosure;

[0023] Figure 4 A flowchart of a special effects display method provided by an embodiment of the present disclosure;

[0024] Figure 5A A flowchart of a special effects display method provided by an embodiment of the present disclosure;

[0025] Figure 5B An image to be transformed provided by an embodiment of the present disclosure;

[0026] Figure 5C An image to be processed corresponding to an image to be transformed provided by an embodiment of the present disclosure;

[0027] Figure 5D A hair mask image corresponding to an image to be processed provided by an embodiment of the present disclosure;

[0028] Figure 5E A model transformation image corresponding to the image to be transformed provided by the embodiment of the present disclosure;

[0029] Figure 5F The result after image adjustment of the model transformation provided by the embodiment of the present disclosure;

[0030] Figure 5G A target restoration image corresponding to a model transformation image provided by an embodiment of the present disclosure;

[0031] Figure 5H A hairstyle transformation image corresponding to the image to be processed provided by an embodiment of the present disclosure;

[0032] Figure 6 A schematic structural diagram of a special effects display device provided by an embodiment of the present disclosure;

[0033] Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0035] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0036] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0037] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0038] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0039] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0040] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0041] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0042] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0043] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0044] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0045] Figure 1 This is a flow chart of a special effects display method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case where, in response to a user's special effects triggering operation, a hairstyle transformation image corresponding to an image to be transformed is generated and displayed to transform the hairstyle information in the image to be transformed into other hairstyle information. The method can be executed by a special effects display method device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0046] like Figure 1 As shown, the method includes:

[0047] S110: receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area.

[0048] The hairstyle transformation effect can be used to transform the hairstyle in an image and generate an image containing the transformed hairstyle. Exemplarily, the hairstyle transformation effect can include at least one of a hairstyle length transformation effect, a hairstyle curvature transformation effect, a hairstyle color transformation effect, and a hairstyle transparency transformation effect. The hairstyle information can include display information of at least a portion of pixels in a hair region or a hair region in the image to be transformed.

[0049] Specifically, a user can generate a special effect trigger operation for displaying a hairstyle transformation special effect by triggering a hairstyle transformation control on a visual interface of a terminal device. For example, when a user clicks a hairstyle transformation button on a visual interface, it is determined that a special effect trigger operation for a hairstyle transformation special effect has been received; or when a user uploads an image containing hairstyle information, it is determined that a special effect trigger operation for a hairstyle transformation special effect has been received; or when a user double-clicks a hairstyle portion of an image displayed on a visual interface, it is determined that a special effect trigger operation for a hairstyle transformation special effect has been received; or when a user clicks a preset hairstyle, it is determined that a special effect trigger operation for a hairstyle transformation special effect has been received, etc.

[0050] Specifically, after a special effect triggering operation is detected, an image display area can be displayed on the visual interface. The image display area can be a global area or a local area on the visual interface. Specifically, after a special effect triggering operation is detected, an image shooting control and an image selection control can be displayed on the visual interface. If it is detected that the user triggers the image shooting control, the camera module is called to start the camera to shoot the image to be transformed; if it is detected that the user triggers the image selection control, the historical images or videos in the local memory are called to facilitate the user to select the image to be transformed. Alternatively, after a special effect triggering operation is detected, the user's live display screen can be obtained and the live display screen can be used as the image to be transformed.

[0051] Of course, after obtaining the image to be transformed in the user's shooting, selection or live broadcast interface, it is also possible to detect whether there is hairstyle information in the image to be transformed. If not, a prompt message is displayed on the visual interface to allow the user to re-determine the image to be transformed until the hairstyle information exists in the image to be transformed.

[0052] Furthermore, an image to be transformed, including the hairstyle information, is displayed in the image display area. The image to be transformed may be an image requiring a hairstyle transformation, and the hairstyle information in the image to be transformed may be understood as the initial hairstyle information. Alternatively, the image to be transformed may be an image captured by the user, an image frame from a recorded video, or a live broadcast display.

[0053] S120: When it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area.

[0054] The hairstyle information in the image to be transformed and the hairstyle information in the hairstyle-transformed image are at least partially different. For example, pixel display information of the hair region or hair area in the image to be transformed and the pixel display information of the hair region or hair area in the hairstyle-transformed image are completely different; or, the pixel display information of the hair region or hair area in the image to be transformed and the pixel display information of the hair region or hair area in the hairstyle-transformed image may have some identical pixel display information and some different pixel display information.

[0055] The hairstyle information in the hairstyle transformation image can be understood as the target hairstyle information. In the disclosed embodiment, when a special effect triggering operation is detected, the target hairstyle to be transformed by the user is determined, and then, based on the target hairstyle and the image to be transformed, a hairstyle transformation image corresponding to the image to be transformed is generated.

[0056] Among them, determining the target hairstyle that the user needs to change to can be: determining the target hairstyle based on a special effect trigger operation, such as determining the preset hairstyle clicked or selected by the user as the target hairstyle; or determining the target hairstyle based on the user's historical hairstyle change record, such as determining the preset hairstyle that the user has changed most times as the target hairstyle; or determining the target hairstyle corresponding to the image to be changed based on the hairstyle information in the image to be changed and a preset hairstyle transformation mapping table, wherein the preset hairstyle mapping table includes each initial hairstyle and the transformed hairstyle corresponding to each initial hairstyle.

[0057] Exemplarily, generating a hairstyle transformation image corresponding to the image to be transformed based on the target hairstyle and the image to be transformed may be: generating a hairstyle transformation image corresponding to the image to be transformed based on a template image corresponding to the target hairstyle and the image to be transformed; or obtaining a pre-trained hairstyle transformation model corresponding to the target hairstyle, inputting the image to be transformed into the hairstyle transformation model, and obtaining a hairstyle transformation image corresponding to the image to be transformed.

[0058] Specifically, the embodiment of the present disclosure may display a hairstyle transformation image obtained by transforming the hairstyle information in the image to be transformed in the image display area when it is detected that a preset transformation effect display condition is met.

[0059] The preset transformation effect display conditions may include, for example, the generation of a hairstyle transformation image, the elapse of a preset time period after the display of the image to be transformed, or the user triggering a transformation effect display operation. Specifically, upon detecting that a hairstyle transformation image has been generated, upon detecting that a preset time period has elapsed after the display of the image to be transformed, or upon receiving a transformation effect display operation, the hairstyle transformation image corresponding to the image to be transformed may be displayed.

[0060] Optionally, the image to be transformed and the hairstyle transformation image are displayed in comparison in the image display area, or the image to be transformed displayed in the image display area is replaced with the hairstyle transformation image.

[0061] The technical solution of the embodiment of the present disclosure receives a special effect trigger operation for a hairstyle transformation special effect, displays an image display area, and displays an image to be transformed containing hairstyle information in the image display area. Then, when it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area to display a hairstyle transformation image that is at least partially different from the hairstyle information in the image to be transformed, thereby realizing automatic replacement of the hairstyle in the image. This method can replace the hairstyle in the image in real time in response to the user's special effect trigger operation without the need for subsequent manual operation by the user, thereby improving the efficiency and real-time performance of replacing the hairstyle in the image, and solving the technical problems of complex and inefficient operation of replacing the hairstyle in the image in the prior art. In addition, the method is not limited to professional users, has universal applicability, is simple to operate, and improves the user experience.

[0062] Figure 2 This is a flow chart of a special effects display method provided by an embodiment of the present disclosure. Based on the above embodiments, this embodiment of the present disclosure exemplifies the process of generating a hairstyle transformation image corresponding to the image to be processed. Figure 2 , the special effects display method includes:

[0063] S210: receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area.

[0064] S220: Determine an image to be processed and a hair mask image corresponding to the image to be processed according to the image to be transformed.

[0065] The image to be processed may be a pre-processed image of the image to be transformed. The hair mask image may be a hair mask image of the image to be processed. Specifically, in the disclosed embodiments, the image to be transformed may be pre-processed to obtain the image to be processed, and then the image to be processed may be processed to obtain a hair mask image corresponding to the image to be processed.

[0066] Exemplarily, determining an image to be processed and a hair mask image corresponding to the image to be processed based on the image to be transformed includes: performing image preprocessing on the image to be transformed to obtain an image to be processed corresponding to the image to be transformed, wherein the image preprocessing includes at least one of facial alignment processing, resizing processing, and sharpening processing; and processing the image to be processed based on a preset hair segmentation algorithm to obtain a hair mask image corresponding to the image to be processed.

[0067] Specifically, performing facial alignment on the image to be transformed may involve aligning the facial region in the image to be transformed with the facial region in a preset template image, where the facial region in the preset template image may be the center region of the preset template image. For example, the facial region in the image to be transformed may be aligned with the facial region in the preset template image by performing operations such as rotation, scaling, and translation on the image to be transformed.

[0068] Resizing the image to be transformed may involve scaling and / or cropping the image to a predetermined size, such as 512*512. Sharpening the image to be transformed may involve enhancing edges and compensating for high-frequency components in the image to make the image clearer.

[0069] It should be noted that the purpose of performing face alignment on the image to be transformed is to move the face of the image to the center, facilitating subsequent hairstyle transformation and improving the accuracy of the processed image. The purpose of performing resizing on the image to be transformed is to adjust the image to a preset size, facilitating subsequent hairstyle transformation. The purpose of performing sharpening on the image to be transformed is to enhance the edges of the image to improve the accuracy of the processed image.

[0070] Optionally, the image to be transformed may be first subjected to face alignment processing, and then subjected to size transformation processing. Further, the image to be transformed may be subjected to sharpening processing to obtain the image to be processed.

[0071] Furthermore, after obtaining the image to be processed, the image to be processed can be segmented using a preset hair segmentation algorithm to obtain hair regions in the image to be processed, and then a hair mask image corresponding to the image to be processed can be determined based on the segmented hair regions. Optionally, the preset hair segmentation algorithm can be an image edge segmentation algorithm, an image threshold segmentation algorithm, a region-based segmentation algorithm, or a semantic segmentation algorithm.

[0072] In this exemplary embodiment, by performing at least one of facial alignment, resizing, and sharpening processing on the image to be transformed, an image to be processed corresponding to the image to be transformed is obtained, and then a hair mask image corresponding to the image to be processed is determined according to a hair segmentation algorithm, thereby achieving accurate determination of the image to be processed and the hair mask image. By preprocessing the image to be processed, it is convenient to subsequently directly determine the hairstyle transformation image based on the image to be processed, thereby improving the clarity and generation efficiency of the hairstyle transformation image. In addition, by determining the hair mask image, it is convenient to subsequently generate the hairstyle transformation image based on the hair mask image, thereby further improving the efficiency of generating the hairstyle transformation image.

[0073] S230 , transforming the hairstyle information in the image to be transformed according to the image to be processed, the hair mask image, and the pre-trained hairstyle transformation model to obtain a model-transformed image corresponding to the image to be transformed.

[0074] The hairstyle transformation model can be a pre-trained deep learning model or neural network model. Exemplarily, the hairstyle transformation model can be a generative adversarial network model, a convolutional neural network model, a deep cross-learning model, an attention factorization machine model, or a deep reinforcement learning network. Specifically, the hairstyle transformation model is used to transform the hairstyle information in an input image and output an image with the transformed hairstyle information.

[0075] Optionally, a hairstyle transformation network can be pre-built and trained using a pre-built sample set. A loss function is calculated based on the predicted image output by the hairstyle transformation network and the labeled image in the sample set. The parameters in the hairstyle transformation network are then adjusted inversely based on the loss function calculation result until an iteration stopping condition is met, thereby obtaining a hairstyle transformation model. The hairstyle transformation model can be trained in an unsupervised or supervised manner, which is not limited in the present embodiment.

[0076] Specifically, the trained hairstyle transformation model can output a model-transformed image corresponding to the image to be transformed based on the input image to be processed and the hair mask image. The model-transformed image can be an image output by the hairstyle transformation model after transforming the hairstyle information in the image to be transformed.

[0077] Exemplarily, the hairstyle transformation model may include two inputs, a first input receiving an image to be processed and a second input receiving a hair mask image. After the image to be processed and the hair mask image pass through the image processing layer of the hairstyle transformation model, relevant features of the image to be processed and the hair mask image can be extracted. After passing through the image output layer, a model-transformed image corresponding to the image to be transformed can be output. Alternatively, the hairstyle transformation model may include a single input, wherein the image to be processed and the hair mask image are concatenated or fused, and the result of the concatenation or fusion is input into the hairstyle transformation model to obtain a model-transformed image output by the hairstyle transformation model. The concatenation of the image to be processed and the hair mask image may involve concatenating their channels. For example, if the image to be processed is a 3-channel RGB (Red, Green, Blue) image and the hair mask image is a 1-channel grayscale image, a 4-channel image can be obtained after concatenation.

[0078] S240: Determine a hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained region restoration model.

[0079] The region inpainting model is used to inpaint at least one region of interest in the model-transformed image. In the disclosed embodiment, the region of interest may be the area surrounding the hair region in the model-transformed image. Exemplarily, the region of interest includes at least one of a facial region, a clothing region, and an image background region.

[0080] Exemplarily, the regional repair model may include a background repair model. Specifically, considering that in the process of performing hairstyle transformation on the image to be transformed, such as changing curly hair to straight hair, background information obscured by the curly hair may be lost, therefore, in the embodiment of the present disclosure, a background repair model for repairing the background information in the image can be pre-trained to perform background repair on the model transformation image output by the hairstyle transformation model to obtain the hairstyle transformation model.

[0081] Alternatively, the regional repair model may include a clothing repair model. Specifically, considering that in the process of performing hairstyle transformation on the transformed image, such as changing long hair to short hair, the details of the clothing part obscured by the long hair may be lost, therefore, in the embodiment of the present disclosure, a clothing repair model for repairing the clothing details in the image can be pre-trained to perform clothing repair on the model transformation image output by the hairstyle transformation model to obtain a hairstyle transformation image.

[0082] In the disclosed embodiments, the clothing repair model can be a deep learning model or a neural network model. Exemplarily, the clothing repair model can be a generative adversarial network model, a convolutional neural network model, or a deep reinforcement learning network. Exemplarily, a clothing repair network can be pre-constructed and trained using a pre-constructed sample set. A loss function is calculated based on the predicted image output by the clothing repair network and the labeled images in the sample set. The parameters in the clothing repair network are then reversely adjusted based on the loss function calculation results until an iteration stopping condition is met, thereby obtaining a clothing repair model.

[0083] Specifically, a trained clothing restoration model can output a hairstyle transformation image corresponding to an input model transformation image based on the input model transformation image. For example, the clothing restoration model can detect clothing regions within the input model transformation image, detect blurred regions within the clothing regions, and perform patching on the blurred regions to obtain a hairstyle transformation image. The blurred regions within the clothing regions can be areas covered by the hairstyle information difference between the hairstyle information in the image to be processed and the hairstyle information in the model transformation image. For example, patching of the blurred regions can involve updating the value of each pixel in the blurred region based on the values ​​of other pixels in its neighboring region.

[0084] In the embodiment of the present disclosure, considering that before the model transformation image is output through the hairstyle transformation model, the image to be transformed is pre-processed by performing facial object, size transformation or sharpening on the image to be transformed, therefore, after obtaining the model transformation image, the model transformation image can also be post-processed before being input into the regional restoration model to ensure that the model transformation image is consistent with the facial position, image orientation or size and other information of the image to be transformed.

[0085] For example, before determining a hairstyle transformation image corresponding to the image to be processed based on the model transformation image and a pre-trained regional restoration model, the method further includes adjusting the model transformation image based on the image to be transformed so that the model transformation image is aligned with the face of the image to be transformed. Adjusting the model transformation image includes translating, rotating, or scaling the model transformation image.

[0086] Specifically, the orientation or size of the model transformation image can be adjusted to align the facial region in the model transformation image with the facial region in the image to be transformed. For example, multiple anchor points can be set based on the facial region in the image to be transformed, such as the eyebrows, eyes, and nose bridge. Reference points corresponding to each anchor point in the model transformation image are then determined, and the anchor points are spatially aligned with the reference points corresponding to the anchor points, thereby achieving facial alignment between the model transformation image and the image to be transformed.

[0087] In this way, after obtaining the model transformation image of the hairstyle transformation output by the hairstyle transformation model, the model transformation image can be adjusted according to the image to be transformed so that the model transformation image is aligned with the face of the image to be transformed, and then the hairstyle transformation image and the facial position, image direction, image size and other information of the image to be transformed are kept consistent, which is convenient for the subsequent display of the hairstyle transformation image. For example, if the hairstyle transformation image that is consistent with the facial position, image direction and image size of the image to be transformed is directly displayed at the display position of the image to be transformed, the user experience is improved.

[0088] S250: When it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area.

[0089] The technical solution of the disclosed embodiment determines the image to be processed and the hair mask image corresponding to the image to be processed through the image to be transformed, and then obtains the model transformation image based on the image to be processed, the hair mask image and the hairstyle transformation model. The hairstyle transformation image is obtained based on the model transformation image and the regional repair model, thereby realizing the hairstyle transformation and regional repair of the image to be processed. While improving the transformation efficiency of the hairstyle information in the image, it avoids the loss of details of a certain area in the image due to the hairstyle information in the transformed image, thereby further improving the user experience.

[0090] Figure 3 This is a flow chart of a special effects display method provided by an embodiment of the present disclosure. Based on the above embodiments, this embodiment of the present disclosure provides a supplementary explanation of the process of generating a model transformation image corresponding to the image to be transformed based on the hairstyle transformation model. Figure 3 , the special effects display method includes:

[0091] S310: receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area.

[0092] S320: Determine an image to be processed and a hair mask image corresponding to the image to be processed according to the image to be transformed.

[0093] S330. Perform image stitching on the image to be processed and the hair mask image to obtain a target input image. Input the target input image into a pre-trained hairstyle transformation model, and obtain an initial transformed image corresponding to the image to be transformed through the hairstyle transformation layer of the hairstyle transformation model.

[0094] Among them, the hairstyle transformation model includes a hairstyle transformation layer and an image output layer connected to the hairstyle transformation layer. The image output layer is constructed based on the attention mechanism.

[0095] Specifically, the image to be processed and the hair mask image are stitched together by stitching the channels of the image to be processed with the channels of the hair mask image. For example, if the image to be processed is a 3-channel RGB image and the hair mask image is a 1-channel grayscale image, the stitched target input image is a 4-channel image.

[0096] Furthermore, after the target input image is input to the hairstyle transformation model, the hairstyle transformation model processes the target input image according to the hairstyle transformation layer and outputs an initial transformed image. The initial transformed image may be an initial image that transforms the hairstyle information in the image to be processed. Specifically, the initial transformed image includes a red channel, a green channel, a blue channel, and a transparent channel, i.e., the initial transformed image is a four-channel image. The red, green, and blue channels in the initial transformed image may constitute the first image, and the transparent channel in the initial transformed image may constitute the second image.

[0097] S340 , processing the initial transformed image via the image output layer of the hairstyle transformation model to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

[0098] After obtaining the initial transformed image, the image output layer can further process the initial transformed image to obtain the model transformed image. For example, the image output layer can directly output the initial transformed image as the model transformed image. Alternatively, the image output layer can use an attention mechanism to determine a region of interest in the initial transformed image, enhance the region of interest, and output the model transformed image based on the enhanced processing results.

[0099] In an optional embodiment, the initial transformed image is processed through the image output layer of the hairstyle transformation model to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed. This can be: through the image output layer of the hairstyle transformation model, the target input image, a first image composed of the red channel, the green channel and the blue channel in the initial transformed image, and a second image composed of the transparent channel are image fused to obtain the model transformed image after the hairstyle information in the image to be transformed is transformed.

[0100] That is, the image output layer can fuse the target input image, the first image, and the second image, and output the fusion result as the model transformation image. Exemplarily, in the process of fusing the target input image, the first image, and the second image, the image output layer can fuse all the image information in the target input image, the first image, and the second image; or, based on the attention mechanism, the image output layer can select part of the image information in the target input image, the first image, and the second image as the region of interest, and fuse each region of interest, for example, selecting the image region other than the head or hairstyle in the target input image, and selecting the image region corresponding to the head or hairstyle in the first image and the second image.

[0101] In this optional embodiment, through the image output layer in the hairstyle transformation model, the first image and the second image in the initial transformation image output by the hairstyle transformation layer are fused together with the target input image to obtain a model transformation image, thereby realizing the generation of the model transformation image based on the attention mechanism, so that the subsequently determined hairstyle transformation image satisfies the attention mechanism, thereby improving the visual effect of the hairstyle transformation image and further improving the user experience.

[0102] S350: Determine a hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained regional restoration model. When it is detected that a preset transformation effect display condition is met, display the hairstyle transformation image corresponding to the image to be transformed in the image display area.

[0103] The technical solution of the embodiment of the present disclosure performs image stitching on the image to be processed and the hair mask image to obtain a target input image, inputs the target input image into the hairstyle transformation model, and obtains an initial transformation image consisting of a red channel, a green channel, a blue channel, and a transparent channel through the hairstyle transformation layer of the hairstyle transformation model. The initial transformation image is then processed through the image output layer of the hairstyle transformation model constructed based on the attention mechanism to obtain a model transformation image that complies with the attention mechanism, thereby ensuring that the subsequently determined hairstyle transformation image satisfies the attention mechanism, thereby improving the visual effect of the hairstyle transformation image and further improving the user experience.

[0104] Figure 4This is a flow chart of a special effect display method provided by an embodiment of the present disclosure. Based on the above embodiments, this embodiment of the present disclosure exemplarily illustrates the process of fusing the target input image, the first image, and the second image. Figure 4 , the special effects display method includes:

[0105] S410: receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area.

[0106] S420 , determining an image to be processed and a hair mask image corresponding to the image to be processed according to the image to be transformed, and performing image stitching on the image to be processed and the hair mask image to obtain a target input image.

[0107] S430: Input the target input image into a pre-trained hairstyle transformation model, and obtain an initial transformed image corresponding to the image to be transformed through the hairstyle transformation layer of the hairstyle transformation model.

[0108] S440. Obtain an original color image consisting of the red channel, green channel, and blue channel in the target input image through the image output layer of the hairstyle transformation model, and determine a grayscale image of the original color image.

[0109] The original color image composed of the red, green, and blue channels of the target input image can be the image to be processed. Specifically, the image output layer can determine the image to be processed, i.e., the original color image, based on the red, green, and blue channels of the target input image, and perform binarization processing on the original color image to obtain a grayscale image of the original color image.

[0110] S450 , performing filtering processing on the grayscale image and the second image consisting of the transparent channels based on a preset guided filtering algorithm to obtain a target transparent image.

[0111] The preset guided filtering algorithm may be used to perform edge-preserving smoothing on the second image, that is, extract edge details of the second image, retain details of the edge region of the second image, and smooth regions inside and outside the edge.

[0112] Specifically, the grayscale image and the second image composed of the transparent channel are filtered by a preset guided filtering algorithm to obtain the target transparent image. It can be: by using the preset guided filtering algorithm, the grayscale image is used as a guiding image, and the second image is filtered to obtain the target transparent image.

[0113] The target transparent image obtained in this way is generally similar to the second image, but the texture part is similar to the grayscale image, so that the edge properties of the grayscale image are maintained in the target transparent image. As a result, the model transformation image obtained based on the target transparent image retains more edge details, further improving the image accuracy of the hairstyle transformation image.

[0114] S460 , performing image fusion on the original color image, the first image consisting of the red channel, the green channel, and the blue channel in the initial transformed image, and the target transparent image to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

[0115] Specifically, the embodiment of the present disclosure fuses the original color image, the first image, and the target transparent image, and uses the fused image as the template transformation image. Exemplarily, regions of interest can be selected from the original color image, the first image, and the target transparent image based on the attention mechanism, and the selected regions of interest are fused to obtain a model transformation image; wherein the regions of interest can be non-overlapping regions or overlapping regions. For example, the background region can be selected from the original color image as the region of interest, and the region other than the background region can be selected from the first image as the region of interest. Of course, the original color image, the first image, and the target transparent image can also be directly fused.

[0116] In an optional embodiment, the original color image, the first image composed of the red channel, the green channel, and the blue channel of the initial transformed image, and the target transparent image are subjected to image fusion to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed. The method may be: multiplying the first image composed of the red channel, the green channel, and the blue channel of the initial transformed image and the target transparent image to obtain a foreground image of the model transformed image; determining a background image of the model transformed image based on the original color image and the target transparent image; and adding the foreground image and the background image to obtain the model transformed image after the hairstyle information in the image to be transformed is performed.

[0117] Specifically, the target transparent image can be used as the foreground mask image corresponding to the first image. The first image and the target transparent image are multiplied to obtain a foreground image containing the person. A background mask image corresponding to the original color image can be determined based on the target transparent image. The original color image and the background mask image are multiplied to obtain a background image containing information other than the person. Furthermore, the foreground image and the background image are fused to obtain a model transformation image.

[0118] In this optional embodiment, a foreground image is obtained by using the first image and the target transparent image, and a background image is obtained by using the original color image and the target transparent image. Then, a model transformation image is constructed based on the foreground image and the background image, thereby realizing the determination of the model transformation image based on the attention mechanism, improving the accuracy of the model transformation image, and thereby improving the accuracy of the hairstyle transformation image.

[0119] Exemplarily, determining the background image of the model transformation image based on the original color image and the target transparent image includes: determining a third image based on a preset transparency value and a transparency value of the target transparent image; and performing a product operation on the original color image and the third image to obtain the background image of the model transformation image.

[0120] The preset transparency value may be a pre-set transparency threshold. For example, if the transparency value of the target transparent image is a normalized value, the preset transparency value may be 1; if the transparency value of the target transparent image is not normalized, the preset transparency value may be 255. Specifically, determining the third image based on the preset transparency value and the transparency value of the target transparent image may include: constructing a reference image based on the preset transparency value, wherein the transparency value of each pixel in the reference image is the preset transparency value; subtracting the reference image from the target transparent image; and determining the third image based on the subtraction result.

[0121] Furthermore, after obtaining the third image, ie, the background mask image corresponding to the original color image, the original color image and the third image may be multiplied, and the multiplication result may be used as the background image.

[0122] In this exemplary embodiment, the third image is determined by pre-setting the transparency value and the transparency value of the target transparent image, thereby realizing the determination of the background mask image corresponding to the original color image, and then obtaining the background image by combining the original color image and the third image, realizing background extraction based on the attention mechanism, thereby improving the accuracy of the model transformation image and improving the user experience.

[0123] S470: Determine a hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained regional restoration model. When it is detected that a preset transformation effect display condition is met, display the hairstyle transformation image corresponding to the image to be transformed in the image display area.

[0124] The technical solution of the embodiment of the present disclosure obtains the original color image composed of the red channel, green channel and blue channel in the target input image, determines the grayscale image of the original color image, and then filters the grayscale image and the second image according to a preset guided filtering algorithm to obtain a target transparent image. The original color image, the first image and the target transparent image are fused to obtain a model transformation image, thereby realizing the determination of the model transformation image based on the attention mechanism, so that the hairstyle transformation image satisfies the attention mechanism and improves the visual effect of the hairstyle transformation image. At the same time, the guided filtering can also enable the model transformation image to retain more edge details, further improving the accuracy of the hairstyle transformation image.

[0125] Figure 5A This is a flow chart of a special effects display method provided by an embodiment of the present disclosure. Based on the above embodiments, this embodiment of the present disclosure provides an exemplary description of the process of determining a hairstyle transformation image corresponding to an image to be processed based on a pre-trained region restoration model. Figure 5A , the special effects display method includes:

[0126] S510: receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area.

[0127] S520: Determine an image to be processed and a hair mask image corresponding to the image to be processed according to the image to be transformed.

[0128] S530 , transforming the hairstyle information in the image to be transformed according to the image to be processed, the hair mask image, and the pre-trained hairstyle transformation model to obtain a model-transformed image corresponding to the image to be transformed.

[0129] S540: Input the model transformation image into a pre-trained clothing restoration model to restore the clothing information in the model transformation image to obtain a target restoration image, and determine a hairstyle transformation image corresponding to the image to be processed based on the target restoration image.

[0130] Specifically, the regional restoration model may include a clothing restoration model. In the disclosed embodiment, a model transformation image may be input into a pre-trained clothing restoration model, and the clothing restoration model outputs a target restoration image that restores the clothing in the model transformation image. Furthermore, the target restoration image may be directly determined as a hairstyle transformation image, or the target restoration image may be processed and the result of the processing used as the hairstyle transformation image. For example, the target restoration image may be beautified or brightness enhanced.

[0131] Exemplarily, determining a hairstyle transformation image corresponding to the image to be processed based on the target inpainted image includes: adjusting the target inpainted image by at least one of the following methods to obtain a hairstyle transformation image corresponding to the image to be processed:

[0132] Performing cropping processing on the target restoration image so that the display position of the facial area in the target restoration image meets a preset position condition;

[0133] Determine the area to be processed in the target restoration image and perform sharpening on the area to be processed;

[0134] Blur the black edge pixels in the target repaired image.

[0135] Among them, the cropping processing of the target repaired image can be: locating the facial area in the target repaired image, and intercepting the facial area in the target repaired image, and scaling and adjusting the intercepted facial area so that the display position of the facial area meets the preset position condition, wherein the preset position condition can be located in the middle area of ​​the image; or, dividing the image into nine grids and locating them in the set grid area.

[0136] Sharpening the target inpainted image may include enhancing the edges of the target inpainted image. Blurring the black-edge pixels in the target inpainted image may include blurring the border of the target inpainted image; that is, determining a border region in the target inpainted image, determining pixels within the border region as black-edge pixels, and blurring the black-edge pixels.

[0137] Optionally, the target repair image is firstly cropped, and further, the cropped target repair image is sharpened, and finally, the black edge pixels in the sharpened target repair image are blurred.

[0138] In this embodiment, by performing at least one of cropping, sharpening, and blurring processing on the target repaired image output by the clothing repair model, adjustments to the hairstyle transformation and the image after clothing repair are achieved, so that the visual effect of the final output hairstyle transformation image is stronger, thereby improving the user experience.

[0139] S550: When it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area.

[0140] The technical solution of the embodiment of the present disclosure is that after obtaining the model transformation image output by the hairstyle transformation model, the model transformation image is input into a pre-trained clothing restoration model to obtain a target restoration image output by the clothing restoration model, thereby realizing the hairstyle transformation in the image and the clothing restoration after the hairstyle transformation. In addition, the hairstyle transformation image is determined based on the target restoration image, and the image after the clothing restoration can be further processed to obtain the final output hairstyle transformation image, such as sharpening processing, cropping processing, etc., thereby further improving the visual effect of the hairstyle transformation image.

[0141] For example, the present disclosure also provides a method for displaying special effects, which includes the following steps:

[0142] Step 1: Receive a special effect triggering operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed containing hairstyle information in the image display area. Figure 5B , Figure 5B An image to be transformed is provided in an embodiment of the present disclosure.

[0143] Step 2: Perform image preprocessing on the image to be transformed to obtain an image to be processed corresponding to the image to be transformed, wherein the image preprocessing includes at least one of face alignment processing, size conversion processing, and sharpening processing; and process the image to be processed based on a preset hair segmentation algorithm to obtain a hair mask image corresponding to the image to be processed. Figure 5C-5D , Figure 5C An image to be processed corresponding to an image to be transformed provided by an embodiment of the present disclosure, Figure 5D A hair mask image corresponding to an image to be processed is provided in an embodiment of the present disclosure.

[0144] Step 3: Based on the image to be processed, the hair mask image and the pre-trained hairstyle transformation model, the hairstyle information in the image to be transformed is transformed to obtain a model transformation image corresponding to the image to be transformed. Figure 5E , Figure 5E A model transformation image corresponding to an image to be transformed is provided in an embodiment of the present disclosure.

[0145] Step 4: Adjust the model transformation image according to the image to be transformed so that the face of the model transformation image is aligned with the face of the image to be transformed. Figure 5F , Figure 5F This is the result of image adjustment after the model transformation provided by the embodiment of the present disclosure.

[0146] Step 5: Input the model-transformed image into the pre-trained clothing restoration model to restore the clothing information in the model-transformed image and obtain the target restoration image. Figure 5G , Figure 5GA target restoration image corresponding to a model transformation image provided by an embodiment of the present disclosure.

[0147] Step 6: Crop the target inpainted image so that the display position of the facial area in the target inpainted image meets the preset position condition; determine the area to be processed in the target inpainted image and perform sharpening on the area to be processed; blur the black edge pixels in the target inpainted image to obtain a hairstyle transformation image corresponding to the image to be processed. Figure 5H , Figure 5H This is a hairstyle transformation image corresponding to the image to be processed, provided by an embodiment of the present disclosure.

[0148] Through the above steps, the hairstyle in the image is transformed, and the clothing after the hairstyle transformation is restored, thereby improving the visual effect of the image; and after the restoration, the facial information in the image is highlighted through cropping, sharpening and blurring, further improving the visual effect of the hairstyle transformation image.

[0149] Figure 6 This is a structural diagram of a special effect display device provided by an embodiment of the present disclosure, such as Figure 6 As shown, the device includes: a first display module 610 and a second display module 620.

[0150] The first display module 610 is configured to receive a special effect triggering operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed including hairstyle information in the image display area;

[0151] The second display module 620 is configured to display a hairstyle transformation image corresponding to the image to be transformed in the image display area when it is detected that a preset transformation effect display condition is met;

[0152] The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image.

[0153] The technical solution provided by the embodiment of the present disclosure receives a special effect trigger operation for a hairstyle transformation special effect through a first display module, displays an image display area, and displays an image to be transformed containing hairstyle information in the image display area. Then, through a second display module, when it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area, so as to display a hairstyle transformation image that is at least partially different from the hairstyle information in the image to be transformed, thereby realizing automatic replacement of the hairstyle in the image. This method can replace the hairstyle in the image in real time in response to the user's special effect trigger operation without the need for subsequent manual operation by the user, thereby improving the efficiency and real-time performance of replacing the hairstyle in the image, and solving the technical problems of complex and inefficient operation of replacing the hairstyle in the image in the prior art. In addition, the method is not limited to professional users, has universal applicability, is simple to operate, and improves the user experience.

[0154] Optionally, the special effects display device includes a transformation image generation module, which is used to determine the image to be processed and the hair mask image corresponding to the image to be processed based on the image to be transformed; transform the hairstyle information in the image to be transformed according to the image to be processed, the hair mask image and a pre-trained hairstyle transformation model to obtain a model transformation image corresponding to the image to be transformed; determine the hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained regional repair model.

[0155] Optionally, the transformed image generation module includes a preprocessing unit, which is used to perform image preprocessing on the image to be transformed to obtain an image to be processed corresponding to the image to be transformed, wherein the image preprocessing includes at least one of facial alignment processing, size transformation processing and sharpening processing; and the image to be processed is processed based on a preset hair segmentation algorithm to obtain a hair mask image corresponding to the image to be processed.

[0156] Optionally, the pre-processing unit is further configured to adjust the model transformation image according to the image to be transformed, so that the model transformation image is aligned with the face of the image to be transformed.

[0157] Optionally, the hairstyle transformation model includes a hairstyle transformation layer and an image output layer connected to the hairstyle transformation layer, and the image output layer is constructed based on an attention mechanism; the transformation image generation module includes a first model output unit, and the first model output unit is used to perform image splicing on the image to be processed and the hair mask image to obtain a target input image; the target input image is input into a pre-trained hairstyle transformation model, and an initial transformation image corresponding to the image to be transformed is obtained through the hairstyle transformation layer of the hairstyle transformation model, wherein the initial transformation image includes a red channel, a green channel, a blue channel and a transparent channel; the initial transformation image is processed through the image output layer of the hairstyle transformation model to obtain a model transformation image after the hairstyle information in the image to be transformed is transformed.

[0158] Optionally, the first model output unit is further used to perform image fusion on the target input image, the first image composed of the red channel, green channel and blue channel in the initial transformed image, and the second image composed of the transparent channel through the image output layer of the hairstyle transformation model to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

[0159] Optionally, the first model output unit is also used to obtain an original color image composed of the red channel, green channel and blue channel in the target input image, and determine the grayscale image of the original color image; filter the grayscale image and the second image composed of the transparent channel based on a preset guided filtering algorithm to obtain a target transparent image; and perform image fusion on the original color image, the first image composed of the red channel, green channel and blue channel in the initial transformed image, and the target transparent image.

[0160] Optionally, the first model output unit is further used to multiply a first image composed of a red channel, a green channel, and a blue channel in the initial transformed image and the target transparent image to obtain a foreground image of the model transformed image; determine a background image of the model transformed image based on the original color image and the target transparent image; and add the foreground image and the background image to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

[0161] Optionally, the first model output unit is further used to determine a third image based on a preset transparency value and the transparency value of the target transparent image; and perform a product operation on the original color image and the third image to obtain a background image of the model transformation image.

[0162] Optionally, the regional repair model includes a clothing repair model; the transformation image generation module includes a second model output unit, which is used to input the model transformation image into a pre-trained clothing repair model to repair the clothing information in the model transformation image to obtain a target repair image; and determine the hairstyle transformation image corresponding to the image to be processed based on the target repair image.

[0163] The second model output unit is further used to adjust the target repair image in at least one of the following ways to obtain a hairstyle transformation image corresponding to the image to be processed: cropping the target repair image so that the display position of the facial area in the target repair image meets a preset position condition; determining the area to be processed in the target repair image and sharpening the area to be processed; and blurring the black-edge pixels in the target repair image.

[0164] The special effects display device provided in the embodiments of the present disclosure can execute the special effects display method provided in any embodiment of the present disclosure, and has functional modules and beneficial effects corresponding to the execution method.

[0165] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0166] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 7 , which shows an electronic device (eg Figure 7 The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0167] like Figure 7As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 are also stored in the RAM 703. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An edit / output (I / O) interface 705 is also connected to the bus 704.

[0168] Typically, the following devices may be connected to the I / O interface 505: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0169] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0170] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0171] The electronic device provided by the embodiment of the present disclosure and the special effects display method provided by the above embodiment belong to the same inventive concept. Technical details not fully described in the embodiment of the present disclosure can be referred to the above embodiment, and the embodiment of the present disclosure has the same beneficial effects as the above embodiment.

[0172] An embodiment of the present disclosure provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the special effects display method provided in the above embodiment.

[0173] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0174] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0175] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0176] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0177] The computer-readable medium carries one or more programs. When executed by the electronic device, the one or more programs cause the electronic device to: receive a special effect trigger operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed containing hairstyle information within the image display area; and, upon detecting that a preset transformation effect display condition has been met, display a hairstyle transformation image corresponding to the image to be transformed within the image display area; wherein the hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image.

[0178] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0179] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0181] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0182] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0183] According to one or more embodiments of the present disclosure, [Example 1] provides a special effects display method, including:

[0184] receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area;

[0185] When it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area;

[0186] The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image.

[0187] According to one or more embodiments of the present disclosure, [Example 2] provides a special effects display method, further comprising:

[0188] Optionally, before displaying the hairstyle transformation image corresponding to the image to be transformed in the image display area, the method further includes:

[0189] Determining an image to be processed and a hair mask image corresponding to the image to be processed according to the image to be transformed;

[0190] transforming the hairstyle information in the image to be transformed according to the image to be processed, the hair mask image, and a pre-trained hairstyle transformation model to obtain a model-transformed image corresponding to the image to be transformed;

[0191] According to the model transformation image and the pre-trained region restoration model, a hairstyle transformation image corresponding to the image to be processed is determined.

[0192] According to one or more embodiments of the present disclosure, [Example 3] provides a special effects display method, further comprising:

[0193] Optionally, determining the image to be processed and the hair mask image corresponding to the image to be processed according to the image to be transformed includes:

[0194] Performing image preprocessing on the image to be transformed to obtain an image to be processed corresponding to the image to be transformed, wherein the image preprocessing includes at least one of face alignment processing, size conversion processing, and sharpening processing;

[0195] The image to be processed is processed based on a preset hair segmentation algorithm to obtain a hair mask image corresponding to the image to be processed.

[0196] According to one or more embodiments of the present disclosure, [Example 4] provides a special effects display method, further comprising:

[0197] Optionally, before determining the hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained region restoration model, the method further includes:

[0198] The model transformation image is adjusted according to the image to be transformed so that the model transformation image is aligned with the face of the image to be transformed.

[0199] According to one or more embodiments of the present disclosure, [Example 5] provides a special effects display method, further comprising:

[0200] Optionally, the hairstyle transformation model includes a hairstyle transformation layer and an image output layer connected to the hairstyle transformation layer, and the image output layer is constructed based on an attention mechanism;

[0201] The step of transforming the hairstyle information in the image to be transformed based on the image to be processed, the hair mask image, and a pre-trained hairstyle transformation model to obtain a model-transformed image corresponding to the image to be transformed includes:

[0202] Performing image stitching on the image to be processed and the hair mask image to obtain a target input image;

[0203] Inputting the target input image into a pre-trained hairstyle transformation model, and obtaining an initial transformed image corresponding to the image to be transformed through the hairstyle transformation layer of the hairstyle transformation model, wherein the initial transformed image includes a red channel, a green channel, a blue channel, and a transparent channel;

[0204] The initial transformed image is processed via the image output layer of the hairstyle transformation model to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

[0205] According to one or more embodiments of the present disclosure, [Example 6] provides a special effects display method, further comprising:

[0206] Optionally, obtaining, via the image output layer of the hairstyle transformation model, a model-transformed image after the hairstyle information in the image to be transformed is transformed, includes:

[0207] The target input image, a first image consisting of the red channel, the green channel, and the blue channel in the initial transformed image, and a second image consisting of the transparent channel are fused through the image output layer of the hairstyle transformation model to obtain a model-transformed image after the hairstyle information in the image to be transformed is transformed.

[0208] According to one or more embodiments of the present disclosure, [Example 7] provides a special effects display method, further comprising:

[0209] Optionally, performing image fusion on the target input image, a first image consisting of the initial transformed image including a red channel, a green channel, and a blue channel, and a second image consisting of a transparent channel includes:

[0210] Acquire an original color image consisting of a red channel, a green channel, and a blue channel in the target input image, and determine a grayscale image of the original color image;

[0211] Performing filtering processing on the grayscale image and the second image consisting of the transparent channel based on a preset guided filtering algorithm to obtain a target transparent image;

[0212] The original color image, the first image composed of the red channel, the green channel and the blue channel in the initial transformed image, and the target transparent image are subjected to image fusion.

[0213] According to one or more embodiments of the present disclosure, [Example 8] provides a special effects display method, further comprising:

[0214] Optionally, the fusing the original color image, the first image consisting of the red channel, the green channel, and the blue channel in the initial transformed image, and the target transparent image to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed includes:

[0215] multiplying a first image composed of a red channel, a green channel, and a blue channel in the initial transformed image and the target transparent image to obtain a foreground image of the model transformed image;

[0216] Determine a background image of a model transformation image according to the original color image and the target transparent image;

[0217] The foreground image and the background image are added together to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

[0218] According to one or more embodiments of the present disclosure, [Example 9] provides a special effects display method, further comprising:

[0219] Optionally, determining the background image of the model transformation image according to the original color image and the target transparent image includes:

[0220] determining a third image according to a preset transparency value and a transparency value of the target transparent image;

[0221] The original color image is multiplied by the third image to obtain a background image of the model transformation image.

[0222] According to one or more embodiments of the present disclosure, [Example 10] provides a special effects display method, further comprising:

[0223] Optionally, the region restoration model includes a clothing restoration model; and determining a hairstyle transformation image corresponding to the image to be processed based on the model transformation image and a pre-trained region restoration model includes:

[0224] Inputting the model-transformed image into a pre-trained clothing restoration model to restore the clothing information in the model-transformed image to obtain a target restoration image;

[0225] A hairstyle transformation image corresponding to the image to be processed is determined according to the target restoration image.

[0226] According to one or more embodiments of the present disclosure, [Example 11] provides a special effects display method, further comprising:

[0227] Optionally, determining the hairstyle transformation image corresponding to the image to be processed according to the target restoration image includes:

[0228] The target restoration image is adjusted by at least one of the following methods to obtain a hairstyle transformation image corresponding to the image to be processed:

[0229] Performing cropping processing on the target inpainted image so that a display position of a facial region in the target inpainted image satisfies a preset position condition;

[0230] Determining a region to be processed in the target inpainted image, and performing sharpening processing on the region to be processed;

[0231] Blurring is performed on the black edge pixels in the target repaired image.

[0232] According to one or more embodiments of the present disclosure, [Example 12] provides a special effects display device, including:

[0233] A first display module is configured to receive a special effect triggering operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed including hairstyle information in the image display area;

[0234] A second display module is configured to display a hairstyle transformation image corresponding to the image to be transformed in the image display area when it is detected that a preset transformation effect display condition is met;

[0235] The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image.

[0236] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0237] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0238] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A special effects display method, characterized in that: include: receiving a special effect triggering operation for a hairstyle transformation special effect, displaying an image display area, and displaying an image to be transformed including hairstyle information in the image display area; When it is detected that a preset transformation effect display condition is met, a hairstyle transformation image corresponding to the image to be transformed is displayed in the image display area; The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image; Before displaying the hairstyle transformation image corresponding to the image to be transformed in the image display area, the method further includes: Determining an image to be processed and a hair mask image corresponding to the image to be processed according to the image to be transformed; transforming the hairstyle information in the image to be transformed based on the image to be processed, the hair mask image, and a pre-trained hairstyle transformation model to obtain a model-transformed image corresponding to the image to be transformed, wherein the hairstyle information includes at least hair shape information, and the hairstyle transformation model includes a hairstyle transformation layer and an image output layer connected to the hairstyle transformation layer, wherein the image output layer is constructed based on an attention mechanism; According to the model transformation image and the pre-trained region restoration model, a hairstyle transformation image corresponding to the image to be processed is determined.

2. The special effects display method according to claim 1, characterized in that: The step of determining the image to be processed and the hair mask image corresponding to the image to be processed according to the image to be transformed includes: Performing image preprocessing on the image to be transformed to obtain an image to be processed corresponding to the image to be transformed, wherein the image preprocessing includes at least one of face alignment processing, size conversion processing, and sharpening processing; The image to be processed is processed based on a preset hair segmentation algorithm to obtain a hair mask image corresponding to the image to be processed.

3. The special effects display method according to claim 2, characterized in that: Before determining the hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained region restoration model, the method further includes: The model transformation image is adjusted according to the image to be transformed so that the model transformation image is aligned with the face of the image to be transformed.

4. The special effects display method according to claim 1, characterized in that: The step of transforming the hairstyle information in the image to be transformed based on the image to be processed, the hair mask image, and a pre-trained hairstyle transformation model to obtain a model-transformed image corresponding to the image to be transformed includes: Performing image stitching on the image to be processed and the hair mask image to obtain a target input image; Inputting the target input image into a pre-trained hairstyle transformation model, and obtaining an initial transformed image corresponding to the image to be transformed through the hairstyle transformation layer of the hairstyle transformation model, wherein the initial transformed image includes a red channel, a green channel, a blue channel, and a transparent channel; The initial transformed image is processed via the image output layer of the hairstyle transformation model to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

5. The special effect display method according to claim 4, characterized in that: The step of obtaining a model-transformed image after the hairstyle information in the image to be transformed is transformed via the image output layer of the hairstyle transformation model, comprising: The target input image, a first image consisting of the red channel, the green channel, and the blue channel in the initial transformed image, and a second image consisting of the transparent channel are fused through the image output layer of the hairstyle transformation model to obtain a model-transformed image after the hairstyle information in the image to be transformed is transformed.

6. The special effect display method according to claim 5, characterized in that: The image fusion of the target input image, a first image consisting of the initial transformed image including a red channel, a green channel, and a blue channel, and a second image consisting of a transparent channel comprises: Acquire an original color image consisting of a red channel, a green channel, and a blue channel in the target input image, and determine a grayscale image of the original color image; Performing filtering processing on the grayscale image and the second image consisting of the transparent channel based on a preset guided filtering algorithm to obtain a target transparent image; The original color image, the first image composed of the red channel, the green channel and the blue channel in the initial transformed image, and the target transparent image are subjected to image fusion.

7. The special effect display method according to claim 6, characterized in that: The step of fusing the original color image, the first image composed of the red channel, the green channel, and the blue channel in the initial transformed image, and the target transparent image to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed includes: multiplying a first image composed of a red channel, a green channel, and a blue channel in the initial transformed image and the target transparent image to obtain a foreground image of the model transformed image; Determine a background image of a model transformation image according to the original color image and the target transparent image; The foreground image and the background image are added together to obtain a model transformed image after the hairstyle information in the image to be transformed is transformed.

8. The special effect display method according to claim 7, characterized in that: The step of determining the background image of the model transformation image according to the original color image and the target transparent image includes: determining a third image according to a preset transparency value and a transparency value of the target transparent image; The original color image is multiplied by the third image to obtain a background image of the model transformation image.

9. The special effect display method according to claim 1, characterized in that: The regional restoration model includes a clothing restoration model; determining a hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained regional restoration model includes: Inputting the model-transformed image into a pre-trained clothing restoration model to restore the clothing information in the model-transformed image to obtain a target restoration image; A hairstyle transformation image corresponding to the image to be processed is determined according to the target restoration image.

10. The special effect display method according to claim 9, characterized in that: The step of determining a hairstyle transformation image corresponding to the image to be processed according to the target restoration image includes: The target restoration image is adjusted by at least one of the following methods to obtain a hairstyle transformation image corresponding to the image to be processed: Performing cropping processing on the target inpainted image so that a display position of a facial region in the target inpainted image satisfies a preset position condition; Determining a region to be processed in the target restoration image, and performing sharpening processing on the region to be processed; Blurring is performed on the black edge pixels in the target repaired image.

11. A special effects display device, characterized in that: include: A first display module is configured to receive a special effect triggering operation for a hairstyle transformation special effect, display an image display area, and display an image to be transformed including hairstyle information in the image display area; A second display module is configured to display a hairstyle transformation image corresponding to the image to be transformed in the image display area when it is detected that a preset transformation effect display condition is met; The hairstyle information in the image to be transformed is at least partially different from the hairstyle information in the hairstyle transformation image; The special effects display device includes a transformation image generation module, which is used to determine an image to be processed and a hair mask image corresponding to the image to be processed based on the image to be transformed; transform the hairstyle information in the image to be transformed based on the image to be processed, the hair mask image and a pre-trained hairstyle transformation model to obtain a model transformation image corresponding to the image to be transformed, wherein the hairstyle information at least includes hair shape information, the hairstyle transformation model includes a hairstyle transformation layer and an image output layer connected to the hairstyle transformation layer, and the image output layer is constructed based on an attention mechanism; determine the hairstyle transformation image corresponding to the image to be processed based on the model transformation image and the pre-trained regional repair model.

12. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the special effects display method as described in any one of claims 1-10.

13. A storage medium containing computer-executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the special effects display method as described in any one of claims 1 to 10.

Images