Interactive method, system and related devices for image beautification platform

Abstract

The application provides an image beautification platform interaction method, system and related equipment, the interaction method of the application constructs a perfect user system, after the user triggers a one-key beautification instruction, the platform needs to judge whether the user account has activated a subscription or holds sufficient virtual points; when the user account has activated the subscription or holds sufficient virtual points, the beautification parameter set is called to perform beautification processing on the original image; when the video template is matched, the visual special effect, dynamic effect transition and background music associated with the video template can be automatically matched based on the content of the beautified image; further, the application also provides a one-key sharing interface, the user can share the beautified image and / or dynamic short video to a social platform; after the beautified image and / or dynamic short video is successfully shared, the user account can also get a preset amount of virtual points. The image beautification interaction system of the application is more perfect, has higher playability and can better meet the user demand.

Description

Technical Field

[0001] This invention relates to the field of image interaction technology, and in particular to an interactive method, system and related equipment for an image enhancement platform. Background Technology

[0002] With the rapid development of mobile internet and social media, users' demand for personalized image enhancement and engaging content generation is growing. While there are numerous image processing and enhancement applications on the market, most suffer from limited functionality, fragmented modules, and inconsistent user experiences. Image-to-video conversion lacks intelligent adaptation algorithms based on image content, resulting in monotonous outputs and poor interactivity and playability. Furthermore, user management lacks a robust user system and a closed-loop social sharing mechanism, leading to insufficient user engagement. There is an urgent need for an interactive image enhancement method that integrates image enhancement, video generation, user systems, and social sharing.

[0003] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0004] This invention provides an interactive method, system, and related equipment for an image enhancement platform. The main objective of this invention is to solve the technical problems mentioned in the background section of the prior art.

[0005] The first aspect of this invention provides an interactive method for an image enhancement platform, comprising: Receive user login requests and establish user account sessions; Receive the original image uploaded by the user and generate an enhancement interface; In response to a user-triggered one-click beautification command, determine whether the user's account is in active subscription status or has sufficient virtual points; When the user account has activated a subscription or holds sufficient virtual points, the preset beautification parameter set is automatically invoked, and the original image is beautified according to the beautification parameter set to generate a beautified image; In response to a user-triggered video generation command, a preset video template selection interface is generated; Receive the video template selected by the user and automatically match visual effects, motion transitions and background music associated with the video template based on the content of the beautified image; An automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithm is used to merge the beautified image with the visual effects, the motion transitions and the background music to generate a dynamic short video; The system receives instructions from users to share the enhanced image and / or the dynamic short video to a social media platform via a pre-defined one-click sharing interface. Upon successful sharing of the beautified image and / or the dynamic short video, a preset amount of virtual points is issued to the user's account.

[0006] In an optional embodiment of the first aspect of the present invention, when the user account has activated a subscription or holds sufficient virtual points, automatically calling a preset beautification parameter set, and beautifying the original image according to the beautification parameter set to generate a beautified image includes: When it is detected that the user account has activated a valid subscription status or holds sufficient virtual points, the facial proportions, skin type and chest / hip contour features in the original image are identified based on the convolutional neural network model, and the optimal beautification parameter set is matched from the preset parameter library. The beautification parameter set is input into the image enhancement model of the generative adversarial network, and the image enhancement model performs multi-level beautification processing on the original image. The beautification parameter set includes skin color HSV adjustment threshold, skin smoothing intensity factor, breast / buttock deformation displacement vector and pore preservation coefficient. The skin and non-skin regions of the processed original image are fused pixel-level to output the beautified image. In an optional embodiment of the first aspect of the present invention, the step of inputting the beautification parameter set into a generative adversarial network image enhancement model, and performing multi-level beautification processing on the original image through the image enhancement model, includes: In the skin color optimization layer, the original image is converted to a color space based on the skin color HSV adjustment threshold, and the hue, saturation and lightness components are dynamically adjusted in the HSV domain. In the skin texture enhancement layer, texture smoothing is performed in the skin mask area by a generator network based on the skin smoothing intensity factor and the pore retention coefficient, and the original skin micro-texture is preserved by the pore retention coefficient constraint term; Based on the chest / hip deformation displacement vector, a triangular mesh is constructed at the key points of the chest / hip in the shape optimization layer, and local deformation is achieved through the elastic deformation function. The hierarchical processing results of the skin color optimization layer, the skin texture enhancement layer, and the body shape optimization layer are fused using Poisson fusion.

[0007] In an optional embodiment of the first aspect of the present invention, the step of receiving a video template selected by the user and automatically matching visual effects, motion transitions, and background music associated with the video template based on the content of the beautified image includes: The depth feature vector of the beautified image is extracted using a pre-trained ResNet convolutional neural network. Based on the detection results of key points of the face and body in the beautified image, the facial expression index is calculated. The HSV color space of the beautified image is analyzed to generate a dominant color histogram; Based on the video template identifier, load a predefined set of visual effects, a set of motion transition rules, and a pool of background music candidates from the template database; Calculate the similarity between each effect vector in the visual effects set and the depth feature vector to obtain an effects recommendation set; The target transition animation is selected from the set of animation transition rules based on the numerical range of the expression index; The deep feature vector and the dominant color histogram are input into a preset music matching model, which outputs a music feature vector and retrieves the target background music from the background music candidate pool based on the Euclidean distance nearest neighbor method of MFCC.

[0008] In an optional embodiment of the first aspect of the present invention, calculating the facial expression index based on the detection results of key points of the face and body in the beautified image includes: Based on 106 facial key points detected by the Dlib algorithm, the center point of the upper jaw, the left corner of the mouth, and the right corner of the mouth were extracted; A local coordinate system is constructed with the tip of the nose as the origin and the line connecting the centers of the eyes as the x-axis. The left corner of the mouth and the right corner of the mouth are projected onto the frontal plane using a rotation matrix. The corner of the mouth displacement is calculated and the neural network expression confidence score is obtained. The facial expression index is obtained by multiplying the corner of the mouth displacement, the neural network expression confidence score and the preset illumination compensation coefficient.

[0009] In an optional embodiment of the first aspect of the present invention, the step of using an automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithms to fuse the beautified image with the visual effects, the motion transitions, and the background music to generate a dynamic short video includes: Frame-level motion signals are generated based on lip and head movement parameters mapped from the acoustic features of the background music. The frame-level motion signal is mapped into a pixel-level deformation field. The beautified image is spatially deformed and holes are repaired through a deformable mesh to generate a primary animation frame sequence. The timing of the primary animation frame sequence is optimized. The primary animation frame sequence is video synthesized, and the beat points and rhythmic features of the background music are extracted by a beat detection algorithm to control the timing of motion effect transitions to align with the music beats. The visual effects are superimposed on the synthesized video, and color correction and frame rate smoothing are performed. The video is then encoded and output in a video container format to obtain a dynamic short video.

[0010] In an optional embodiment of the first aspect of the present invention, the generation of frame-level motion signals from lip and head movement parameters based on the acoustic feature mapping of the background music includes: The background music was subjected to a short-time Fourier transform to obtain its spectrum. A multidimensional audio embedding vector is extracted from the spectrogram using a pre-trained deep convolutional network, and rhythm intensity features are calculated. The audio embedding vector is input into the lip-driven LSTM network, which outputs frame-level lip shape parameters. A head rotation vector is generated based on the rhythm intensity feature; The frame-level lip shape parameters are processed by a three-dimensional face deformation function to generate lip motion signals; The head rotation vector is processed by a Kalman filter to generate a head motion signal.

[0011] A second aspect of the present invention provides an interactive system for an image enhancement platform, the interactive system comprising: The login receiving module is used to receive user login requests and establish user account sessions; The image receiving module is used to receive the original images uploaded by the user and generate an enhancement interface; The account verification module is used to determine whether the user's account has activated its subscription status or holds sufficient virtual points in response to the user's one-click beautification command. The image enhancement module is used to automatically call a preset enhancement parameter set when the user account has activated a subscription or holds sufficient virtual points, and to enhance the original image according to the enhancement parameter set to generate an enhanced image. The template provides a module for generating a preset video template selection interface in response to video generation instructions; The template matching module is used to receive the video template selected by the user and automatically match the visual effects, motion transitions and background music associated with the video template based on the content of the beautified image. The video compositing module is used to use an automatic compositing engine based on frame animation combination and audio-visual synchronization algorithms to merge the beautified image with the visual effects, the motion transitions and the background music to generate a dynamic short video; The resource sharing module is used to receive instructions from users to share the beautified image and / or the dynamic short video to a social media platform via a preset one-click sharing interface; The points reward module is used to issue a preset amount of virtual points to the user account in response to the successful sharing of the beautified image and / or the dynamic short video.

[0012] A third aspect of the present invention provides an image enhancement device, the image enhancement device comprising: a memory and at least one processor, the memory storing instructions, and the memory and the at least one processor being interconnected via a circuit; The at least one processor invokes the instructions in the memory to cause the image enhancement device to execute the interactive method of the image enhancement platform as described in any one of the first aspects of the present invention.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the interactive method of the image enhancement platform as described in any one of the first aspects of the present invention.

[0014] Beneficial Effects: This invention provides an interactive method, system, and related equipment for an image enhancement platform. The interactive method constructs a comprehensive user system. After a user triggers a one-click enhancement command, the platform determines whether the user's account has activated a subscription or holds sufficient virtual points. Only when the user's account has activated a subscription or holds sufficient virtual points will the enhancement parameter set be invoked to enhance the original image. During video template matching, the platform can automatically match visual effects, motion transitions, and background music associated with the video template based on the content of the enhanced image. Furthermore, this invention provides a one-click sharing interface, allowing users to share the enhanced image and / or dynamic short videos to social media platforms. After successful sharing of the enhanced image and / or dynamic short video, the user's account also receives a preset amount of virtual points. The image enhancement interactive system of this invention is more complete, more playable, and better meets user needs. Attached Figure Description

[0015] Figure 1 This is a schematic diagram illustrating one embodiment of the main steps of an interactive method for an image enhancement platform according to the present invention; Figure 2 This is a schematic diagram of an embodiment of the interactive system of an image enhancement platform according to the present invention; Figure 3 This is a schematic diagram of an embodiment of an image enhancement device according to the present invention. Detailed Implementation

[0016] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 The first aspect of this invention provides an interactive method for an image enhancement platform, comprising: S100: Receive the user's login request and establish a user account session; In one exemplary case of the present invention, the user may submit a login request through a mobile terminal, and complete the identity verification based on a mobile phone number, email address, or third-party social account to establish a user account session.

[0018] This invention's image enhancement platform introduces a virtual points (such as gold coins) mechanism, which can be used to redeem advanced enhancement functions, premium video templates, or watermark removal privileges. Users can earn points through daily tasks, social sharing, or top-ups. The platform also offers various subscription packages (such as monthly, quarterly, and annual cards), with subscribed users enjoying more privileges and unlocked content. The payment function integrates with a third-party platform to ensure payment security and a smooth experience, while establishing a points-and-subscription linkage strategy to incentivize frequent user engagement and paid conversion.

[0019] S200: Receive the original image uploaded by the user and generate a beautification operation interface; for example, in this step, the original image may also be uploaded or taken by the user through a mobile terminal. The beautification operation interface includes a one-click beautification command button, as well as an original image display area and a beautified image display area for the user to compare the images before and after beautification.

[0020] S300: In response to a user-triggered one-click beautification command, determine whether the user account has an active subscription or sufficient virtual points. In a specific implementation scenario of this invention, the one-click beautification command can be subdivided into multiple levels. A standard one-click beautification command does not require subscription or points purchase, while a high-level one-click beautification command requires verification of whether a subscription is active or sufficient virtual points are available. The difference between standard and high-level one-click beautification commands may lie in the different image beautification items targeted and / or the different beautification models used. Taking a high-level one-click beautification command as an example, when providing advanced image beautification processing, it first determines whether the user account has an active subscription or sufficient virtual points. If the beautification operation option involves advanced functions and the user account has not activated a valid subscription and lacks sufficient virtual points, the user is prompted to obtain points or subscribe.

[0021] S400. When the user account has activated its subscription or holds sufficient virtual points, a preset beautification parameter set is automatically invoked. The original image is then beautified according to the beautification parameter set to generate a beautified image. In this invention, image beautification is mainly accomplished through integrated image processing algorithms and AI recognition models. The model performs facial and body key point detection, region segmentation, and feature extraction on images uploaded or captured by the user. The system automatically identifies key facial features, skin texture, chest, buttocks, and other key areas. Users can choose a one-click beautification mode (invoking the preset beautification parameter set) or manually adjust beautification parameters (such as skin tone optimization, breast and buttock augmentation ratios, etc.). The beautification effect can be achieved by an image enhancement model based on generative adversarial networks, ensuring naturalness and diversity, and all operations can be previewed in real time on mobile devices.

[0022] In an optional embodiment of step S400 of the present invention, when the user account has activated a subscription or holds sufficient virtual points, automatically calling a preset beautification parameter set, and beautifying the original image according to the beautification parameter set to generate a beautified image includes: S401. When it is detected that the user account has an active and valid subscription or holds sufficient virtual points, the system identifies the facial proportions, skin type, and chest / hip contour features in the original image based on a convolutional neural network model, and matches the optimal beautification parameter set from a preset parameter library. The preset parameter library stores a large number of "beautification parameter sets" for different combinations of image features. Each "beautification parameter set" contains a specific set of image processing instructions and intensity values. The system uses the specific features of the current user image identified by the CNN (e.g., round face, oily skin with acne scars, specific chest contour) as "query conditions." The system searches the preset parameter library for the parameter set that best matches these feature conditions. The matching process can use a machine learning model for recommendation, predicting which parameter combination will achieve the highest performance score for the current features.

[0023] S402. Input the beautification parameter set into the image enhancement model of the generative adversarial network, and perform multi-level beautification processing on the original image through the image enhancement model. The beautification parameter set includes a skin tone HSV adjustment threshold, a skin smoothing intensity factor, a breast / hip deformation displacement vector, and a pore preservation coefficient. Specifically, this step may include: performing color space conversion on the original image in the skin tone optimization layer based on the skin tone HSV adjustment threshold (non-RGB direct operation to avoid color bias), dynamically adjusting the hue, saturation, and brightness components in the HSV domain; and in the skin texture enhancement layer, adjusting the skin smoothing intensity factor and the pore preservation coefficient... (Solving the problem of plastic-like skin smoothing in traditional skin smoothing), texture smoothing is performed within the skin mask area through a generator network, and the original skin micro-texture is preserved through the pore preservation coefficient constraint term; in the body optimization layer, triangular meshes are constructed at the key points of the chest / buttocks based on the chest / buttock deformation displacement vector, and local deformation is achieved through an elastic deformation function (the deformation weight can be configured to decay exponentially with the distance to the key point (ensuring a natural transition)); the hierarchical processing results of the skin color optimization layer, the skin texture enhancement layer, and the body optimization layer are fused using Poisson fusion. This invention solves the problem of boundary artifacts in the hierarchical processing results by using Poisson fusion instead of direct superposition.

[0024] S403. The skin region and non-skin region of the processed original image are fused pixel-level to output the beautified image. In this step, a skin region mask M can be obtained first, where M is a binary image, the pixel value of the skin region is 1, and the pixel value of the non-skin region is 0; Gaussian blur is applied to the mask M to obtain a weight map W, where each pixel value in W represents the weight of the skin region at that position, and the value changes continuously in [0, 1]; based on the weight map W, the fused image is calculated by a weighted mixing formula, and edge-guided filtering is applied to the fused image to eliminate mixing boundary artifacts.

[0025] S500: In response to the video generation command triggered by the user, a preset video template selection interface is generated; the video template selection interface of the present invention provides various types of video templates for the user to choose from, and different video template types have different sets of visual effects, sets of motion transition rules and background music candidate pools.

[0026] S600: Receive the video template selected by the user and, based on the content of the beautified image, automatically match visual effects, motion transitions, and background music associated with the video template. After image beautification is completed, in this invention, the platform provides the function of generating short videos from static images. Users can choose from preset fun templates (such as magic transformation, character dress-up, dynamic filters, etc.), and the platform automatically matches the most suitable visual effects, motion transitions, and music rhythm based on the image content. The video generation process uses an automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithms. Users can customize the cover and rhythm points, making the gameplay both personalized and entertaining.

[0027] In an optional embodiment of step S600 of the present invention, receiving the video template selected by the user and automatically matching visual effects, motion transitions, and background music associated with the video template based on the content of the beautified image includes: S601. Extract the depth feature vector of the beautified image through a pre-trained ResNet convolutional neural network; In this invention, after the static image is input into the ResNet convolutional neural network, it will perform operations such as face / body key point detection, surface / volume semantic segmentation, depth / normal estimation, and extraction of identity features (ID embedding).

[0028] S602. Based on the detection results of key points of the face and body in the beautified image, calculate the facial expression index; specifically, this step may include: extracting the center point of the maxilla, the left corner of the mouth, and the right corner of the mouth from 106 key points detected by the Dlib algorithm; constructing a local coordinate system with the tip of the nose as the origin and the line connecting the centers of the eyes as the x-axis; projecting the left corner of the mouth and the right corner of the mouth onto the frontal plane through a rotation matrix; calculating the corner of the mouth displacement and obtaining the neural network expression confidence score; and obtaining the facial expression index by multiplying the corner of the mouth displacement, the neural network expression confidence score, and a preset illumination compensation coefficient.

[0029] S603. Analyze the HSV color space of the beautified image to generate a dominant hue histogram. In this step, the process may involve converting the image from the RGB color space to the HSV color space, statistically analyzing the H hue channels in the HSV, and ignoring areas with low saturation and brightness. The hues are then binned, and the number of pixels (or weights) in each bin is counted. Finally, smoothing is performed, or the peak value is selected as the dominant hue.

[0030] S604. Based on the video template identifier, load the predefined visual effects set, motion transition rule set, and background music candidate pool from the template database. In this invention, different video templates have different video template identifiers (IDs). By matching video template identifiers, the target visual effects set, target motion transition rule set, and target background music candidate pool corresponding to the target video template identifier can be obtained.

[0031] S605. Calculate the similarity between each effect vector in the visual effects set and the depth feature vector to obtain an effect recommendation set; for example, the feature similarity can be calculated using the following formula s e =cos( ϕ e ( f ), ψ e ( e i )),in ϕ e For special effects encoder, ψ e For image feature mapping networks, select s e The recommended set of special effects is >0.85.

[0032] S606. Select a target transition animation from the animation transition rule set based on the numerical range of the expression index; for example, based on the expression index. E Select transition intensity: E <0.3 → fade in / fade out, 0.3≤ E <0.7 → Sliding transition E ≥0.7 → Particle explosion transition.

[0033] S607. Input the depth feature vector and the dominant color histogram into a preset music matching model, output a music feature vector, and retrieve the target background music from the background music candidate pool based on the Euclidean distance nearest neighbor method of MFCC (Melbourne Frequency Cepstral Coefficients). Input the concatenated vector of image features and color histogram into the music model (3-layer MLP+ReLU), calculate the nearest neighbor between the output music feature vector and the feature vector of each music in the music candidate pool based on the L2 distance (i.e., Euclidean distance) of MFCC (Melbourne Frequency Cepstral Coefficients), and finally retrieve the target background music.

[0034] S700: An automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithm is used to merge the beautified image with the visual effects, the motion transitions and the background music to generate a dynamic short video. In this invention, the video generation process uses an automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithm. For example, based on the frame animation combination principle, multiple animation keyframes (start position, scaling, rotation, transparency) are set for each static image, and a smooth frame sequence is generated through interpolation functions (linear interpolation / cubic spline / ease-in-out curve).

[0035] In an optional embodiment of step S700 of the present invention, the step of using an automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithms to fuse the beautified image with the visual effects, the motion transitions, and the background music to generate a dynamic short video includes: S701. Generate frame-level motion signals based on lip and head movement parameters mapped from the acoustic features of the background music. Specifically, this step may include: performing a short-time Fourier transform on the background music to obtain a spectrogram; extracting multi-dimensional (e.g., 128-dimensional) audio embedding vectors from the spectrogram using a pre-trained deep convolutional network (VGG), and calculating rhythm intensity features (focusing on the 100-500Hz frequency band sensitive to the human ear); inputting the audio embedding vectors into a lip-driven LSTM network to output frame-level lip shape parameters; generating head rotation vectors (including yaw and pitch angles) based on the rhythm intensity features; processing the frame-level lip shape parameters using a three-dimensional face deformation function (e.g., the Blendshape function) to generate lip motion signals; and processing the head rotation vectors using a Kalman filter (to eliminate random micro-vibrations) to generate head motion signals. In some alternative embodiments of the present invention, in addition to audio driving, frame-level motion signals can also be driven by preset motion templates (breathing, blinking, head turning, smiling, etc.); user dragging of key points / control handles; or driving from short video examples (driving video or key point sequences).

[0036] S702. The frame-level motion signal is mapped to a pixel-level deformation field. The beautified image is spatially deformed and hole repair is performed through a deformable mesh to generate a primary animation frame sequence. In this step, for example, the motion signal can be mapped to a pixel-level displacement field / optical flow / deformable mesh to perform warping and inpainting on the original image, generating preliminary results for each frame. A generative network (GAN) is then used to repair holes caused by warping, preserve texture, and remove artifacts.

[0037] S703. Perform time-series optimization on the primary animation frame sequence. In this invention, after using a generative network to repair holes caused by warping, preserve texture, and remove artifacts, it is also necessary to impose time-series consistency constraints on the primary animation frame sequence.

[0038] S704. Perform video synthesis on the primary animation frame sequence, extract the beat points and rhythmic features of the background music through a beat detection algorithm, and control the timing of motion effect transitions to align with the music beats. To achieve beat synchronization between the animation frame sequence and the background music, the key is to accurately extract the music beat points and dynamically control the timing of animation frame switching / transitions. In this step, an audio processing library (such as LibROSA) can be used to extract the music beat time points: when performing time synchronization control, the beat timestamps are mapped to the video timeline to dynamically control frame switching.

[0039] S705. The visual effects are superimposed on the synthesized video, and color correction and frame rate smoothing are performed. The video is then encoded and output in a video container format to obtain a dynamic short video. In this step, visual effects are superimposed on each frame of the synthesized video. Color correction can be achieved by adjusting brightness, contrast, and saturation, etc. Frame rate smoothing can be achieved through interpolation or motion blur, which helps to avoid jumps caused by beat switching.

[0040] S800: Receive an instruction from a user to share the beautified image and / or the dynamic short video to a social platform via a preset one-click sharing interface; In this invention, after generating content, the user can share it to mainstream social platforms (such as Instagram, TikTok, Xiaohongshu, etc.) with one click.

[0041] S900: In response to the successful sharing of the beautified image and / or the dynamic short video, a preset amount of virtual points is issued to the user's account. In this invention, users can obtain extra points or exclusive content packages after sharing, forming a closed loop of content dissemination. At the same time, the platform provides users with a history of works, a draft box, a personal homepage, and a level system, building a foundation for user stickiness and community interaction.

[0042] In summary, the image enhancement platform of this invention has four core functionalities: (1) Intelligent Image Recognition and Personalized Enhancement: Through integrated image processing algorithms and AI recognition models, the system performs face and body key point detection, region segmentation, and feature extraction on user-uploaded or captured images. The system automatically recognizes key facial features, skin texture, chest, buttocks, and other key areas. Users can choose a one-click enhancement mode or manually adjust enhancement parameters (such as skin tone optimization, breast and buttock augmentation ratios, etc.). The enhancement effect is achieved by an image enhancement model based on GAN (Generative Adversarial Network), ensuring naturalness and diversity, and all operations can be previewed in real time on mobile devices.

[0043] (2) Intelligent generation of images to videos and integration of gameplay templates: After image beautification, the platform provides the function of generating short videos from static images. Users can choose from preset fun templates (such as magic transformation, character dress-up, dynamic filters, etc.), and the platform automatically matches the most suitable visual effects, motion transitions and music rhythms according to the image content. The video generation process adopts an automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithms. Users can customize the cover and rhythm points, making the gameplay content both personalized and entertaining.

[0044] (3) Integration of Points and Subscription Payment System: The system introduces a virtual points (such as gold coins) mechanism, which can be used to redeem privileged content such as advanced beautification functions, advanced video templates, or watermark removal. Users can earn points through daily tasks, social sharing, or top-ups. The platform also offers a variety of subscription packages (such as monthly cards, quarterly cards, and annual cards), and subscribed users can enjoy more privileges and content unlocks. The payment function is integrated with third-party platforms (such as Google Play) to ensure payment security and a smooth experience, while establishing a points and subscription linkage strategy to incentivize users to use the service more frequently and convert to paid subscriptions.

[0045] (4) User System Construction and Social Sharing Closed Loop: The platform supports multiple login methods, including mobile phone number, email, and third-party social accounts (such as Google, Facebook, etc.), and provides a complete registration, login, and password retrieval mechanism. After generating content, users can share it to mainstream social platforms (such as Instagram, TikTok, Xiaohongshu, etc.) with one click. After sharing, users can get extra points or exclusive content packages, forming a closed loop of content dissemination. At the same time, the platform provides users with a history of works, drafts, personal homepage, and a level system to build the foundation for user stickiness and community interaction.

[0046] See Figure 2 The second aspect of the present invention provides an interactive system for an image enhancement platform, the interactive system of which includes: The login receiving module 10 is used to receive user login requests and establish user account sessions. Image receiving module 20 is used to receive the original image uploaded by the user and generate an enhancement operation interface; The account judgment module 30 is used to determine whether the user's account has activated the subscription status or has sufficient virtual points in response to the one-click beautification command triggered by the user. The image enhancement module 40 is used to automatically call a preset enhancement parameter set when the user account has activated a subscription or holds sufficient virtual points, and to enhance the original image according to the enhancement parameter set to generate an enhanced image. Template provides module 50, which generates a preset video template selection interface in response to video generation instructions; The template matching module 60 is used to receive the video template selected by the user and automatically match the visual effects, motion transitions and background music associated with the video template based on the content of the beautified image. The video compositing module 70 is used to use an automatic compositing engine based on frame animation combination and audio-visual synchronization algorithms to merge the beautified image with the visual effects, the motion transitions and the background music to generate a dynamic short video. The resource sharing module 80 is used to receive instructions from users to share the beautified image and / or the dynamic short video to a social platform via a preset one-click sharing interface; The points reward module 90 is used to issue a preset amount of virtual points to the user account in response to the successful sharing of the beautified image and / or the dynamic short video.

[0047] In an optional embodiment of the second aspect of the present invention, the image enhancement module includes: The parameter loading unit is used to identify the facial proportions, skin type, and chest / hip contour features in the original image based on a convolutional neural network model when it is detected that the user account has activated a valid subscription or holds sufficient virtual points, and to match the optimal set of beautification parameters from a preset parameter library. The beautification processing unit is used to input the beautification parameter set into the image enhancement model of the generative adversarial network, and perform multi-level beautification processing on the original image through the image enhancement model. The beautification parameter set includes skin color HSV adjustment threshold, skin smoothing intensity factor, breast / buttock deformation displacement vector and pore preservation coefficient. The region fusion unit is used to perform pixel-level fusion of the skin region and non-skin region of the processed original image to output the beautified image. In an optional embodiment of a second aspect of the present invention, the beautification processing unit includes: The skin color optimization subunit is used to perform color space conversion on the original image based on the skin color HSV adjustment threshold in the skin color optimization layer, and dynamically adjust the hue, saturation and lightness components in the HSV domain. The skin texture enhancement subunit is used to perform texture smoothing in the skin mask area through a generator network based on the skin smoothing intensity factor and the pore retention coefficient in the skin texture enhancement layer, and to retain the original skin micro-texture through the pore retention coefficient constraint term; The shape optimization sub-unit is used to construct triangular meshes at key points of the chest / hips based on the chest / hip deformation displacement vector in the shape optimization layer, and realize local deformation through elastic deformation function; The hierarchical fusion subunit is used to perform Poisson fusion on the hierarchical processing results of the skin color optimization layer, the skin texture enhancement layer, and the body shape optimization layer.

[0048] In an optional embodiment of the second aspect of the present invention, the template matching module includes: The feature vector extraction unit is used to extract the depth feature vector of the beautified image through a pre-trained ResNet convolutional neural network. The facial expression index calculation unit is used to calculate the facial expression index based on the detection results of key points of the face and body in the beautified image; The color space analysis unit is used to analyze the HSV color space of the beautified image and generate a main color histogram. The template data loading unit is used to load a predefined set of visual effects, a set of motion transition rules, and a pool of background music candidates from the template database according to the video template identifier. A visual effects recommendation unit is used to calculate the similarity between each effect vector in the visual effects set and the deep feature vector to obtain an effects recommendation set. The transition effect recommendation unit is used to select target transition effects from the set of animation transition rules based on the numerical range of the expression index; The background music recommendation unit is used to input the deep feature vector and the dominant color histogram into a preset music matching model, output a music feature vector, and retrieve the target background music from the background music candidate pool based on the Euclidean distance nearest neighbor method of MFCC.

[0049] In an optional embodiment of the second aspect of the present invention, the expression index calculation unit includes: The key point detection subunit is used to detect 106 facial key points based on the Dlib algorithm, and extract the maxillary center point, left corner of mouth point and right corner of mouth point; A coordinate system construction sub-unit is used to construct a local coordinate system with the tip of the nose as the origin and the line connecting the centers of the eyes as the x-axis. The left corner of the mouth and the right corner of the mouth are projected onto the frontal plane through a rotation matrix. The facial expression index calculation subunit is used to calculate the corner of the mouth displacement and obtain the neural network expression confidence score. The facial expression index is obtained by multiplying the corner of the mouth displacement, the neural network expression confidence score, and a preset illumination compensation coefficient.

[0050] In an optional embodiment of the second aspect of the present invention, the video synthesis module includes: A motion signal generation unit is used to generate frame-level motion signals based on lip and head movement parameters mapped from the acoustic features of the background music. An animation frame sequence generation unit is used to map the frame-level motion signal into a pixel-level deformation field, perform spatial deformation on the beautified image through a deformable mesh and perform hole repair to generate a primary animation frame sequence. A frame sequence timing optimization unit is used to perform timing optimization on the primary animation frame sequence; The beat alignment unit is used to perform video synthesis on the primary animation frame sequence, extract the beat points and rhythmic features of the background music through a beat detection algorithm, and control the timing of motion effect transitions to align with the music beats. The special effects overlay unit is used to overlay the visual effects onto the synthesized video, perform color correction and frame rate smoothing processing, and encode the output into a video container format to obtain a dynamic short video.

[0051] In an optional embodiment of the second aspect of the present invention, the motion signal generation unit includes: The spectrum extraction subunit is used to perform a short-time Fourier transform on the background music to obtain a spectrum. The vector extraction and intensity calculation subunit is used to extract multidimensional audio embedding vectors from the spectrogram through a pre-trained deep convolutional network and calculate rhythm intensity features. The lip shape parameter output subunit is used to input the audio embedding vector into the lip-driven LSTM network and output frame-level lip shape parameters. A head motion vector generation subunit is used to generate a head rotation vector based on the rhythm intensity feature; The lip motion signal generation subunit is used to process the frame-level lip shape parameters through a three-dimensional face deformation function to generate lip motion signals. The head motion signal generation subunit is used to process the head rotation vector through a Kalman filter to generate a head motion signal.

[0052] Figure 3 This is a schematic diagram of the structure of an image enhancement device provided in an embodiment of the present invention. The image enhancement device can vary significantly due to differences in configuration or performance, and may include one or more processors 80 (central processing units, CPUs) (e.g., one or more processors) and memory 90, and one or more storage media 100 (e.g., one or more mass storage devices) for storing applications or data. The memory and storage media can be temporary or persistent storage. The program stored in the storage media may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the image enhancement device. Furthermore, the processor may be configured to communicate with the storage media to execute the series of instruction operations stored in the storage media on the image enhancement device.

[0053] The image enhancement device of the present invention may further include one or more power supplies 110, one or more wired or wireless network interfaces 120, one or more input / output interfaces 130, and / or one or more operating systems, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated image enhancement device structure does not constitute a limitation on the image enhancement device, and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0054] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the interactive method of the image enhancement platform.

[0055] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system or system / unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0056] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0057] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An interactive method for an image enhancement platform, characterized in that, include: Receive user login requests and establish user account sessions; Receive the original image uploaded by the user and generate an enhancement interface; In response to a user-triggered one-click beautification command, determine whether the user's account is in active subscription status or has sufficient virtual points; When the user account has activated a subscription or holds sufficient virtual points, the preset beautification parameter set is automatically invoked, and the original image is beautified according to the beautification parameter set to generate a beautified image; In response to a user-triggered video generation command, a preset video template selection interface is generated; The system receives a video template selected by the user and, based on the content of the beautified image, automatically matches visual effects, motion transitions, and background music associated with the video template. This automatic matching includes: The depth feature vector of the beautified image is extracted using a pre-trained ResNet convolutional neural network. Based on the 106 facial key points detected by the Dlib algorithm in the beautified image, a local coordinate system is constructed with the tip of the nose as the origin and the line connecting the centers of the eyes as the x-axis. The left and right corner points of the mouth are projected onto the frontal plane through a rotation matrix. The displacement of the corner of the mouth is calculated and multiplied with the neural network expression confidence score output by the ResNet convolutional neural network and the preset illumination compensation coefficient to obtain the facial expression index. The HSV color space of the beautified image is analyzed to generate a dominant color histogram; Based on the video template identifier, load a predefined set of visual effects, a set of motion transition rules, and a pool of background music candidates from the template database; Calculate the similarity between each effect vector in the visual effects set and the depth feature vector to obtain an effects recommendation set; The target transition animation is selected from the set of animation transition rules based on the numerical range of the expression index; The deep feature vector and the dominant color histogram are input into a preset music matching model, the music feature vector is output, and the target background music is retrieved from the background music candidate pool based on the Euclidean distance nearest neighbor method of MFCC. An automatic synthesis engine based on frame animation combination and audio-visual synchronization algorithm is used to fuse the beautified image with the visual effects, the motion transitions and the background music to generate a dynamic short video. The fusion includes: The background music is subjected to a short-time Fourier transform to obtain a spectrogram. A multidimensional audio embedding vector is extracted from the spectrogram using a pre-trained deep convolutional network, and rhythm intensity features are calculated. The audio embedding vector is input into the lip-driven LSTM network, which outputs frame-level lip shape parameters. A head rotation vector is generated based on the rhythm intensity features. The frame-level lip shape parameters are processed by a three-dimensional face deformation function to generate lip motion signals. The head rotation vector is processed by a Kalman filter to generate head motion signals, thereby obtaining a frame-level motion signal that includes the lip motion signals and the head motion signals. The frame-level motion signal is mapped to a pixel-level deformation field. The beautified image is spatially deformed and hole repair is performed through a deformable mesh to generate a primary animation frame sequence. The primary animation frame sequence is then time-optimized. The primary animation frame sequence is video synthesized, and the beat points and rhythmic features of the background music are extracted by a beat detection algorithm to control the timing of motion effect transitions to align with the music beats. The visual effects are superimposed on the synthesized video, and color correction and frame rate smoothing are performed. The video is then encoded and output in a video container format to obtain a dynamic short video. The system receives instructions from users to share the enhanced image and / or the dynamic short video to a social media platform via a pre-defined one-click sharing interface. Upon successful sharing of the beautified image and / or the dynamic short video, a preset amount of virtual points is issued to the user's account.

2. The interactive method of the image enhancement platform according to claim 1, characterized in that, When the user account has activated its subscription or holds sufficient virtual points, a preset beautification parameter set is automatically invoked, and the original image is beautified according to the beautification parameter set to generate a beautified image, including: When it is detected that the user account has activated a valid subscription status or holds sufficient virtual points, the facial proportions, skin type and chest / hip contour features in the original image are identified based on the convolutional neural network model, and the optimal beautification parameter set is matched from the preset parameter library. The beautification parameter set is input into the image enhancement model of the generative adversarial network, and the image enhancement model performs multi-level beautification processing on the original image. The beautification parameter set includes skin color HSV adjustment threshold, skin smoothing intensity factor, breast / buttock deformation displacement vector and pore preservation coefficient. The skin and non-skin regions of the processed original image are fused pixel-level to output the beautified image.

3. The interactive method of the image enhancement platform according to claim 2, characterized in that, The step of inputting the beautification parameter set into the image enhancement model of the generative adversarial network, and performing multi-level beautification processing on the original image through the image enhancement model includes: In the skin color optimization layer, the original image is converted to a color space based on the skin color HSV adjustment threshold, and the hue, saturation and lightness components are dynamically adjusted in the HSV domain. In the skin texture enhancement layer, texture smoothing is performed in the skin mask area by a generator network based on the skin smoothing intensity factor and the pore retention coefficient, and the original skin micro-texture is preserved by the pore retention coefficient constraint term; Based on the chest / hip deformation displacement vector, a triangular mesh is constructed at the key points of the chest / hip in the shape optimization layer, and local deformation is achieved through the elastic deformation function. The hierarchical processing results of the skin color optimization layer, the skin texture enhancement layer, and the body shape optimization layer are fused using Poisson fusion.

4. An interactive system for an image enhancement platform, characterized in that, The interactive system of the image enhancement platform includes: The login receiving module is used to receive user login requests and establish user account sessions; The image receiving module is used to receive the original images uploaded by the user and generate an enhancement interface; The account verification module is used to determine whether the user's account has activated its subscription status or holds sufficient virtual points in response to the user's one-click beautification command. The image enhancement module is used to automatically call a preset enhancement parameter set when the user account has activated a subscription or holds sufficient virtual points, and to enhance the original image according to the enhancement parameter set to generate an enhanced image. The template provides a module for generating a preset video template selection interface in response to video generation instructions; The template matching module is used to receive the video template selected by the user and automatically match the visual effects, motion transitions and background music associated with the video template based on the content of the beautified image. The video compositing module is used to use an automatic compositing engine based on frame animation combination and audio-visual synchronization algorithms to merge the beautified image with the visual effects, the motion transitions and the background music to generate a dynamic short video; The resource sharing module is used to receive instructions from users to share the beautified image and / or the dynamic short video to a social media platform via a preset one-click sharing interface; The points reward module is used to issue a preset amount of virtual points to the user account in response to the successful sharing of the beautified image and / or the dynamic short video; The template matching module includes: The feature vector extraction unit is used to extract the depth feature vector of the beautified image through a pre-trained ResNet convolutional neural network. The facial expression index calculation unit is used to calculate the facial expression index based on the detection results of key points of the face and body in the beautified image; The color space analysis unit is used to analyze the HSV color space of the beautified image and generate a main color histogram. The template data loading unit is used to load a predefined set of visual effects, a set of motion transition rules, and a pool of background music candidates from the template database according to the video template identifier. A visual effects recommendation unit is used to calculate the similarity between each effect vector in the visual effects set and the deep feature vector to obtain an effects recommendation set. The transition effect recommendation unit is used to select target transition effects from the set of animation transition rules based on the numerical range of the expression index; The background music recommendation unit is used to input the deep feature vector and the dominant color histogram into a preset music matching model, output a music feature vector, and retrieve the target background music from the background music candidate pool based on the Euclidean distance nearest neighbor method of MFCC. The facial expression index calculation unit includes: The key point detection subunit is used to detect 106 facial key points in the beautified image based on the Dlib algorithm, and extract the maxillary center point, left corner of mouth point and right corner of mouth point; A coordinate system construction sub-unit is used to construct a local coordinate system with the tip of the nose as the origin and the line connecting the centers of the eyes as the x-axis. The left corner of the mouth and the right corner of the mouth are projected onto the frontal plane through a rotation matrix. The facial expression index calculation subunit is used to calculate the corner of the mouth displacement and obtain the neural network expression confidence score output by the ResNet convolutional neural network. The facial expression index is obtained by multiplying the corner of the mouth displacement, the neural network expression confidence score and the preset illumination compensation coefficient. The video synthesis module includes: A motion signal generation unit is used to generate frame-level motion signals based on lip and head movement parameters mapped from the acoustic features of the background music. An animation frame sequence generation unit is used to map the frame-level motion signal into a pixel-level deformation field, perform spatial deformation on the beautified image through a deformable mesh and perform hole repair to generate a primary animation frame sequence. A frame sequence timing optimization unit is used to perform timing optimization on the primary animation frame sequence; The beat alignment unit is used to perform video synthesis on the primary animation frame sequence, extract the beat points and rhythmic features of the background music through a beat detection algorithm, and control the timing of motion effect transitions to align with the music beats. The special effects overlay unit is used to overlay the visual effects onto the synthesized video, perform color correction and frame rate smoothing processing, and encode the output into a video container format to obtain a dynamic short video. The motion signal generation unit includes: The spectrum extraction subunit is used to perform a short-time Fourier transform on the background music to obtain a spectrum. The vector extraction and intensity calculation subunit is used to extract multidimensional audio embedding vectors from the spectrogram through a pre-trained deep convolutional network and calculate rhythm intensity features. The lip shape parameter output subunit is used to input the audio embedding vector into the lip-driven LSTM network and output frame-level lip shape parameters. A head motion vector generation subunit is used to generate a head rotation vector based on the rhythm intensity feature; The lip motion signal generation subunit is used to process the frame-level lip shape parameters through a three-dimensional face deformation function to generate lip motion signals. The head motion signal generation subunit is used to process the head rotation vector through a Kalman filter to generate a head motion signal, wherein the frame-level motion signal includes the lip motion signal and the head motion signal.

5. An image enhancement device, characterized in that, The image enhancement device includes: a memory and at least one processor, wherein the memory stores instructions, and the memory and the at least one processor are interconnected via a line; The at least one processor invokes the instructions in the memory to cause the image enhancement device to execute the interactive method of the image enhancement platform as described in any one of claims 1-3.

6. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements the interactive method of the image enhancement platform as described in any one of claims 1-3.

Images