Artificial intelligence-based data processing method and device, computer device and medium

Abstract

The application belongs to the technical field of artificial intelligence, and relates to a data processing method and device based on artificial intelligence, a computer device and a storage medium, which comprise the following steps: receiving input facial image data; pre-processing the facial image data based on a processing strategy to obtain a texture map sequence and a shape vector; calling a synthesis processing model; wherein the synthesis processing model comprises a pyramid decomposition network, a parameter generation model and a fusion network; processing the texture map sequence based on the pyramid decomposition network to obtain texture parameters; processing the shape vector and the texture parameters based on the parameter generation model to obtain target parameters; performing fusion processing on the texture parameters and the target parameters based on the fusion network to obtain a synthesis sequence; and optimizing the synthesis sequence based on an optimization strategy to obtain target synthesis data. The application can be applied to the micro-expression data generation business scene in the fields of financial technology and medical health, and effectively improves the generation quality of synthesis data.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology and can be applied to fields such as fintech and healthcare, particularly to data processing methods, devices, computer equipment, and storage media based on artificial intelligence. Background Technology

[0002] Micro-expression recognition technology has significant application value in psychological assessment in the medical and health field and face-to-face interviews in the financial and insurance field. However, the extremely short duration of micro-expressions (typically 1 / 25 to 1 / 5 of a second) and the weak amplitude of movement (muscle contraction intensity less than 0.5 mm) result in extremely high costs for real data collection and annotation. Existing micro-expression datasets (such as CASME II and SAMM) have limited sample sizes, severely restricting the performance improvement of deep learning models. To alleviate the data shortage problem, generation techniques based on synthetic data have become a research hotspot. Existing synthesis techniques based on 3D deformation models (3DMM) (such as FacialPrior Guided Generation) can control the generated shape changes through geometric parameters, but the fidelity of texture dynamics (such as the propagation of skin wrinkles) is insufficient, resulting in low-quality generated data that is difficult to meet actual business needs.

[0003] For example, in insurance interviews in the financial sector, traditional interviews rely primarily on manual observation of the insured's facial expressions to determine the potential for fraud. If existing synthetic data-trained recognition models are used to replace manual interviews, the direction of skin folds in the synthetic data may not match the actual micro-expressions. The model might misinterpret normal texture fluctuations caused by lighting changes as deceptive micro-expressions, leading to numerous false positives. Interviewers would then need to review each mislabeled sample, increasing review costs and reducing efficiency. Similarly, in psychological assessments in the medical field, traditional assessments rely mainly on patient self-reports and scale scores, lacking objective quantitative analysis of facial micro-expressions. If existing synthetic data-trained models are used for sharing, the spatial distribution of wrinkles in the synthetic data may not match the direction of real facial muscle fibers. The model might fail to correctly identify subtle texture changes in the corner of the eye when a patient recalls a traumatic event, misinterpreting genuine pain micro-expressions as neutral expressions. This results in insufficient analytical evidence and delays in intervention.

[0004] Therefore, there is an urgent need to provide a high-fidelity micro-expression synthesis data generation method to overcome the bottleneck of real data shortage and improve the practical application effectiveness of micro-expression recognition technology in scenarios such as face-to-face interviews and psychological assessments. Summary of the Invention

[0005] The purpose of this application is to propose a data processing method, apparatus, computer device and storage medium based on artificial intelligence, so as to solve the technical problem that although the existing synthesis technology based on 3D deformation model can control the generation of shape changes through geometric parameters, the dynamic fidelity of texture is insufficient, resulting in low quality of generated data.

[0006] Firstly, an artificial intelligence-based data processing method is provided, including: Receive input facial image data; The facial image data is preprocessed based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector; A preset synthesis processing model is invoked; wherein, the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; The texture map sequence is processed based on the pyramid decomposition network to obtain the corresponding texture parameters; Based on the parameter generation model, the shape vector and the texture parameters are processed to obtain the corresponding target parameters; The texture parameters and the target parameters are fused based on the fusion network to obtain the corresponding synthesized sequence; The synthesized sequence is optimized based on a preset optimization strategy to obtain the corresponding target synthesized data.

[0007] Secondly, an artificial intelligence-based data processing device is provided, comprising: A receiving module is used to receive input facial image data; The preprocessing module is used to preprocess the facial image data based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector; The calling module is used to call a preset synthesis processing model; wherein, the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; The first processing module is used to process the texture map sequence based on the pyramid decomposition network to obtain the corresponding texture parameters; The second processing module is used to process the shape vector and the texture parameters based on the parameter generation model to obtain the corresponding target parameters; The fusion module is used to fuse the texture parameters and the target parameters based on the fusion network to obtain the corresponding synthesized sequence; The optimization module is used to optimize the synthesized sequence based on a preset optimization strategy to obtain the corresponding target synthesized data.

[0008] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described artificial intelligence-based data processing method.

[0009] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the aforementioned artificial intelligence-based data processing method.

[0010] In the aforementioned scheme implemented by the artificial intelligence-based data processing method, apparatus, computer equipment, and storage medium, input facial image data is first received; and the facial image data is preprocessed based on a preset processing strategy to obtain corresponding texture mapping sequences and shape vectors; then a preset synthesis processing model is invoked; wherein, the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; subsequently, the texture mapping sequence is processed based on the pyramid decomposition network to obtain corresponding texture parameters; subsequently, the shape vectors and texture parameters are processed based on the parameter generation model to obtain corresponding target parameters; further, the texture parameters and target parameters are fused based on the fusion network to obtain a corresponding synthesis sequence; finally, the synthesis sequence is optimized based on a preset optimization strategy to obtain corresponding target synthesis data. Based on the above automated processing flow, this application provides an intelligent data processing method. It uses a pyramid decomposition network to hierarchically deconstruct the texture map sequence according to spatial frequency, allowing texture parameters to be independently represented on the low-frequency global tone layer and the high-frequency skin micro-detail layer. This avoids the distortion of wrinkle propagation direction and spatial distribution misalignment caused by the overall dynamic mixing of textures in traditional 3DMM methods. Furthermore, a parameter generation model jointly encodes the shape vector and the layered texture parameters to generate target parameters that are strictly aligned with shape changes in spatial distribution. This ensures that dynamic texture changes evolve synchronously with shape changes in the correct facial region, solving the core defects of existing synthetic data, such as the disconnect between shape and texture changes and the mismatch between the spatial distribution of skin wrinkles and the direction of real facial muscle fibers. Finally, a fusion network performs pixel-level fusion of the layered texture parameters and target parameters, and an optimization strategy is used to optimize the synthetic sequence to output the target synthetic data. This method systematically improves the spatial frequency fidelity of texture dynamics and the shape-texture alignment accuracy in synthetic data through a full-link design from texture decomposition and joint parameter generation to fusion optimization. This effectively improves the generation quality of synthetic data and is beneficial for providing high-fidelity training data for micro-expression recognition models. Attached Figure Description

[0011] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the artificial intelligence-based data processing method according to this application; Figure 3 This is a schematic diagram of a structure of an embodiment of the artificial intelligence-based data processing apparatus according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0014] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0015] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0016] like Figure 1As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Terminal device 101 may be a laptop 1011, tablet 1012, or mobile phone 1013. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0017] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various communication client applications can be installed on terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0018] Terminal device 101 can be various electronic devices with a display screen and support web browsing. In addition to laptops 1011, tablets 1012, or mobile phones 1013, terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, and a desktop computer, etc.

[0019] Server 103 can be a server that provides various services, such as a backend server that provides support for the pages displayed on terminal device 101.

[0020] It should be noted that the artificial intelligence-based data processing method provided in the embodiments of this application is generally executed by a server / terminal device, and correspondingly, the artificial intelligence-based data processing device is generally set in the server / terminal device.

[0021] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0022] Continue to refer to Figure 2This document illustrates a flowchart of an embodiment of the AI-based data processing method according to this application. The order of steps in the flowchart can be changed, and some steps can be omitted, depending on different needs. The AI-based data processing method provided in this application can be applied to any scenario requiring data processing, and thus can be applied to products in these scenarios, such as data processing products in the financial insurance and healthcare fields. The AI-based data processing method includes the following steps: Step S201: Receive the input facial image data.

[0023] In this embodiment, the data processing method based on artificial intelligence runs on an electronic device (e.g., Figure 1 The server / terminal device shown can acquire input facial image data via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future known wireless connection methods. The executing entity of this application is specifically a data processing system, which may be simply referred to as the system.

[0024] The aforementioned facial image data consists of neutral facial video (no less than 30 frames, 256×256 pixels) or a single still image received by the system. The original format of the input facial image data is an RGB image sequence, which will be subsequently expanded to four channels (RGB additive lines).

[0025] Furthermore, this application can be applied to data synthesis business scenarios in the financial insurance and healthcare sectors. For example, in a face-to-face verification scenario in the financial insurance sector, the input facial image data can be a neutral facial video: the policyholder faces the camera for identity verification at the insurance counter for approximately 5 to 10 seconds, maintaining a natural standing posture with no significant facial expression changes; this constitutes a neutral face. Alternatively, the facial image data can also be a static image: a photo of the policyholder taken online during the application process, or a screenshot from a liveness detection scan, showing a calm expression without smiling, frowning, or squinting.

[0026] Furthermore, in psychological assessment scenarios within the healthcare field, the input facial image data can be a neutral facial video: the patient faces the doctor's camera in a psychiatric clinic for an initial emotional baseline acquisition, lasting approximately 30 seconds, during which the doctor guides the patient to relax and refrain from making any specific facial expressions, recording a neutral facial state without emotional load. Alternatively, the facial image data can also be a static image: a selfie uploaded by the patient to an online psychological assessment platform, showing a relaxed face without obvious emotional coloration.

[0027] Step S202: Preprocess the facial image data based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector.

[0028] In this embodiment, the specific implementation process of preprocessing the facial image data based on the preset processing strategy to obtain the corresponding texture map sequence and shape vector will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0029] Step S203: Invoke a preset synthesis processing model; wherein the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network.

[0030] In this embodiment, the above-mentioned synthesis processing model includes a parallel processing architecture of a dynamic texture branch (i.e., a pyramid decomposition network) and an AAM (Active Appearance Mode) branch (i.e., a parameter generation model). The dynamic texture branch decomposes the input texture through a Laplacian pyramid, with each level driven by a GRU (Gated Recurrent Unit) network to evolve the latent space; the AAM branch couples attention weights with shape / appearance parameters through a Transformer encoder. The outputs of the two branches are then aligned for multi-scale features in a fusion network.

[0031] Step S204: Process the texture map sequence based on the pyramid decomposition network to obtain the corresponding texture parameters.

[0032] In this embodiment, the specific implementation process of processing the texture map sequence based on the pyramid decomposition network to obtain the corresponding texture parameters will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0033] Step S205: Process the shape vector and the texture parameters based on the parameter generation model to obtain the corresponding target parameters.

[0034] In this embodiment, the specific implementation process of processing the shape vector and the texture parameters based on the parameter generation model to obtain the corresponding target parameters will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0035] Step S206: Based on the fusion network, the texture parameters and the target parameters are fused to obtain the corresponding synthesized sequence.

[0036] In this embodiment, the specific implementation process of fusing the texture parameters and the target parameters based on the fusion network to obtain the corresponding synthetic sequence will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0037] Step S207: Optimize the synthesized sequence based on a preset optimization strategy to obtain the corresponding target synthesized data.

[0038] In this embodiment, the specific implementation process of optimizing the synthesized sequence based on the preset optimization strategy to obtain the corresponding target synthesized data will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0039] This application first receives input facial image data; then preprocesses the facial image data based on a preset processing strategy to obtain corresponding texture mapping sequences and shape vectors; then calls a preset synthesis processing model; wherein the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; subsequently, the texture mapping sequence is processed based on the pyramid decomposition network to obtain corresponding texture parameters; subsequently, the shape vectors and texture parameters are processed based on the parameter generation model to obtain corresponding target parameters; further, the texture parameters and target parameters are fused based on the fusion network to obtain a corresponding synthesis sequence; finally, the synthesis sequence is optimized based on a preset optimization strategy to obtain the corresponding target synthesis data. Based on the above automated processing flow, this application provides an intelligent data processing method. It uses a pyramid decomposition network to hierarchically deconstruct the texture map sequence according to spatial frequency, allowing texture parameters to be independently represented on the low-frequency global tone layer and the high-frequency skin micro-detail layer. This avoids the distortion of wrinkle propagation direction and spatial distribution misalignment caused by the overall dynamic mixing of textures in traditional 3DMM methods. Furthermore, a parameter generation model jointly encodes the shape vector and the layered texture parameters to generate target parameters that are strictly aligned with shape changes in spatial distribution. This ensures that dynamic texture changes evolve synchronously with shape changes in the correct facial region, solving the core defects of existing synthetic data, such as the disconnect between shape and texture changes and the mismatch between the spatial distribution of skin wrinkles and the direction of real facial muscle fibers. Finally, a fusion network performs pixel-level fusion of the layered texture parameters and target parameters, and an optimization strategy is used to optimize the synthetic sequence to output the target synthetic data. This method systematically improves the spatial frequency fidelity of texture dynamics and the shape-texture alignment accuracy in synthetic data through a full-link design from texture decomposition and joint parameter generation to fusion optimization. This effectively improves the generation quality of synthetic data and is beneficial for providing high-fidelity training data for micro-expression recognition models.

[0040] In some alternative implementations, step S202 includes the following steps: Step S2021: Perform channel expansion on the facial image data to obtain the corresponding target image data.

[0041] In this embodiment, the aforementioned facial image data is a neutral facial video (no less than 30 frames, 256×256 pixels) or a single static image received by the system. The original facial image data is a three-channel RGB image sequence. For each frame, a corresponding normal map can be calculated based on the original RGB information (the surface normal vector is obtained by estimating the neighborhood gradient pixel by pixel). The RGB three channels are then concatenated with the normal channel to expand into four-channel (RGB + normal) texture data, which serves as the unified input format for subsequent texture modeling, i.e., the target image data, or simply...

[0042] The target image data is used to detect key points based on a preset feature extraction network to obtain the corresponding key point coordinates.

[0043] In this embodiment, the feature extraction network is specifically an improved HRNet network. Each frame's four-channel image is fed into the improved HRNet network, which extracts features using the four-channel data as input, outputting the two-dimensional pixel coordinates (x, y) of 68 FACS keypoints. These points cover key facial regions such as eyebrows, eye area, nose, lips, and jaw, forming the shape skeleton of the face in that frame. This detection is performed frame-by-frame on the video input, resulting in a 68×2×T keypoint coordinate sequence (T being the frame number), i.e., the keypoint coordinates.

[0044] The key point coordinates are subjected to non-rigid registration processing based on a preset registration algorithm to obtain the corresponding deformation field.

[0045] In this embodiment, the registration algorithm specifically employs a non-rigid ICP (Iterative Closest Point) registration algorithm. By using 68 keypoint coordinates as the source point set and 68 corresponding target points on a predefined standard 3D facial topology as the target point set, non-rigid ICP registration is performed. Unlike rigid registration, non-rigid ICP allows independent deformation of local regions. By iteratively optimizing the correspondence between the source and target point sets, neutral faces of different individuals are mapped to the same parameterized standard topological space, thereby eliminating differences in facial structure between individuals. During this registration iteration process, a standard lighting model on the target topology is simultaneously used to perform reverse lighting compensation on the source image. That is, the lighting deviation of the original image is inferred from the deformation field obtained through registration and corrected, achieving lighting normalization and eliminating uneven lighting caused by different shooting angles and light source positions, thus obtaining the registered texture map sequence.

[0046] Based on the deformation field, the target image data is subjected to non-rigid deformation mapping to obtain the corresponding texture map sequence.

[0047] In this embodiment, after registration is completed, the original four-channel texture image, i.e. the target image data, is non-rigidly deformed based on the deformation field obtained from registration, so that the texture is aligned with the standard topology, and a normalized texture map sequence (size 256×256, four channels) is obtained, which is the first core output.

[0048] Obtain the two-dimensional coordinates of the registered key points on the preset standard topology.

[0049] In this embodiment, the two-dimensional coordinates of the 68 registered key points on the standard topology can be obtained.

[0050] The two-dimensional coordinates are sequentially concatenated to obtain the corresponding shape vector.

[0051] In this embodiment, the two-dimensional coordinates of the 68 registered key points on the standard topology are sequentially spliced ​​together to form an initial shape vector with a dimension of 136 (68×2), which serves as the corresponding shape vector.

[0052] This application obtains target image data by channel expansion of facial image data; then, keypoint detection is performed on the target image data based on a preset feature extraction network to obtain the corresponding keypoint coordinates; subsequently, non-rigid registration processing is performed on the keypoint coordinates based on a preset registration algorithm to obtain the corresponding deformation field; and non-rigid deformation mapping is performed on the target image data based on the deformation field to obtain the corresponding texture map sequence; subsequently, the two-dimensional coordinates of the registered keypoints on a preset standard topology are obtained; and the two-dimensional coordinates are sequentially concatenated to obtain the corresponding shape vector. Based on the above processing flow, this application, by accurately extracting keypoints using a feature extraction network and registering them to a standard topology using non-rigid ICP, can automatically and accurately unify neutral faces of different people into the same parameter space, enabling subsequent dynamic texture processing and AAM modeling processing to be performed in a consistent coordinate system. Furthermore, illumination normalization ensures that the generated results will not produce artifacts due to differences in input illumination. In addition, this application can accurately transform raw pixel data into structured parameters operable by the model, ensuring the accuracy of the obtained texture map sequence and shape vector.

[0053] In some optional implementations of this embodiment, the pyramid decomposition network has multiple layers; step S204 includes the following steps: The texture map sequence is decomposed layer by layer based on the pyramid decomposition network to obtain multiple independent texture features.

[0054] In this embodiment, the aforementioned pyramid decomposition network specifically employs a five-layer Laplacian pyramid decomposition network. By inputting a texture map sequence into the pyramid decomposition network, the network first performs Gaussian downsampling on the four-channel texture and calculates the residuals between adjacent layers, thus separating five independent texture levels layer by layer. , , , , The first layer Preserves global hue and brightness distribution, with the lowest spatial frequency; Layer 5 It preserves the skin's microstructure, including pores, fine lines, and wrinkles, and has the highest spatial frequency. The bandwidth of each layer decreases by a factor of 2, i.e., the... The spatial frequency bandwidth contained in the layer is the first Half of layer 1 is used to ensure that the five texture layers do not overlap in the frequency domain and completely cover all the frequency information of the original texture. After decomposition, the features of the five texture layers are used as the initial inputs for the independent temporal evolution of each subsequent layer.

[0055] An independent recurrent neural network is initialized for each layer in the pyramid decomposition network, and the texture parameters are subjected to their respective temporal evolution processing based on the recurrent neural network to obtain the corresponding latent variables.

[0056] In this embodiment, each layer of the pyramid decomposition network is... ( Each independent GRU (Recurrent Neural Network) is initialized, and the GRU uses the latent variables corresponding to the texture of that layer. In the hidden state, the temporal evolution of latent variables is driven sequentially according to frame order. In frame t, the GRU receives the latent variables from the previous frame. As input, the latent variables of the current frame are calculated through the internal control mechanism. Its evolution formula is: ,in, and For the first The learnable weight matrix of a layered GRU The injected hierarchical Gaussian noise is used. Meanwhile, the noise variance for each layer is... Apply an exponential decay constraint: ,in, =0.6, Let V be the variance of the first-level base noise. Therefore, the variance of the first-level noise is... The second layer is 0.6. The third layer is The 4th floor is The 5th floor is This attenuation rule ensures that low-frequency layers (such as global brightness) allow for relatively large random fluctuations to simulate subtle changes in natural lighting, while noise in high-frequency layers (such as skin micro-textures) is significantly suppressed, guaranteeing extremely subtle changes in micro-textures that conform to the physiological characteristics of micro-expressions. The design of independent evolution of each GRU layer allows textures at different spatial frequencies to have their own independent dynamic rhythms, avoiding the problem of high-frequency details being submerged by low-frequency motion caused by global uniform evolution.

[0057] Texture reconstruction is performed by upsampling layer by layer based on the latent variables to obtain the corresponding reconstruction results.

[0058] In this embodiment, texture reconstruction starts from the coarsest layer ( =1) Start by synthesizing a complete texture layer by layer upwards. For the first layer... layer( ≥2), first, the first Latent variables of the current frame output by the layer GRU Through invertible basis matrices Mapping to the texture feature space yields At the same time, the first 1 layer reconstructed texture Perform upsampling operation This makes its spatial resolution comparable to the first Layer alignment. The mapping results are then... Compared with upsampling results Concatenate the features along the channel dimension. Then, feed the concatenated feature maps into a 3×3 convolutional layer for fusion and smoothing, outputting the first... Layer Reconstruction Texture ,Right now: The invertible basis matrix is... Implemented by an invertible neural network (INN), it ensures that the mapping from the latent space to the texture feature space is bidirectionally invertible, allowing the gradient of the reconstruction loss to be backpropagated losslessly to the weight matrix of the GRU network. , and basis matrix It achieves end-to-end joint optimization. The reconstruction of Layer 1 is directly handled by... through After mapping, it is obtained by 3×3 convolution. After iterating through the five layers to the fifth layer, the five reconstructed textures are stacked layer by layer according to the pyramid compositing rules, finally obtaining the complete reconstructed texture of the t-th frame. (Size 256×256×4), which is the output of the dynamic texture branch.

[0059] The reconstruction result is used as the texture parameter.

[0060] In this embodiment, after completing the texture reconstruction of the 5th layer (the highest frequency layer), further processing can be performed... An additional local differential constraint loss is applied. This constraint has its domain as the set of pixels P representing facial anatomical regions (including the frontalis muscle region, orbicularis oculi muscle region, perioral region, etc.), and calculates the second-order spatial derivative (i.e., curvature) at each pixel location p∈P. The Frobenius norm of the given values ​​is calculated, and the summation is applied over all partitioned pixels: The physical meaning of this loss is: it forces the curvature direction of the generated skin wrinkles to be consistent with the fiber orientation of the corresponding muscle region. For example, in the orbicularis oculi muscle region, wrinkles should be radially distributed rather than randomly oriented. The gradient of this loss propagates back along the reconstruction path in step three, directly updating the weight parameters and invertible basis matrix of each layer of GRU. The parameters are used to embed the facial anatomy knowledge of FACS into the texture generation process, ensuring that the dynamic changes of high-frequency textures conform to the physiological laws of real muscle movement.

[0061] This application decomposes texture mapping sequences layer by layer using a pyramid decomposition network to obtain multiple independent texture features. Then, an independent recurrent neural network is initialized for each layer of the pyramid decomposition network, and the texture features are subjected to corresponding temporal evolution processing based on the recurrent neural network to obtain corresponding latent variables. Subsequently, texture reconstruction is performed by layer-by-layer upsampling based on the latent variables to obtain the corresponding reconstruction results. These reconstruction results are then used as texture parameters. Based on the above processing flow, the core idea of ​​the layered dynamic texture modeling provided in this application is that "textures with different spatial frequencies should have different dynamic characteristics." In real faces, large muscle contractions (such as cheek lifting) mainly affect low-frequency textures (overall skin tone and brightness), while micro-wrinkles of the skin (such as fine lines around the eyes) only undergo extremely small deformations with the muscles. This application uses a five-layer pyramid decomposition plus exponentially decaying noise to mathematically accurately simulate this physical phenomenon. Local differential constraints embed the anatomical knowledge of FACS into the texture generation process, preventing the texture from moving arbitrarily but rather allowing it to move according to the direction of muscle fibers, achieving end-to-end joint optimization from high-frequency texture constraints to low-level dynamic evolution. This is something that traditional GAN ​​methods cannot achieve, because GANs operate directly in pixel space and lack explicit modeling of spatial frequencies and anatomical structures, thus effectively improving the accuracy and quality of the generated texture parameters.

[0062] In some optional implementations, the parameter generation model includes a shape model and an appearance model; step S205 includes the following steps: Get a predefined number of action units.

[0063] In this embodiment, the shape parameters are obtained first. (68 2D key points) This parameter is formed by sequentially stitching together the two-dimensional coordinates of 68 FACS key points (the first 68 elements are the x-coordinates of each key point, and the last 68 elements are the y-coordinates of each key point). It serves as the shape reference for frame 0, and the shape of all subsequent frames is superimposed and evolved frame by frame from this starting point.

[0064] At the same time, the system predefines 20 action units (AUs), each (i∈{1,2,…,20}) corresponds to an attention mask The mask length is 68, corresponding one-to-one with the number of keypoints. The value of each element in the mask represents the weight of the deformation influence of that AU on the corresponding keypoint. For example, the mask for AU6 (cheek lift) has a weight of approximately 0.9 on the cheekbone keypoint and approximately 0.1 on the forehead keypoint. These masks are not manually set, but are automatically learned during the pre-training phase through gradient backpropagation of the AU detector: the AU detector scores the activation level of each AU in the generated result, and the gradient of this score propagates back along the mask parameters, causing the mask to automatically converge to focus attention on the distribution of facial regions that actually participate in the movement of that AU. After loading, there are 20 masks. to With 20 standard shape deformation vectors to (each) , which represents the standard displacement of 68 key points when the i-th AU is activated alone, together constitute the driving template library for shape evolution.

[0065] Based on a preset frame-by-frame shape evolution calculation strategy, the shape model is used to process the action unit and the shape vector to obtain the corresponding shape parameters.

[0066] In this embodiment, for the t-th frame, the system first obtains the activation intensity scalar of each of the 20 AUs in that frame. , … These activation intensities are output by the upstream micro-expression timing control module. Subsequently, for each Activate its strength With the corresponding standard shape deformation vector Perform element-wise dot product to obtain the global deformation contribution of the AU in frame t. · Then contribute this global deformation to the attention mask of the AU. Perform element-wise dot product, i.e. · ⊙ This ensures that the deformation of the AU only affects the key point regions with high weights in the mask, while the deformation in low-weight regions is suppressed to near zero. Finally, the local deformation contributions of each of the 20 AUs after mask filtering are summed element-wise and superimposed onto the initial shape. The final shape parameters for frame t are obtained from the above. Therefore, the shape parameters of the t-th frame ∈R¹³ 6 This refers to the shape parameters output by the AAM branch (parameter generation model).

[0067] Based on the appearance model, the shape parameters and texture parameters are processed to generate corresponding appearance parameters.

[0068] In this embodiment, the specific implementation process of generating the shape parameters and texture parameters based on the appearance model to obtain the corresponding appearance parameters will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0069] The shape parameters and appearance parameters are coupled based on a preset spatial deformation function to obtain the corresponding generation parameters.

[0070] In this embodiment, the obtained shape parameters Compared with the appearance parameters obtained in step four The common input is the predefined spatial deformation function Warp. Warp uses... The displacement field of the 68 key points driven is geometrically guided, for The encoded appearance feature map undergoes pixel-by-pixel spatial warping, i.e., according to The displacement of each key point in the image is calculated using thin-plate spline interpolation (TPS) to determine the corresponding offset of all pixels. The appearance features are then spatially remapped according to shape changes to obtain... (Size 256×256×4), which is the appearance rendering result guided by shape, and serves as the generation parameter of the final output of the AAM branch, i.e., the target parameter.

[0071] The generated parameters are used as the target parameters.

[0072] This application obtains a predefined number of action units; then, based on a preset frame-by-frame shape evolution calculation strategy, it processes the action units and shape vectors using a shape model to obtain corresponding shape parameters; subsequently, it generates appearance parameters based on an appearance model, coupling the shape parameters and appearance parameters using a preset spatial deformation function to obtain corresponding generated parameters; finally, it uses the generated parameters as target parameters. Based on this processing flow, traditional AAM uses a set of global shape parameters to control the deformation of the entire face, which is too coarse for micro-expressions, which often only involve two or three small muscles. This application, through the use of an attention-guided mechanism, upgrades AAM from "global deformation" to "local precise deformation." Through the attention masks of 20 individual AUs, the system achieves the effect of "whichever AU moves, only the keypoints in the corresponding area move." Transformer appearance encoding further ensures that appearance changes (such as skin tone and gloss) are spatially aligned with shape changes, thus solving the problems of "shape changes but texture doesn't keep up" or "texture changes but shape is incorrect" in traditional 3DMM methods, improving the quality and accuracy of the generated target parameters.

[0073] In some alternative implementations, step S2053 includes the following steps: The shape parameters are encoded based on the position encoding module in the appearance model to obtain the corresponding shape features; wherein, the appearance model includes a position encoding module, a feature extraction module, a multi-head attention layer, and a mapping layer.

[0074] In this embodiment, the appearance model is a sub-network with the Transformer encoder at its core. The position encoding module, the pre-trained feature extraction network (i.e., the feature extraction module), the MultiHeadAttn (i.e., the multi-head attention layer), and the MLP mapping layer (i.e., the mapping layer) are all functional modules / components within the appearance model.

[0075] Among them, the shape parameters output by the shape model can be used. Send it to the position encoding module, and The coordinate information of 68 key points is mapped into a high-dimensional feature vector (i.e., shape feature), thus obtaining the shape position encoding. . It carries the spatial semantics of the shape changes of the current frame, that is, which regions are moving and how much they are moving.

[0076] Based on the feature extraction module, the texture parameters are extracted to obtain the corresponding target texture features.

[0077] In this embodiment, the texture parameters output by the dynamic texture branch (i.e., the pyramid decomposition network), i.e., the reconstructed texture of frame t, are used. (Dimensions 256×256×4) The image is fed into a pre-trained feature extraction network. After global feature aggregation of the entire image, its CLS token is extracted to obtain the target texture features. . It carries global semantics at the texture level of the current frame, namely appearance information such as skin color, gloss, and brightness.

[0078] The shape feature and the target texture feature are concatenated to obtain the corresponding concatenated feature.

[0079] In this embodiment, the shape features With target texture features Vector concatenation is performed along the feature dimension to obtain the concatenated features. This is the data integration step within the appearance model, preparing for subsequent attention calculations.

[0080] The splicing features are calculated and processed based on the multi-head attention layer to obtain the corresponding output data.

[0081] In this embodiment, the obtained spliced ​​features are fed into a multi-head attention layer (MultiHeadAttn). During training, each attention head in this layer is constrained to focus on the "expression core region" defined by FACS. For example, when AU12 is activated, the attention head automatically focuses on the levator palpebrae oris region, and when AU4 is activated, it automatically focuses on the glabella region. This means that computational resources for appearance changes are automatically allocated to the facial regions that are actually undergoing deformation, rather than being uniformly calculated across the entire face. The multi-head attention layer is the core computational unit of the appearance model. By receiving the spliced ​​feature vectors, each attention head is constrained to focus on the expression core region defined by FACS during training, achieving spatial adaptive allocation of computational resources and obtaining the corresponding attention output, i.e., output data.

[0082] The output data is nonlinearly transformed based on the mapping layer to obtain the corresponding transformed data.

[0083] In this embodiment, the output data of multi-head attention is nonlinearly transformed through an MLP mapping layer to finally obtain the appearance parameters of the t-th frame. .

[0084] The transformation data is used as the appearance parameter.

[0085] This application encodes shape parameters using a positional encoding module within an appearance model to obtain corresponding shape features. The appearance model includes a positional encoding module, a feature extraction module, a multi-head attention layer, and a mapping layer. Then, the feature extraction module extracts texture parameters to obtain corresponding target texture features. Next, the shape features and target texture features are concatenated to obtain concatenated features. Subsequently, the concatenated features are processed using the multi-head attention layer to obtain corresponding output data. Furthermore, the output data undergoes a nonlinear transformation using the mapping layer to obtain corresponding transformed data. Finally, the transformed data is used as appearance parameters. Based on the above processing flow, this application uses the appearance model and the Transformer multi-head attention mechanism to jointly encode shape parameters and texture parameters in the feature space. This allows the generated appearance parameter a_t to be subject to the dual constraints of "where it is moving" (shape guidance) and "how it is moving" (texture content). The attention head automatically focuses on the expression core area defined by FACS, thereby accurately allocating the computational resources for appearance changes to the facial regions where deformation actually occurs. This avoids the resource waste and spatial misalignment problems caused by the disconnect between shape and appearance and global uniform computation in traditional methods, effectively improving the quality and accuracy of the generated appearance parameters.

[0086] In some optional implementations of this embodiment, step S206 includes the following steps: Based on the fusion network, multi-layer texture feature extraction and motion kernel convolution processing are performed on the texture parameters to obtain the corresponding motion response map.

[0087] In this embodiment, the texture features output by the pyramid decomposition network are fed layer by layer into a pre-trained five-layer feature extraction network using a fusion network to obtain corresponding five-layer texture feature maps. The spatial resolution of each feature map decreases sequentially, while the semantic information is enhanced sequentially. For each feature map layer... ( =1 to 5), respectively perform the convolution operation Conv( This involves using learnable convolutional kernels to spatially filter the features of the layer and extract the local motion response of that layer. The results of the five convolutional layers are then summed element-wise along the channel dimension to obtain an aggregated motion response map.

[0088] Obtain the pre-trained motor kernels.

[0089] In this embodiment, the pre-trained motion kernel described above . The principal component matrix extracted by performing PCA principal component analysis on the optical flow field of a real micro-expression dataset encodes the motion direction patterns of typical micro-expressions (such as the oblique diffusion direction of the nasolabial folds and the vertical compression direction between the eyebrows). This matrix is ​​kept fixed and not updated throughout the training process to ensure that the motion patterns always conform to the statistical regularities of real micro-expressions.

[0090] The motion response map is multiplied element-wise with the motion kernel to obtain the corresponding gating weight map.

[0091] In this embodiment, the obtained aggregated motion response map and the pre-trained motion kernel Perform element-wise multiplication, and then feed the result into the Sigmoid function. The values ​​are mapped to the [0,1] interval to obtain a kinematic gating weight map of the same size as the image (256×256). ,Right now . The value of each pixel position indicates whether the pyramid decomposition network (value close to 1) or the parameter generation model (value close to 0) should be used at that position.

[0092] The texture parameters and the target parameters are then fused pixel-by-pixel according to the gated weight map to obtain the corresponding fused data.

[0093] In this embodiment, the texture parameters output by the texture branch (i.e., the pyramid decomposition network) are... The target parameters output by the AAM branch According to the obtained gating weight graph Perform pixel-by-pixel weighted fusion. Specifically, in At each pixel position (i,j), with (i,j) represents the weight. (i,j), with 1 (i,j) represents the weight. The final value of a pixel is obtained by weighted summing of (i,j) and (i,j). This can be expressed by the formula: Where ⊙ denotes element-wise multiplication. The physical meaning of this fusion is: in the area between the eyebrows where a frown is being made, the texture changes drastically while the shape changes relatively gently. Approaching 1, the synthesized result branches with texture. The design prioritizes preserving high-frequency details such as wrinkles; in areas where the cheeks are generally lifted but the skin does not exhibit obvious wrinkles, the shape changes are significant while the texture changes are minimal. When the result is close to 0, the synthesis result follows the AAM branch (i.e., the parameter generation model). This is the primary factor, ensuring the geometric consistency of the overall facial motion. This yields the final synthesized output for frame t. (Dimensions 256×256×4), thus obtaining the fused data.

[0094] The fused data is used as the synthesized sequence.

[0095] In this embodiment, the generated synthetic sequence can also be fed into an adversarial discriminator. The discriminator processes the synthetic sequence and outputs a realism score. This realism score is used as a feedback signal to propagate back to each branch of the generator, driving the texture branch and the AAM branch to optimize in tandem. This makes the fused synthetic frame continuously approach real micro-expression data in terms of visual realism, and finally outputs a high-quality synthetic data sequence with physiological rationality.

[0096] This application obtains a motion response map by performing multi-layer texture feature extraction and motion kernel convolution processing on texture features based on a fusion network; then, a pre-trained motion kernel is obtained; subsequently, the motion response map and the motion kernel are multiplied element-wise to obtain a corresponding gating weight map; next, the texture parameters and target parameters are weighted and fused pixel-wise according to the gating weight map to obtain the corresponding fused data; finally, the fused data is used as a synthetic sequence. Based on the above processing flow, this application uses a kinematic gating mechanism, which does not use a global weight, but rather gives each pixel its own weight, and this weight is jointly determined by five layers of texture features and is controlled by the motion kernel. Constraints. This means that the fusion process itself knows where texture should dominate and where shape should dominate, thus accurately achieving physically reasonable regional fusion. Furthermore, The non-updating design ensures that the motion pattern always conforms to the statistical regularity of real micro-expressions and will not be overfitted by the generator, thus effectively guaranteeing the quality of the generated synthetic sequence.

[0097] In some optional implementations of this embodiment, step S207 includes the following steps: The synthesized sequence is subjected to intensity normalization processing with respect to the action unit to obtain the corresponding first synthesized sequence.

[0098] In this embodiment, after the synthesized sequence is generated, the intensity value of each action unit (AU) is detected frame by frame using an AU detector. Let the target AU intensity vector be... (e.g., AU4=0.3, AU15=0.2), the actual intensity detected is .like and If a deviation exists, it is normalized and corrected according to FACS standards to obtain the corresponding first synthetic sequence. For example, if the detected AU4 intensity is 0.28 and the target is 0.3, it is automatically corrected to 0.3. The correction is based on the AU intensity-muscle contraction degree mapping table defined by FACS.

[0099] The first synthesized sequence is subjected to time-series reparameterization to obtain the corresponding second synthesized sequence.

[0100] In this embodiment, the typical temporal characteristics of micro-expressions are a "burst-like three-stage" pattern: rapid onset, brief peak, and rapid decay. The system can reparameterize the timeline of the first synthesized sequence to ensure that the AU intensity curve conforms to this pattern, thereby obtaining the corresponding second synthesized sequence. Specifically, the total duration τ is set to be controllable within the range of 40ms to 500ms, with frame generation in steps of 16.7ms (corresponding to 60fps). The time curve is constrained as follows: the intensity rises rapidly in the initial stage (approximately 20% of the total duration), the intensity remains near the target value in the peak stage (approximately 30% of the total duration), and the intensity drops rapidly to zero in the decay stage (approximately 50% of the total duration).

[0101] The second synthesized sequence is subjected to illumination consistency optimization processing to obtain the corresponding third synthesized sequence.

[0102] In this embodiment, by checking the lighting attributes of each frame of the entire second synthetic sequence, it is ensured that the skin's reflective properties (such as the location of highlight points and the intensity of diffuse reflection) remain consistent over time. If the lighting attributes of a certain frame are found to differ from those of the preceding and following frames by more than a threshold, histogram matching is used to align it to the lighting distribution of the adjacent frames to avoid artifacts such as "flickering" or "flickering," thereby obtaining the processed third synthetic sequence.

[0103] The third synthetic sequence is used as the target synthetic data.

[0104] This application obtains a first synthesized sequence by performing intensity normalization on the synthesized sequence with respect to action units; then, it performs temporal reparameterization on the first synthesized sequence to obtain a second synthesized sequence; subsequently, it performs illumination consistency optimization on the second synthesized sequence to obtain a third synthesized sequence; and finally, it uses the third synthesized sequence as the target synthesized data. Based on the above processing flow, the optimization process provided by this application is a key step in improving the quality from "looking like a micro-expression" to "conforming to the physiological definition of micro-expressions." The generated synthesized sequence may be visually realistic, but it may have deviations in AU intensity accuracy and temporal characteristics. This application ensures that the generated expression strictly conforms to the intensity level defined by FACS through AU intensity normalization; temporal reparameterization ensures that the time curve conforms to the explosive characteristics of micro-expressions, which are "fast-coming and fast-going," rather than slow changes; and illumination consistency optimization eliminates the illumination drift that may accumulate during frame-by-frame generation. These three calibrations together ensure that the output data is not only "realistic but also accurate," effectively improving the quality of the generated target synthesized data, which can be directly used to train a micro-expression recognition model.

[0105] In some alternative implementations, the user information obtained is subject to user consent and complies with relevant laws and policies.

[0106] Furthermore, any software tools or components not belonging to our company that appear in the embodiments of this application are merely illustrative examples and do not represent actual use.

[0107] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0108] It should be emphasized that, in order to further ensure the privacy and security of the aforementioned target synthetic data, the aforementioned target synthetic data can also be stored in a node of a blockchain.

[0109] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0110] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0111] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0112] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0113] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0114] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a data processing device based on artificial intelligence, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0115] like Figure 3 As shown, the artificial intelligence-based data processing device 300 described in this embodiment includes: a receiving module 301, a preprocessing module 302, a calling module 303, a first processing module 304, a second processing module 305, a fusion module 306, and an optimization module 307. Wherein: The receiving module 301 is used to receive input facial image data; The preprocessing module 302 is used to preprocess the facial image data based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector; Module 303 is used to invoke a preset synthesis processing model; wherein, the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; The first processing module 304 is used to process the texture map sequence based on the pyramid decomposition network to obtain the corresponding texture parameters; The second processing module 305 is used to process the shape vector and the texture parameters based on the parameter generation model to obtain the corresponding target parameters; The fusion module 306 is used to fuse the texture parameters and the target parameters based on the fusion network to obtain the corresponding synthesized sequence; The optimization module 307 is used to optimize the synthesized sequence based on a preset optimization strategy to obtain the corresponding target synthesized data.

[0116] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0117] In some optional implementations of this embodiment, the preprocessing module 302 includes: An expansion submodule is used to perform channel expansion on the facial image data to obtain the corresponding target image data; The detection submodule is used to perform key point detection on the target image data based on a preset feature extraction network to obtain the corresponding key point coordinates. The registration submodule is used to perform non-rigid registration processing on the coordinates of the key points based on a preset registration algorithm to obtain the corresponding deformation field; The mapping submodule is used to perform non-rigid deformation mapping on the target image data based on the deformation field to obtain the corresponding texture map sequence; The first acquisition submodule is used to acquire the two-dimensional coordinates of the registered key points on a preset standard topology; The splicing submodule is used to sequentially splice the two-dimensional coordinates to obtain the corresponding shape vector.

[0118] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0119] In some optional implementations of this embodiment, the pyramid decomposition network has multiple layers; the first processing module 304 includes: The decomposition submodule is used to decompose the texture map sequence layer by layer based on the pyramid decomposition network to obtain multiple independent texture features; The first processing submodule is used to initialize an independent recurrent neural network for each layer in the pyramid decomposition network, and perform corresponding temporal evolution processing on the texture parameters based on the recurrent neural network to obtain the corresponding latent variables. The reconstruction submodule is used to perform layer-by-layer upsampling texture reconstruction based on the latent variables to obtain the corresponding reconstruction results; The first determining submodule is used to use the reconstruction result as the texture parameter.

[0120] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0121] In some optional implementations of this embodiment, the parameter generation model includes a shape model and an appearance model; the second processing module 305 includes: The second acquisition submodule is used to acquire a predefined specified number of action units; The second processing submodule is used to process the action unit and the shape vector using the shape model based on a preset frame-by-frame shape evolution calculation strategy to obtain the corresponding shape parameters. A generation submodule is used to perform parameter generation processing on the shape parameters and texture parameters based on the appearance model to obtain the corresponding appearance parameters; The coupling submodule is used to couple the shape parameters and the appearance parameters based on a preset spatial deformation function to obtain the corresponding generation parameters; The second determining submodule is used to use the generated parameters as the target parameters.

[0122] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0123] In some optional implementations of this embodiment, the generation submodule includes: An encoding unit is used to encode the shape parameters based on the position encoding module in the appearance model to obtain the corresponding shape features; wherein, the appearance model includes a position encoding module, a feature extraction module, a multi-head attention layer, and a mapping layer; An extraction unit is used to extract features from the texture parameters based on the feature extraction module to obtain the corresponding target texture features; The splicing unit is used to perform splicing operations on the shape feature and the target texture feature to obtain the corresponding spliced ​​feature; The computing unit is used to perform calculations on the splicing features based on the multi-head attention layer to obtain the corresponding output data; A transformation unit is used to perform a nonlinear transformation on the output data based on the mapping layer to obtain corresponding transformed data; A determining unit is used to use the transformation data as the appearance parameter.

[0124] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0125] In some optional implementations of this embodiment, the fusion module 306 includes: The third processing submodule is used to perform multi-layer texture feature extraction and motion kernel convolution processing on the texture parameters based on the fusion network to obtain the corresponding motion response map. The third acquisition submodule is used to acquire pre-trained motion kernels; The multiplication submodule is used to perform element-wise multiplication of the motion response graph and the motion kernel to obtain the corresponding gated weight graph; The fusion submodule is used to perform pixel-by-pixel weighted fusion processing on the texture parameters and the target parameters according to the gated weight map to obtain the corresponding fused data. The third determining submodule is used to use the fused data as the synthesized sequence.

[0126] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0127] In some optional implementations of this embodiment, the optimization module 307 includes: The fourth processing submodule is used to perform intensity normalization processing on the synthesized sequence with respect to the action unit to obtain the corresponding first synthesized sequence; The fifth processing submodule is used to perform time-series reparameterization processing on the first synthesized sequence to obtain the corresponding second synthesized sequence; The sixth processing submodule is used to perform illumination consistency optimization processing on the second synthesized sequence to obtain the corresponding third synthesized sequence; The fourth determining submodule is used to use the third synthetic sequence as the target synthetic data.

[0128] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the artificial intelligence-based data processing method in the aforementioned embodiments, and will not be repeated here.

[0129] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0130] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0131] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0132] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for data processing methods based on artificial intelligence. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0133] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the artificial intelligence-based data processing method.

[0134] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0135] Compared with the prior art, the embodiments of this application have the following beneficial effects: In this embodiment, an intelligent data processing method is provided. A pyramid decomposition network is used to hierarchically deconstruct the texture map sequence according to spatial frequency, allowing texture parameters to be independently represented in the low-frequency global tone layer and the high-frequency skin micro-detail layer. This avoids the distortion of wrinkle propagation direction and spatial distribution misalignment caused by the overall mixing of texture dynamics in traditional 3DMM methods. Furthermore, a parameter generation model is used to jointly encode the shape vector and the layered texture parameters, generating target parameters that are strictly aligned with shape changes in spatial distribution. This ensures that texture dynamics evolve synchronously with shape changes in the correct facial region, solving the core defects of existing synthetic data such as the disconnect between shape and texture changes and the mismatch between the spatial distribution of skin wrinkles and the direction of real facial muscle fibers. Finally, a fusion network performs pixel-level fusion of the layered texture parameters and target parameters, and an optimization strategy is used to optimize the synthetic sequence to output the target synthetic data. This method, from texture decomposition and joint parameter generation to fusion optimization, systematically improves the spatial frequency fidelity and shape-texture alignment accuracy of texture dynamics in synthetic data, thereby effectively improving the quality of generated synthetic data and providing high-fidelity training data for micro-expression recognition models.

[0136] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the artificial intelligence-based data processing method described above.

[0137] Compared with the prior art, the embodiments of this application have the following main advantages: In this embodiment, an intelligent data processing method is provided. A pyramid decomposition network is used to hierarchically deconstruct the texture map sequence according to spatial frequency, allowing texture parameters to be independently represented in the low-frequency global tone layer and the high-frequency skin micro-detail layer. This avoids the distortion of wrinkle propagation direction and spatial distribution misalignment caused by the overall mixing of texture dynamics in traditional 3DMM methods. Furthermore, a parameter generation model is used to jointly encode the shape vector and the layered texture parameters, generating target parameters that are strictly aligned with shape changes in spatial distribution. This ensures that texture dynamics evolve synchronously with shape changes in the correct facial region, solving the core defects of existing synthetic data such as the disconnect between shape and texture changes and the mismatch between the spatial distribution of skin wrinkles and the direction of real facial muscle fibers. Finally, a fusion network performs pixel-level fusion of the layered texture parameters and target parameters, and an optimization strategy is used to optimize the synthetic sequence to output the target synthetic data. This method, from texture decomposition and joint parameter generation to fusion optimization, systematically improves the spatial frequency fidelity and shape-texture alignment accuracy of texture dynamics in synthetic data, thereby effectively improving the quality of generated synthetic data and providing high-fidelity training data for micro-expression recognition models.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0139] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

[0140] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

Claims

1. A data processing method based on artificial intelligence, characterized in that, Includes the following steps: Receive input facial image data; The facial image data is preprocessed based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector; A preset synthesis processing model is invoked; wherein, the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; The texture map sequence is processed based on the pyramid decomposition network to obtain the corresponding texture parameters; Based on the parameter generation model, the shape vector and the texture parameters are processed to obtain the corresponding target parameters; The texture parameters and the target parameters are fused based on the fusion network to obtain the corresponding synthesized sequence; The synthesized sequence is optimized based on a preset optimization strategy to obtain the corresponding target synthesized data.

2. The data processing method based on artificial intelligence according to claim 1, characterized in that, The step of preprocessing the facial image data based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector specifically includes: Channel expansion is performed on the facial image data to obtain the corresponding target image data; Based on a preset feature extraction network, key point detection is performed on the target image data to obtain the corresponding key point coordinates; Based on a preset registration algorithm, non-rigid registration processing is performed on the coordinates of the key points to obtain the corresponding deformation field; Based on the deformation field, the target image data is subjected to non-rigid deformation mapping to obtain the corresponding texture map sequence; Obtain the two-dimensional coordinates of the registered key points on the preset standard topology; The two-dimensional coordinates are sequentially concatenated to obtain the corresponding shape vector.

3. The data processing method based on artificial intelligence according to claim 1, characterized in that, The pyramid decomposition network has multiple layers; the step of processing the texture map sequence based on the pyramid decomposition network to obtain the corresponding texture parameters specifically includes: The texture map sequence is decomposed layer by layer based on the pyramid decomposition network to obtain multiple independent texture features; An independent recurrent neural network is initialized for each layer in the pyramid decomposition network, and the texture parameters are subjected to corresponding temporal evolution processing based on the recurrent neural network to obtain the corresponding latent variables; Based on the latent variables, a layer-by-layer upsampling texture reconstruction process is performed to obtain the corresponding reconstruction result; The reconstruction result is used as the texture parameter.

4. The data processing method based on artificial intelligence according to claim 1, characterized in that, The parameter generation model includes a shape model and an appearance model; the step of processing the shape vector and the texture parameters based on the parameter generation model to obtain the corresponding target parameters specifically includes: Get a predefined specified number of action units; Based on a preset frame-by-frame shape evolution calculation strategy, the shape model is used to process the action unit and the shape vector to obtain the corresponding shape parameters; Based on the appearance model, the shape parameters and texture parameters are processed to generate corresponding appearance parameters. The shape parameters and appearance parameters are coupled based on a preset spatial deformation function to obtain the corresponding generation parameters; The generated parameters are used as the target parameters.

5. The data processing method based on artificial intelligence according to claim 4, characterized in that, The step of generating parameters for the shape parameters and texture parameters based on the appearance model to obtain the corresponding appearance parameters specifically includes: The shape parameters are encoded based on the position encoding module in the appearance model to obtain the corresponding shape features; wherein, the appearance model includes a position encoding module, a feature extraction module, a multi-head attention layer, and a mapping layer; Based on the feature extraction module, the texture parameters are extracted to obtain the corresponding target texture features; The shape feature and the target texture feature are concatenated to obtain the corresponding concatenated feature; The splicing features are calculated and processed based on the multi-head attention layer to obtain the corresponding output data; The output data is nonlinearly transformed based on the mapping layer to obtain the corresponding transformed data. The transformation data is used as the appearance parameter.

6. The data processing method based on artificial intelligence according to claim 1, characterized in that, The step of fusing the texture parameters and the target parameters based on the fusion network to obtain the corresponding synthesized sequence specifically includes: Based on the fusion network, multi-layer texture feature extraction and motion kernel convolution processing are performed on the texture parameters to obtain the corresponding motion response map; Obtain the pre-trained motor kernels; The motion response map is multiplied element-wise with the motion kernel to obtain the corresponding gating weight map; The texture parameters and the target parameters are fused pixel-by-pixel according to the gated weight map to obtain the corresponding fused data. The fused data is used as the synthesized sequence.

7. The data processing method based on artificial intelligence according to claim 1, characterized in that, The step of optimizing the synthesized sequence based on a preset optimization strategy to obtain the corresponding target synthesized data specifically includes: The synthesized sequence is subjected to intensity normalization processing with respect to the action unit to obtain the corresponding first synthesized sequence; The first synthesized sequence is subjected to time-series reparameterization to obtain the corresponding second synthesized sequence; The second synthesized sequence is subjected to illumination consistency optimization processing to obtain the corresponding third synthesized sequence; The third synthetic sequence is used as the target synthetic data.

8. A data processing device based on artificial intelligence, characterized in that, include: A receiving module is used to receive input facial image data; The preprocessing module is used to preprocess the facial image data based on a preset processing strategy to obtain the corresponding texture map sequence and shape vector; The calling module is used to call a preset synthesis processing model; wherein, the synthesis processing model includes a pyramid decomposition network, a parameter generation model, and a fusion network; The first processing module is used to process the texture map sequence based on the pyramid decomposition network to obtain the corresponding texture parameters; The second processing module is used to process the shape vector and the texture parameters based on the parameter generation model to obtain the corresponding target parameters; The fusion module is used to fuse the texture parameters and the target parameters based on the fusion network to obtain the corresponding synthesized sequence; The optimization module is used to optimize the synthesized sequence based on a preset optimization strategy to obtain the corresponding target synthesized data.

9. A computer device, characterized in that, It includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the data processing method based on artificial intelligence as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the data processing method based on artificial intelligence as described in any one of claims 1 to 7.

Images