Image generation method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to an image generation method, which comprises the following steps: acquiring text description input by a user; converting the text description into a semantic vector based on a pre-trained language model; performing posture estimation processing on the text description based on a pre-trained posture predictor to obtain an initial posture parameter; performing parameter optimization on the initial posture parameter based on a preset biomechanical constraint module and a posture condition diffusion module to obtain a target posture parameter; converting the target posture parameter into a corresponding posture heat map; processing the semantic vector and the posture heat map based on a preset image generator to obtain a target image; and performing output processing on the target image. The invention further provides an image generation device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the target image can be stored in a block chain. The method and the device can be applied to text map scenes in the financial field and the medical field, and the accuracy and the reliability of image generation are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence technology, which can be applied in the fields of financial technology, digital medical treatment, etc., and in particular to an image generation method and device, a computer device and a storage medium. BACKGROUND

[0002] In traditional text-to-image generation technology, early methods mainly rely on conditional generative adversarial networks (GANs) and variational autoencoders (VAEs) and other models to generate images of specific poses through pose keypoint annotations. However, these methods have significant defects when dealing with complex pose generation tasks. Due to the lack of deep consideration of pose kinematics and physical laws, the generated images often have kinematic distortion or physical irrationality, such as joint angles exceeding the physiological limits of the human body, imbalance of the body's center of gravity, etc., resulting in low accuracy and reliability of the generated images, and making it difficult to meet the demand for high-quality images in actual applications. Specifically, traditional methods often only focus on the position information of pose key points, ignoring the motion constraints between joints and the balance of the whole body. This simple and rough processing method cannot effectively capture the internal laws of complex poses, resulting in a significant discount in visual effect and physical rationality of the generated images.

[0003] This deficiency is also reflected in the intelligent financial advisor image display scenario in the financial field. In intelligent financial advisor services, in order to more intuitively show customers the expected effect of different investment strategies, it is necessary to convert strategy information into visual images. Traditional image generation methods, when generating images to display complex poses (here, "pose" can be analogous to complex data display forms) such as investment strategy yield curves and asset allocation proportions, cannot accurately grasp the logical relationship between data and display rules, which may result in data presentation disorder and proportion imbalance in the images, making it difficult for customers to accurately understand the core points of the investment strategy and reducing the effectiveness of the intelligent financial advisor service and customer experience. For example, when displaying complex investment portfolio strategies involving multiple asset classes, different investment horizons, and risk levels, the images generated by traditional methods may not clearly present the correlation between assets and the overall yield and risk characteristics, misleading customers into making incorrect investment decisions. This image generation bias not only affects customers' trust in intelligent financial advisor services, but also may hinder the application and development of financial technology in the investment field.

[0004] Therefore, there is an urgent need to provide a text-to-image generation technology that can accurately control poses and comply with physical laws to improve the quality and practicality of image generation and promote technological progress and service upgrades in related fields. SUMMARY

[0005] The embodiment of the application aims to provide an image generation method and device, computer equipment and a storage medium, to solve the technical problem that the existing text-to-image generation technology based on conditional GANs (Generative Adversarial Networks) and VAEs (Variational Autoencoder) and other models often generates images with kinematic distortion or physical irrationality, resulting in low accuracy and reliability of the generated images.

[0006] In a first aspect, an image generation method is provided, comprising:

[0007] obtaining a text description input by a user;

[0008] converting the text description into a corresponding semantic vector based on a pre-trained language model;

[0009] performing pose estimation processing on the text description based on a pre-trained pose predictor to obtain corresponding initial pose parameters;

[0010] performing parameter optimization on the initial pose parameters based on a preset biomechanics constraint module and a pose condition diffusion module to obtain corresponding target pose parameters;

[0011] converting the target pose parameters into a corresponding pose heat map;

[0012] processing the semantic vector and the pose heat map based on a preset image generator to obtain a corresponding target image;

[0013] performing output processing on the target image.

[0014] In a second aspect, an image generation device is provided, comprising:

[0015] an obtaining module configured to obtain a text description input by a user;

[0016] a first conversion module configured to convert the text description into a corresponding semantic vector based on a pre-trained language model;

[0017] a first processing module configured to perform pose estimation processing on the text description based on a pre-trained pose predictor to obtain corresponding initial pose parameters;

[0018] an optimization module configured to perform parameter optimization on the initial pose parameters based on a preset biomechanics constraint module and a pose condition diffusion module to obtain corresponding target pose parameters;

[0019] a second conversion module configured to convert the target pose parameters into a corresponding pose heat map;

[0020] a second processing module configured to process the semantic vector and the pose heat map based on a preset image generator to obtain a corresponding target image.

[0021] an output module configured to perform output processing on the target image.

[0022] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the image generation method when executing the computer program.

[0023] In a fourth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program implements the steps of the image generation method when executed by a processor.

[0024] In the above-mentioned image generation method, device, computer device, and storage medium, the text description input by the user is first obtained, and then the text description is converted into a corresponding semantic vector based on a pre-trained language model, and the text description is subjected to pose estimation processing based on a pre-trained pose predictor to obtain corresponding initial pose parameters. Then, the initial pose parameters are subjected to parameter optimization based on a pre-set biomechanics constraint module and a pose condition diffusion module to obtain corresponding target pose parameters. Subsequently, the target pose parameters are converted into a corresponding pose heat map. Further, the semantic vector and the pose heat map are processed based on a pre-set image generator to obtain a corresponding target image. Finally, the target image is subjected to output processing. In this application, the text description input by the user is first obtained, and then the text description is converted into a semantic vector based on the use of a pre-trained language model, and the text description is subjected to pose estimation processing based on the use of a pre-trained pose predictor to obtain initial pose parameters. Then, the initial pose parameters are subjected to parameter optimization based on the use of a biomechanics constraint module and a pose condition diffusion module to obtain target pose parameters, and the target pose parameters are converted into a pose heat map. Subsequently, the semantic vector and the pose heat map are processed based on the use of an image generator to obtain a target image and output. Through the above image generation process, this application deeply integrates the biomechanics principle into the generation process of the image generator based on the use of the biomechanics constraint module, and through the synergistic effect of a series of steps such as text conversion, initial pose estimation, initial word number parameter optimization based on the biomechanics constraint module and the pose condition diffusion module, heat map generation, and image generation, the common problem of not conforming to physical laws in pose control can be systematically solved, the pose controllability of the generated image can be greatly improved, and a high-quality target image that conforms to physical laws and has high matching degree with the text description can be efficiently and accurately generated, thereby improving the accuracy and reliability of the generated image. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the solutions in the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0026] Figure 1 is an exemplary system architecture diagram to which the present application can be applied;

[0027] Figure 2 is a flow chart of one embodiment of the image generation method according to the present application;

[0028] Figure 3 is a structural schematic diagram of one embodiment of the image generation apparatus according to the present application;

[0029] Figure 4 is a structural schematic diagram of one embodiment of the computer device according to the present application. DETAILED DESCRIPTION

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs; the terminology used in the description of the specification of the present application is only for the purpose of describing specific embodiments and is not intended to limit the present application; the terms "include" and "have" and any variations thereof in the specification of the present application and the claims and the above description of drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the specification and claims of the present application and the above description of drawings are used to distinguish different objects, not to describe a particular order.

[0031] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings.

[0033] As Figure 1As shown, the system architecture 100 can include a terminal device 101, a network 102 and a server 103. The terminal device 101 can be a notebook computer 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0034] A user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0035] The terminal device 101 can be various electronic devices with display screens and supporting web browsing, in addition to the notebook computer 1011, the tablet computer 1012 or the mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer and a desktop computer, etc.

[0036] The server 103 can be a server providing various services, such as a background server supporting a page displayed on the terminal device 101.

[0037] It should be noted that the image generation method provided by the embodiments of the present application is generally executed by a server / terminal device, and accordingly, the image generation apparatus is generally arranged in a server / terminal device.

[0038] It should be understood that Figure 1 The number of terminal devices, networks and servers in

[0039] With reference to Figure 2 , a flowchart of one embodiment of the image generation method according to the present application is shown. The order of the steps in the flowchart can be changed, and some steps can be omitted according to different needs. The image generation method provided by the embodiments of the present application can be applied to any scene requiring image generation, and then the image generation method can be applied to products in these scenes, such as text-to-image scenes in the financial field and the medical field. The image generation method includes the following steps:

[0040] Step S201, obtaining the text description input by the user.

[0041] In this embodiment, the electronic device (for example, the server / terminal device shown in Figure 1 The server / terminal device shown in FIG. 1 can obtain the text description input by the user through a wired connection or a wireless connection. It should be noted that the wireless connection can include, but is not limited to, 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other now known or future developed wireless connection methods. The execution subject of the present application is an image generation system, which can be referred to as a system. The text description can be a text description information matched with the image to be generated according to the actual needs of the user.

[0042] The system mainly consists of five core components: 1. Text encoder (i.e. language model): a pre-trained language model is used to convert the input text into semantic latent code (semantic vector). 2. Biomechanical constraint module (BCM) and pose conditional diffusion model (PCDP) corresponding to the pose conditioning diffusion process. 3. Heat map generation network: convert the optimized pose parameters into spatial heat map representation. 4. Image generator: based on the improved U-Net architecture, fuse the text latent code and pose heat map to generate the final image. The workflow of the system is as follows: the text description input by the user is first encoded into a semantic vector, and an initial pose estimate is generated by a pre-trained pose predictor. The pose estimate is then optimized in the PCDP with physical constraints, and the BCM calculates biomechanical parameters such as joint range of motion and balance constraints in real time during this process. The optimized pose parameters are converted into a spatial representation by the heat map generation network, and finally input into the image generator together with the text latent code to generate a target image that conforms to the physical laws. In addition, the system adopts a three-stage training process: 1. BCM pre-training: train the constraint predictor on motion capture datasets such as AMASS 2. PCDP fine-tuning: use LAION-5B and other text-image pairs to jointly optimize the diffusion process 3. Adversarial training: introduce a physical discriminator to improve the realism of the generated pose.

[0043] The present application can be applied to the business scenarios of text-to-image in the fields of finance and insurance, medical rehabilitation, etc. For example, in the field of finance and insurance, the user input text description can be a text corresponding to the insurance scheme customization requirement, including: content: "35-year-old male, enterprise employee, annual income 200,000, married and has a son, family has a 1 million housing loan, daily commute mainly by self-driving, hopes to customize a comprehensive insurance scheme covering critical illness, accident, medical and life insurance, focusing on critical illness guarantee limit to cover housing loan and family living expenses for the next 5 years, accident insurance to include high-risk sports protection, medical insurance to hope to have special medical needs and overseas medical reimbursement, life insurance to guarantee the family to maintain the current living standard for more than 10 years after losing the main economic source." Explanation: This text description details the user's basic information (age, occupation, income, family status, etc.), as well as the overall requirement for the insurance scheme, including the type of insurance (critical illness, accident, medical, life insurance) and the specific protection requirements of each type of insurance, which helps the insurance planner to customize a suitable insurance scheme for the user.

[0044] In the field of medical rehabilitation, the user input text description can be a text corresponding to the postoperative rehabilitation training requirement, including: content: "Patient female, 52 years old, underwent arthroscopic surgery for meniscus injury of the knee joint, currently 2 weeks postoperatively, wound healing is good, but the knee joint range of motion is limited, cannot bend and straighten normally, and still has pain when standing and walking. Hope to develop a 3-month rehabilitation training plan, the goal is to gradually restore the normal range of motion of the knee joint, strengthen the leg muscle strength, improve the walking ability, and ultimately restore the basic activities in normal life and work." Explanation: This text description contains the patient's basic information (age, gender), surgical information (arthroscopic surgery for meniscus injury of the knee joint, postoperative time), current physical condition (knee joint range of motion limited, pain), and clear rehabilitation training goals (restore range of motion, strengthen strength, improve walking ability, restore normal activities), which facilitates the rehabilitation therapist to design a personalized rehabilitation training plan for the patient.

[0045] Step S202, converting the text description into a corresponding semantic vector based on the pre-trained language model.

[0046] In this embodiment, the selection of the above-mentioned pre-trained language model is not specifically limited, for example, the CLIP model can be used. The input text description can be converted into a semantic latent code (semantic vector) according to the selected language model. Let the input text be T, and the semantic vector obtained after processing by the language model, this process can be simply represented as: text = TextEncoder(T), where TextEncoder represents the text encoder function of the language model, text contains the semantic information of the input text description, which will be used to guide image generation subsequently.

[0047] In step S203, the pose estimation process is performed on the text description based on the pre-trained pose predictor to obtain initial pose parameters.

[0048] In this embodiment, the selection of the pose predictor is not specifically limited, and a general model with pose estimation processing function can be used. The selected pose predictor can be used to perform initial estimation on the pose corresponding to the input text description, and output the preliminary estimated pose parameters, i.e., the initial pose parameters. Wherein, let the pose predictor be P, then the initial pose estimation can be represented as: p = P(T), wherein p is the pose parameter preliminarily estimated according to the text description, which provides a basis for subsequent pose optimization.

[0049] In step S204, the initial pose parameters are optimized based on the preset biomechanical constraint module and the pose condition diffusion module to obtain corresponding target pose parameters.

[0050] In this embodiment, the specific implementation process of optimizing the initial pose parameters based on the preset biomechanical constraint module and the pose condition diffusion module to obtain the corresponding target pose parameters will be further described in detail in the subsequent specific embodiments, and will not be described in detail here.

[0051] In step S205, the target pose parameters are converted into corresponding pose heat maps.

[0052] In this embodiment, the specific implementation process of converting the target pose parameters into corresponding pose heat maps will be further described in detail in the subsequent specific embodiments, and will not be described in detail here.

[0053] In step S206, the semantic vector and the pose heat map are processed based on the preset image generator to obtain a corresponding target image.

[0054] In this embodiment, the specific implementation process of processing the semantic vector and the pose heat map based on the preset image generator to obtain a corresponding target image will be further described in detail in the subsequent specific embodiments, and will not be described in detail here.

[0055] In step S207, the target image is output processed.

[0056] In this embodiment, the specific implementation process of output processing the target image will be further described in detail in the subsequent specific embodiments, and will not be described in detail here.

[0057] The application first acquires a text description input by a user; then converts the text description into a corresponding semantic vector based on a pre-trained language model; and performs pose estimation processing on the text description based on a pre-trained pose predictor to obtain corresponding initial pose parameters; then performs parameter optimization on the initial pose parameters based on a pre-set biomechanics constraint module and a pose condition diffusion module to obtain corresponding target pose parameters; subsequently converts the target pose parameters into a corresponding pose heat map; further processes the semantic vector and the pose heat map based on a pre-set image generator to obtain a corresponding target image; and finally performs output processing on the target image. The application converts a text description into a semantic vector based on the use of a pre-trained language model, and obtains initial pose parameters by performing pose estimation processing on the text description based on the use of a pre-trained pose predictor, then performs parameter optimization on the initial pose parameters based on the use of a biomechanics constraint module and a pose condition diffusion module to obtain target pose parameters, and converts the target pose parameters into a pose heat map, subsequently processes the semantic vector and the pose heat map based on the use of an image generator to obtain a target image and outputs the target image. Through the above image generation processing process, the application deeply integrates the biomechanics principle into the generation process of the image generator based on the use of the biomechanics constraint module, and through the synergistic effect of a series of steps such as text conversion, initial pose estimation, initial word parameter optimization based on the biomechanics constraint module and the pose condition diffusion module, heat map generation, and image generation, it can systematically solve the common problems in posture control that do not conform to physical laws, can greatly improve the posture controllability of the generated image, and can efficiently and accurately generate a high-quality target image that conforms to physical laws and has high matching degree with the text description, thereby improving the accuracy and reliability of the generated image.

[0058] In some optional implementations, step S204 includes the following steps:

[0059] Perform biomechanics analysis and calculation processing on the initial pose parameters based on the biomechanics constraint function module to obtain corresponding physical condition information.

[0060] In the embodiment, the specific implementation process of performing biomechanics analysis and calculation processing on the initial pose parameters based on the biomechanics constraint function module to obtain corresponding physical condition information will be further described in detail in subsequent specific embodiments, and will not be described in detail here.

[0061] Inject the semantic vector and the physical condition information into the pose condition diffusion module.

[0062] In the embodiment, the semantic vector and the physical condition information are fused and injected into the denoising process in the pose condition diffusion module based on a double-condition injection mechanism, so that the generated image not only conforms to the text description but also meets the physical law.

[0063] The pose condition diffusion module is used to perform physical constraint optimization processing corresponding to the initial pose parameter, and a processed parameter after optimization is obtained.

[0064] In the embodiment, the pose condition diffusion process of the pose condition diffusion module (which can be referred to as PCDP) improves the reverse process of the standard diffusion model, and the denoising step is represented as:

[0065] z t-1 =μ θ (z t ,t,c text ,c physics )+σ t ∈

[0066] wherein z t is a latent variable at the t-th step, representing the image feature state in the current diffusion process, t represents a time step in the diffusion process, c text is a semantic vector, used to guide the generation of an image conforming to the text description, c physics is physical condition information, calculated by a biomechanics constraint module (BCM), containing biomechanical parameter information such as joint range of motion and balance constraint, to ensure that the generated pose conforms to the physical law; μ θ is a neural network model with parameters θ, used to predict the mean of the latent variable after denoising according to the current latent variable z t , the time step t, the semantic vector c text and the physical condition information c physics . σ t is the noise intensity at the t-th step, and ∈ is random noise sampled from a standard normal distribution.

[0067] The innovations of the pose condition diffusion module include: 1. Double-condition injection mechanism: simultaneously fusing text conditions and physical conditions. 2. Constraint-aware sampling: implementing physical constraints through projected gradient descent in each denoising iteration:

[0068]

[0069] wherein z t ′ is the latent variable after constraint-aware sampling, is a set of physical constraints, is a constraint loss calculated by the BCM, is a projection function that projects the latent variable to satisfy the set of physical constraints η is the learning rate, which controls the step size of the parameter update, is the physical constraint loss The gradient of the latent variable z t is calculated by the BCM, and the biomechanical parameter information calculated by the BCM participates in the calculation. The gap between the current latent variable z t and the corresponding pose and physical constraints is measured, and by calculating its gradient and updating the latent variable, the latent variable changes in the direction of satisfying the physical constraints.

[0070] Adaptive noise scheduling: dynamically adjust the noise intensity according to the degree of violation of the physical constraints:

[0071]

[0072] where σ t ′ is the adjusted noise intensity, and α is a hyperparameter that controls the amplitude of noise intensity adjustment, is the norm of the physical constraint loss gradient, reflecting the degree of violation of the physical constraints. When the degree of violation is large, appropriately increasing the noise intensity helps the model to jump out of the local optimum and better satisfy the physical constraints.

[0073] Specifically, the process of performing physical constraint optimization processing on the initial pose parameters based on the pose condition diffusion module includes: double condition injection: injecting the semantic vector of the text description and the physical condition information calculated by the BCM into the denoising process of the PCDP. Constraint-aware sampling: in each denoising iteration, implement the physical constraints by the projected gradient descent method. Assuming that the pose corresponding to the current latent variable exceeds the reasonable range of the knee joint angle, the corresponding gradient will be calculated to guide the update of the latent variable z t in the direction of reducing the knee joint angle exceeding the range, and the projection function ensures that the updated latent variable satisfies the physical constraint set Adaptive noise scheduling: dynamically adjust the noise intensity according to the degree of violation of the physical constraints. If the current pose violates the physical constraints in terms of balance to a large extent, then σ t ′ will be increased accordingly, helping the model to better explore the pose space that satisfies the balance constraints. After multiple iterations of optimization, the optimized pose parameters are obtained and used as the above processing parameters to be used as the final target pose parameters.

[0074] The processing parameters are used as the target pose parameters.

[0075] In this embodiment, after multiple iterations of PCDP denoising and real-time biomechanical parameter calculation and constraint provision of BCM, the potential variables are gradually optimized, and the posture parameters corresponding to the final potential variables are the optimized posture parameters. These optimized posture parameters meet the requirements of the text description and the constraints of the physical laws, such as joint angles within a reasonable range, body balance, muscle coordination consistent with physiological principles, etc.

[0076] The application obtains physical condition information by performing biomechanical analysis and calculation processing on the initial posture parameters based on the biomechanical constraint function module; then injects the semantic vector and the physical condition information into the posture condition diffusion module; and then performs physical constraint optimization processing corresponding to the initial posture parameters based on the posture condition diffusion module to obtain optimized processing parameters; and subsequently uses the processing parameters as the target posture parameters. The application obtains physical condition information by performing biomechanical analysis and calculation processing on the initial posture parameters based on the use of the biomechanical constraint function module, then injects the semantic vector and the physical condition information into the posture condition diffusion module, and then performs physical constraint optimization processing corresponding to the initial posture parameters based on the use of the posture condition diffusion module, so that efficient and accurate parameter optimization processing of the initial posture parameters can be achieved. The application performs parameter optimization of the initial posture parameters based on the denoising iteration of the posture condition diffusion module and the real-time biomechanical parameter calculation and constraint provision of the biomechanical constraint function module, so as to generate optimized posture parameters that meet the physical laws. This combination effectively solves the key problems of posture distortion and physical unreasonableness in traditional text-to-image generation, ensures that the optimized posture parameters meet the requirements of the text description and the constraints of the physical laws, and guarantees the accuracy and compliance of the target posture parameters.

[0077] In some optional implementations of this embodiment, the biomechanical analysis and calculation processing on the initial posture parameters based on the biomechanical constraint function module to obtain corresponding physical condition information includes the following steps:

[0078] Perform joint range of motion analysis on the initial posture parameters based on the biomechanical constraint module to generate corresponding first biomechanical parameter information.

[0079] In this embodiment, the above-mentioned biomechanical constraint module (which can be referred to as BCM) adopts a hierarchical graph neural network architecture, which is mathematically expressed as:

[0080]

[0081] wherein, represents the hidden state of joint i in the lth layer, which contains the state information of joint i in the current layer; rij is the bone vector, which represents the physical connection from joint i to j, and describes the spatial relationship between joints; τ ij It is the joint torque calculated by the embedded muscle model, which reflects the muscle force exerted on the joint during movement; Represents a vector concatenation operation, which is used to concatenate information from different sources as the input of a multi-layer perceptron (MLP).

[0082] The biomechanical constraint module implements constraint propagation through multi-level message passing. Each layer of the network contains joint constraint units, balance assessment units, and muscle coordination units. Specifically:

[0083] 1. Joint constraint unit: A differentiable projection function is used to ensure that the joint angle θ satisfies:

[0084]

[0085] in, is the neutral position of the joint, Δ j The range of motion is anatomical; in addition, the joint restriction unit ensures that the generated posture conforms to the principles of human kinematics by limiting the joint angles within a reasonable range.

[0086] 2. Balance Assessment Unit: Calculates center of gravity projection and ground reaction force using a simplified rigid body dynamics model:

[0087]

[0088] Where M is the total mass of the body, c is the coordinate of the center of gravity, and g is the acceleration of gravity. In addition, the balance assessment unit assesses the balance state of the body to ensure that the generated posture does not cause the body to become unbalanced.

[0089] 3. Muscle synergy unit: This simulates the constraints imposed by muscle activation patterns on joint motion, preventing physiologically impossible muscle synergies. The muscle synergy unit is used to ensure that the generated poses are reasonable in terms of muscle motion.

[0090] The joint range of motion analysis of the initial posture parameters can be performed based on the joint restriction unit, and the calculated joint angle restriction information can be used as the first biomechanical parameter information.

[0091] Performing balance estimation on the initial posture parameters to generate corresponding second biomechanical parameter information.

[0092] In this embodiment, the balance estimation of the initial posture parameters can be performed based on the use of the balance assessment unit, and the calculated balance state information can be used as the second biomechanical parameter information.

[0093] The initial posture parameter is subjected to muscle synergy constraint to generate corresponding third biomechanics parameter information.

[0094] In the embodiment, the initial posture parameter can be subjected to muscle synergy constraint based on the use of the muscle synergy unit, and the calculated muscle synergy constraint information is taken as the third biomechanics parameter information.

[0095] The first biomechanics parameter information, the second biomechanics parameter information and the third biomechanics parameter information are subjected to information integration to obtain the physical condition information.

[0096] In the embodiment, the first biomechanics parameter information, the second biomechanics parameter information and the third biomechanics parameter information are subjected to information integration to obtain corresponding integrated information, and the integrated information is taken as the physical condition information to provide physical constraint information for the posture condition diffusion module.

[0097] The application generates corresponding first biomechanics parameter information by analyzing the joint range of motion of the initial posture parameter based on the biomechanics constraint module, generates corresponding second biomechanics parameter information by estimating the balance of the initial posture parameter, and generates corresponding third biomechanics parameter information by subjecting the initial posture parameter to muscle synergy constraint. The first biomechanics parameter information, the second biomechanics parameter information and the third biomechanics parameter information are subjected to information integration to obtain the physical condition information. The application analyzes the joint range of motion, estimates the balance and subjects to muscle synergy constraint of the initial posture parameter based on the use of the biomechanics constraint function module, and then integrates the generated first biomechanics parameter information, the second biomechanics parameter information and the third biomechanics parameter information, so as to efficiently and accurately complete the biomechanics analysis and calculation of the initial posture parameter, improve the generation efficiency and accuracy of the physical condition information, and facilitate the subsequent use of the combination of the physical condition information and the semantic vector to guide the diffusion model to generate a posture conforming to the physical law, thereby improving the accuracy of the generated target posture parameter.

[0098] In some optional implementations, step S205 includes the following steps:

[0099] The target posture parameter is subjected to normalization processing to obtain corresponding specified parameters.

[0100] In the embodiment, the target pose parameters include joint angles, positions and other information adjusted by physical constraints. These parameters are usually represented in the form of joint coordinates or key point positions. The target pose parameters such as joint coordinates can be normalized to adapt to the standard input range of the image generation network, and corresponding specified parameters are generated. For example, the coordinates are mapped to the range of [-1, 1] for subsequent processing.

[0101] The preset heat map generation network is called.

[0102] In the embodiment, the heat map generation network can specifically adopt a convolutional neural network.

[0103] The specified parameters are subjected to heat map generation processing based on the heat map generation network to obtain corresponding spatial heat map representations.

[0104] In the embodiment, the heat map generation process based on the heat map generation network includes: inputting the normalized specified parameters into the heat map generation network, and these coordinates can represent points in two-dimensional or three-dimensional space; spatial features of joint positions are extracted through multiple convolutional layers and pooling layers in the heat map generation, and these convolutional layers can capture the relative positions and spatial relationships between joints; each joint corresponds to a heat map channel, and each pixel value in the heat map represents the probability of the joint appearing at the position; the joint coordinates are converted into heat maps by using a Gaussian kernel, so that the position of each joint is represented as a peak value of Gaussian distribution in the heat map, for example, the heat map of the knee joint has a relatively high intensity value at the actual position of the knee joint, and the intensity gradually decreases in the surrounding area; finally, the heat map generation network outputs a set of spatial heat map representations, and each heat map corresponds to the position and probability distribution of a joint. These heat maps collectively describe the positions and relative relationships of various joints in the image space.

[0105] The spatial heat map representations are taken as the pose heat map.

[0106] The target pose parameters are normalized to obtain corresponding specified parameters, then a preset heat map generation network is called, then the specified parameters are subjected to heat map generation processing based on the heat map generation network to obtain corresponding spatial heat map representations, and the spatial heat map representations are taken as the pose heat map. The target pose parameters are normalized to obtain specified parameters, then the specified parameters are subjected to heat map generation processing based on the use of the heat map generation network, and the generated spatial heat map representations are taken as the pose heat map, so that the conversion processing of the target pose parameters can be efficiently and accurately completed, the conversion efficiency of the target pose parameters is improved, and the accuracy of the obtained pose heat map is ensured. Moreover, the use of the pose heat map can guide the image generator to accurately and reliably generate images meeting the requirements.

[0107] In some optional implementations, step S206 includes the following steps:

[0108] Obtaining a preset fusion strategy.

[0109] In this embodiment, the selection of the fusion strategy is not specifically limited, for example, splicing, addition or other fusion methods can be used.

[0110] Based on the fusion strategy, the semantic vector and the pose heat map are injected as conditional information into the image generator.

[0111] In this embodiment, the image generator can specifically use a diffusion model based on the U-Net architecture. The semantic vector and the pose heat map can be injected as conditional information into the image generator. These conditional information are injected into different levels of the image generator to affect various stages of the image generation process. In the U-Net architecture, the conditional information can be fused with the feature maps of the image generator through splicing, addition or other fusion methods, so that the text semantics and pose information are considered simultaneously in the image generation process.

[0112] Based on the image generator subjected to the conditional information injection process, the image generation process is performed to obtain a corresponding output image.

[0113] In this embodiment, the image generation process includes: a downsampling stage, in which the downsampling part of the image generator extracts low-level features of the input gradually and compresses the spatial resolution through convolution layers and downsampling operations; an upsampling stage, in which the upsampling part restores the spatial resolution gradually and generates high-resolution images through transpose convolution layers or upsampling operations; and a skip connection, in which the skip connection splices the feature maps of the downsampling stage with the feature maps of the upsampling stage in the U-Net architecture to help restore the detailed information. Through the image generation process, the image generator outputs a final image, which shows the pose consistent with the text description and physical laws.

[0114] The output image is taken as the target image.

[0115] The present application obtains a preset fusion strategy; then, based on the fusion strategy, the semantic vector and the posture heat map are injected into the image generator as conditional information; then, based on the image generator that has been processed with the conditional information injection, the image generation process is executed to obtain a corresponding output image; and subsequently, the output image is used as the target image. The present application obtains a preset fusion strategy; then, based on the fusion strategy, the semantic vector and the posture heat map are injected into the image generator as conditional information; then, based on the image generator that has been processed with the conditional information injection, the image generation process is executed to achieve the goal of guiding the image generator to generate a target image that meets the requirements through the semantic vector and the posture heat map. The target image not only retains the semantic information of the text description, but also ensures the physical rationality of the generated posture, effectively improving the accuracy and reliability of the target image.

[0116] In some optional implementations of this embodiment, step S207 includes the following steps:

[0117] The target image is optimized based on a preset optimization processing strategy to obtain a corresponding processed image.

[0118] In this embodiment, the above-mentioned specific implementation process of optimizing the target image based on the preset optimization processing strategy to obtain the corresponding processed image will be further described in detail in subsequent specific embodiments of this application and will not be elaborated on here.

[0119] Obtain user information of the user.

[0120] In this embodiment, the user information of the user can be obtained by querying the user information, wherein the user information may include the user's name information or ID information.

[0121] A corresponding image output mode is determined based on the user information.

[0122] In this embodiment, the user's preferred data push method can be queried through the acquired user information to determine the interest tag, and the interest tag can be used as the user's corresponding image output method. Interest tags may include pop-up display, interface display, email delivery, and internal message delivery. Furthermore, interest tags matching the user's preferred data push method can be assigned based on user registration options, historical behavior data, or questionnaires.

[0123] Based on the image output mode, the target image is sent to the user.

[0124] In this embodiment, the generated target image can be sent to the user according to the selected image output mode to complete the output processing of the target image.

[0125] The application obtains a corresponding processing image by performing optimization processing on the target image based on a preset optimization processing strategy; then acquires user information of the user; subsequently determines a corresponding image output mode based on the user information; and subsequently sends the target image to the user based on the image output mode. The application performs optimization processing on a target image based on the use of an optimization processing strategy to obtain a processing image, determines an image output mode according to user information of the user, and then sends the target image to the user based on the use of the image output mode, which can effectively ensure the quality and accuracy of the sent target image, improve the intelligent pushing of the target image, and improve the user experience.

[0126] In some optional implementations of the embodiment, the optimization processing on the target image based on the preset optimization processing strategy to obtain a corresponding processing image includes the following steps:

[0127] Performing super-resolution reconstruction processing on the target image to obtain a corresponding first generated image.

[0128] In the embodiment, the target of the super-resolution reconstruction processing includes improving the resolution of the image to make the details clearer. The implementation process of the super-resolution reconstruction processing includes inputting the generated target image into a super-resolution model, wherein a pre-trained super-resolution model (such as ESRGAN, SRCNN, etc.) is used, and these models are usually trained on a large-scale dataset; the model extracts low-level and high-level features of the image through a convolution layer; upsampling is performed through a transposed convolution or an interpolation method to gradually increase the resolution of the image; a residual block or an attention mechanism is used to enhance the details of the image to make it clearer; and a high-resolution image is output, which is more detailed and clearer.

[0129] Performing denoising processing on the first generated image to obtain a corresponding second generated image.

[0130] In the embodiment, the target of the denoising processing includes removing noise in the image to improve the image quality. The implementation process of the denoising processing includes identifying the type and intensity of noise in the image; using a pre-trained denoising model, which is trained on a noise image dataset; the denoising model extracts features of the noise image through a convolution layer; noise is removed through residual learning or other methods while the details of the image are preserved; and a denoised image is output, which has reduced noise and improved quality.

[0131] Performing style transfer processing on the second generated image to obtain a corresponding third generated image.

[0132] In the embodiment, the target of the above-mentioned style transfer processing includes: transferring the style of an image to the generated image to change its visual effect. The implementation process of the style transfer processing includes: selecting a style image containing the desired visual style (such as impressionism, abstraction, etc.). Using the generated target image as the content image; using a pre-trained style transfer model (such as Neural Style Transfer, AdaIN, etc.) to extract the features of the content image and the style image respectively; fusing the style features into the content image through an optimization process, maintaining the content structure while applying the style features; outputting the image after style transfer, which has the structure of the content image and the visual effect of the style image.

[0133] Edge enhancement processing is performed on the third generated image to obtain a corresponding fourth generated image.

[0134] In the embodiment, the target of the above-mentioned edge enhancement processing includes: enhancing the edge information of the image to improve the definition and contrast of the image. The implementation process of the edge enhancement processing includes: using an edge detection algorithm to identify the edges in the image; enhancing the edge information through a sharpening filter; while enhancing the edges, ensuring that other details of the image are not lost; outputting the image after edge enhancement, with clearer edges and improved overall contrast.

[0135] The fourth generated image is used as the processing image.

[0136] The present application obtains a corresponding first generated image by performing super-resolution reconstruction processing on the target image; then obtains a corresponding second generated image by performing denoising processing on the first generated image; then obtains a corresponding third generated image by performing style transfer processing on the second generated image; and subsequently obtains a corresponding fourth generated image by performing edge enhancement processing on the third generated image, and uses the fourth generated image as the processing image. The present application performs super-resolution reconstruction processing, denoising processing, style transfer processing and edge enhancement processing on the target image, so as to efficiently and accurately complete the optimization processing of the target image, so that the generated processing image is significantly improved in resolution, noise, style and definition, thereby improving the image quality and visual effect.

[0137] In some optional implementations, the obtained user information seeks the consent of the user and complies with relevant laws and relevant policies.

[0138] In addition, the non-company software tools or components appearing in the embodiments of the present application are only examples and do not represent actual use.

[0139] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the application.

[0140] It should be emphasized that, in order to further ensure the privacy and security of the target image, the target image can also be stored in a node of a blockchain.

[0141] The blockchain referred to in the present application is a new application mode of distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm and other computer technologies. Blockchain, in essence, is a decentralized database, which is a series of data blocks associated using cryptographic methods, each data block containing information about a batch of network transactions, used to verify the validity (anti-fake) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0142] The embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (Artificial Intelligence, AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Theory, method, technology and application system.

[0143] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, robot technology, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc. Several major directions.

[0144] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by computer readable instructions instructing related hardware, which can be stored in a computer readable storage medium. The program can include the processes of the above-mentioned embodiments when executed, wherein the storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0145] It should be understood that although each step in the flowchart of the accompanying drawings is shown in sequence according to the direction of the arrow, these steps are not necessarily executed in sequence according to the direction of the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and they can be executed in other sequences. Moreover, at least part of the steps in the flowchart of the accompanying drawings can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence is not necessarily sequential, but can be alternately executed with at least part of other steps or sub-steps or stages of other steps.

[0146] Further referring to Figure 3 , as an implementation of the method shown in the above Figure 2 , the present application provides an embodiment of an image generation device, which corresponds to the method embodiment shown in Figure 2 , and the device can be applied to various electronic devices.

[0147] As shown in Figure 3 , the image generation device 300 described in the embodiment includes an acquisition module 301, a first conversion module 302, a first processing module 303, an optimization module 304, a second conversion module 305, a second processing module 306, and an output module 307. Among them:

[0148] The acquisition module 301 is configured to acquire a text description input by a user;

[0149] The first conversion module 302 is configured to convert the text description into a corresponding semantic vector based on a pre-trained language model;

[0150] The first processing module 303 is configured to perform pose estimation processing on the text description based on a pre-trained pose predictor to obtain a corresponding initial pose parameter;

[0151] The optimization module 304 is configured to perform parameter optimization on the initial pose parameter based on a preset biomechanical constraint module and a pose condition diffusion module to obtain a corresponding target pose parameter;

[0152] The second conversion module 305 is configured to convert the target pose parameter into a corresponding pose heat map;

[0153] The second processing module 306 is configured to process the semantic vector and the pose heat map based on a preset image generator to obtain a corresponding target image;

[0154] The output module 307 is configured to perform output processing on the target image.

[0155] In the embodiment, the operations performed by the above modules or units respectively correspond to the steps of the image generation method of the foregoing embodiments, and will not be described here again.

[0156] In some optional implementations of the embodiment, the optimization module 304 includes:

[0157] The first processing submodule is configured to perform biomechanical analysis and calculation processing on the initial pose parameter based on the biomechanical constraint function module, to obtain corresponding physical condition information.

[0158] The first injection submodule is configured to inject the semantic vector and the physical condition information into the pose condition diffusion module.

[0159] The optimization submodule is configured to perform physical constraint optimization processing corresponding to the initial pose parameter based on the pose condition diffusion module, to obtain an optimized processing parameter.

[0160] The first determination submodule is configured to determine the processing parameter as the target pose parameter.

[0161] In the embodiment, the operations performed by the above modules or units respectively correspond to the steps of the image generation method of the foregoing embodiments, and will not be described here again.

[0162] In some optional implementations of the embodiment, the first processing submodule includes:

[0163] The first generation unit is configured to perform joint range of motion analysis on the initial pose parameter based on the biomechanical constraint module, to generate corresponding first biomechanical parameter information.

[0164] The second generation unit is configured to perform balance estimation on the initial pose parameter, to generate corresponding second biomechanical parameter information.

[0165] The third generation unit is configured to perform muscle synergy constraint on the initial pose parameter, to generate corresponding third biomechanical parameter information.

[0166] The integration unit is configured to integrate the first biomechanical parameter information, the second biomechanical parameter information, and the third biomechanical parameter information, to obtain the physical condition information.

[0167] In the embodiment, the operations performed by the above modules or units respectively correspond to the steps of the image generation method of the foregoing embodiments, and will not be described here again.

[0168] In some optional implementations of the embodiment, the second conversion module 305 includes:

[0169] The second processing submodule is configured to normalize the target pose parameter to obtain a corresponding specified parameter.

[0170] The calling submodule is configured to call a preset heat map generation network.

[0171] The third processing submodule is configured to perform heat map generation processing on the specified parameter based on the heat map generation network to obtain a corresponding spatial heat map representation.

[0172] The second determining submodule is configured to take the spatial heat map representation as the pose heat map.

[0173] In the embodiment, the above modules or units are respectively used for performing operations corresponding to steps of the image generation method of the foregoing embodiments, and thus will not be described here again.

[0174] In some optional implementations of the embodiment, the second processing module 306 includes:

[0175] The first obtaining submodule is configured to obtain a preset fusion strategy.

[0176] The second injection submodule is configured to inject the semantic vector and the pose heat map into the image generator as conditional information based on the fusion strategy.

[0177] The execution submodule is configured to perform an image generation process based on the image generator subjected to the conditional information injection processing to obtain a corresponding output image.

[0178] The third determining submodule is configured to take the output image as the target image.

[0179] In the embodiment, the above modules or units are respectively used for performing operations corresponding to steps of the image generation method of the foregoing embodiments, and thus will not be described here again.

[0180] In some optional implementations of the embodiment, the output module 307 includes:

[0181] The fourth processing submodule is configured to perform optimization processing on the target image based on a preset optimization processing strategy to obtain a corresponding processed image.

[0182] The second obtaining submodule is configured to obtain user information of the user.

[0183] The fourth determining submodule is configured to determine a corresponding image output mode based on the user information.

[0184] The sending submodule is configured to send the target image to the user based on the image output mode.

[0185] In the embodiment, the operations performed by the modules or units described above correspond to the steps of the image generation method of the foregoing embodiments one by one, and will not be described here again.

[0186] In some optional implementations of the embodiment, the fourth processing sub-module includes:

[0187] The first processing unit is configured to perform super-resolution reconstruction processing on the target image to obtain a corresponding first generated image.

[0188] The second processing unit is configured to perform denoising processing on the first generated image to obtain a corresponding second generated image.

[0189] The third processing unit is configured to perform style transfer processing on the second generated image to obtain a corresponding third generated image.

[0190] The fourth processing unit is configured to perform edge enhancement processing on the third generated image to obtain a corresponding fourth generated image.

[0191] The determination unit is configured to determine the fourth generated image as the processed image.

[0192] In the embodiment, the operations performed by the modules or units described above correspond to the steps of the image generation method of the foregoing embodiments one by one, and will not be described here again.

[0193] To solve the above technical problems, the embodiment of the present application further provides a computer device. For details, please refer to Figure 4 , Figure 4 The basic structure block diagram of the computer device in the embodiment is shown in FIG. 4.

[0194] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 which are connected to each other through a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented. Among them, those skilled in the art can understand that the computer device here is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0195] The computer device can be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The computer device can interact with a user through a keyboard, a mouse, a remote controller, a touchpad, a voice control device, or the like.

[0196] The memory 41 can include at least one type of readable storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, or the like), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, or the like. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or a memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like. Of course, the memory 41 can include both an internal storage unit and an external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store an operating system and various application software installed in the computer device 4, such as computer readable instructions of the image generation method, or the like. In addition, the memory 41 can also be used to temporarily store various data that has been output or will be output.

[0197] The processor 42 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer readable instructions or process data stored in the memory 41, such as computer readable instructions of the image generation method.

[0198] The network interface 43 can include a wireless network interface or a wired network interface, and is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0199] The present application also provides another embodiment, i.e., a computer readable storage medium storing computer readable instructions, which can be executed by at least one processor to make the at least one processor perform the steps of the image generation method as described above.

[0200] Those skilled in the art can clearly understand the above-mentioned embodiment method can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for making a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) execute the method described in each embodiment of the present application.

[0201] Obviously, the above-described embodiments are only some of the embodiments of the present application, not all the embodiments, and the drawings show the preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms, and conversely, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing specific embodiments, or make equivalent replacements to some of the technical features. Any equivalent structure made by using the content of the specification and drawings, directly or indirectly applied to other related technical fields, is also within the scope of the patent protection of the present application.

Claims

1. An image generation method, characterized in that: The steps include: Get the text description entered by the user; Converting the text description into a corresponding semantic vector based on a pre-trained language model; Performing posture estimation processing on the text description based on a pre-trained posture predictor to obtain corresponding initial posture parameters; Optimizing the initial posture parameters based on a preset biomechanical constraint module and a posture condition diffusion module to obtain corresponding target posture parameters; Converting the target posture parameters into a corresponding posture heatmap; Processing the semantic vector and the posture heat map based on a preset image generator to obtain a corresponding target image; Output processing is performed on the target image.

2. The image generation method according to claim 1, wherein: The step of optimizing the initial posture parameters based on the preset biomechanical constraint module and the posture condition diffusion module to obtain corresponding target posture parameters specifically includes: Performing biomechanical analysis and calculation processing on the initial posture parameters based on the biomechanical constraint function module to obtain corresponding physical condition information; injecting the semantic vector and the physical condition information into the posture condition diffusion module; performing physical constraint optimization processing corresponding to the initial posture parameters based on the posture condition diffusion module to obtain optimized processing parameters; The processing parameters are used as the target posture parameters.

3. The image generation method according to claim 2, wherein: The step of performing biomechanical analysis and calculation processing on the initial posture parameters based on the biomechanical constraint function module to obtain corresponding physical condition information specifically includes: Performing joint range analysis on the initial posture parameters based on the biomechanical constraint module to generate corresponding first biomechanical parameter information; Performing a balance estimation on the initial posture parameters to generate corresponding second biomechanical parameter information; Performing muscle coordination constraints on the initial posture parameters to generate corresponding third biomechanical parameter information; The first biomechanical parameter information, the second biomechanical parameter information, and the third biomechanical parameter information are integrated to obtain the physical condition information.

4. The image generation method according to claim 1, wherein: The step of converting the target posture parameters into a corresponding posture heat map specifically includes: Normalizing the target posture parameters to obtain corresponding specified parameters; Call the preset heat map generation network; Performing heat map generation processing on the specified parameters based on the heat map generation network to obtain a corresponding spatial heat map representation; The spatial heatmap is represented as the posture heatmap.

5. The image generation method according to claim 1, wherein: The step of processing the semantic vector and the posture heat map based on a preset image generator to obtain a corresponding target image specifically includes: Get the preset fusion strategy; Based on the fusion strategy, injecting the semantic vector and the posture heat map as conditional information into the image generator; An image generator that has been processed by conditional information injection performs an image generation process to obtain a corresponding output image; The output image is used as the target image.

6. The image generation method according to claim 1, wherein: The step of outputting the target image specifically includes: Optimizing the target image based on a preset optimization processing strategy to obtain a corresponding processed image; Obtaining user information of the user; determining a corresponding image output mode based on the user information; Based on the image output mode, the target image is sent to the user.

7. The image generation method according to claim 6, wherein: The step of optimizing the target image based on a preset optimization processing strategy to obtain a corresponding processed image specifically includes: Performing super-resolution reconstruction processing on the target image to obtain a corresponding first generated image; performing denoising processing on the first generated image to obtain a corresponding second generated image; performing style transfer processing on the second generated image to obtain a corresponding third generated image; performing edge enhancement processing on the third generated image to obtain a corresponding fourth generated image; The fourth generated image is used as the processed image.

8. An image generating device, characterized in that: include: The acquisition module is used to obtain the text description input by the user; A first conversion module, configured to convert the text description into a corresponding semantic vector based on a pre-trained language model; A first processing module is configured to perform posture estimation processing on the text description based on a pre-trained posture predictor to obtain corresponding initial posture parameters; an optimization module, configured to optimize the initial posture parameters based on a preset biomechanical constraint module and a posture condition diffusion module to obtain corresponding target posture parameters; A second conversion module, configured to convert the target posture parameters into a corresponding posture heat map; A second processing module is configured to process the semantic vector and the posture heat map based on a preset image generator to obtain a corresponding target image; An output module is used to perform output processing on the target image.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the image generation method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the image generation method according to any one of claims 1 to 7.

Images