Image adjustment method and device, processor, storage medium, electronic equipment and program product

Abstract

The invention discloses an image adjusting method and device, a processor, a storage medium, electronic equipment and a program product. The method comprises the steps that a to-be-converted image with the image content containing an initial biological object and a reference image with the image content containing a reference biological object are acquired, and the reference biological image is used for adjusting the initial biological object; adjusting limbs of the initial biological object based on reference limb information of a reference biological object in the reference image to obtain a first biological object; determining first limb attribute information of the first biological object and second limb attribute information of the reference biological object; based on the second limb attribute information and the first limb attribute information, adjusting the first biological object to obtain a target biological object; and converting the to-be-converted image based on the target biological object to obtain a target image. The technical problem that the generated animation is low in matching degree is solved.

Description

Technical Field

[0001] The present application relates to the field of video processing, and in particular to an image adjustment method, device, processor, storage medium, electronic device and program product. Background Art

[0002] Currently, in the field of dynamic video animation generation, how to generate dynamic and realistic video animations from static images is a key challenge. Traditional graphics techniques have been enhanced by data-driven models that leverage large visual datasets to generate more realistic details. However, these methods face challenges in pose transfer and maintaining subject identity.

[0003] In the related technology, traditional graphics technology has been significantly improved with the assistance of data-driven models. However, the above method still has the technical problem of low matching degree of the generated animation.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide an image adjustment method, device, processor, storage medium, electronic device and program product to at least solve the technical problem of low matching degree of generated animation.

[0006] According to one aspect of an embodiment of the present application, a method for adjusting an image is provided, which may include: obtaining an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; adjusting the limbs of the initial biological object based on reference limb information of the reference biological object in the reference image to obtain a first biological object; determining first limb attribute information of the first biological object, and second limb attribute information of the reference biological object; adjusting the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object; converting the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0007] Optionally, the method may further include: extracting the image to be converted to obtain initial limb information of the initial biological object, and extracting the reference biological object to obtain reference limb information.

[0008] Optionally, based on reference limb information of a reference biological object in a reference image, the limbs of an initial biological object are adjusted to obtain a first biological object, including: determining affine transformation parameters based on the reference limb information and the initial limb information, wherein the affine transformation parameters are used to adjust properties of the limbs of the initial biological object; and adjusting the limbs according to the affine transformation parameters to obtain the first biological object.

[0009] Optionally, determining first limb attribute information of a first biological object and second limb attribute information of a reference biological object includes: obtaining multiple first position information of multiple key points in the first biological object and multiple second position information of multiple reference key points in the reference biological object; determining the first limb attribute information based on the multiple first position information, and determining the second limb attribute information based on the multiple second position information.

[0010] Optionally, based on the second limb attribute information and the first limb attribute information, the first biological object is adjusted to obtain a target biological object, including: determining a ratio between the first limb attribute information and the second limb attribute information at the same limb position; in response to the ratio not satisfying a preset range, adjusting the first biological object to obtain the target biological object.

[0011] Optionally, in response to the ratio not satisfying a preset range, the first biological object is adjusted to obtain a target biological object, including: in response to the ratio not satisfying a preset range, determining angle information corresponding to the limb, wherein the angle information is used to characterize the direction of the limb; based on the angle information and the ratio, adjusting the first biological object to obtain the target biological object.

[0012] According to another aspect of an embodiment of the present application, an image adjustment device is also provided, which may include: an acquisition unit, used to acquire an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; a first adjustment unit, used to adjust the limbs of the initial biological object based on reference limb information of the reference biological object in the reference image to obtain a first biological object; a determination unit, used to determine first limb attribute information of the first biological object, and second limb attribute information of the reference biological object; a second adjustment unit, used to adjust the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object; a conversion unit, used to convert the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0013] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the image adjustment method of the embodiment of the present application.

[0014] According to another aspect of an embodiment of the present application, a processor is further provided, the processor being used to run a program, wherein the program, when run by the processor, executes the image adjustment method of the embodiment of the present application.

[0015] According to another aspect of an embodiment of the present application, a computer program product is further provided. The computer program product includes computer instructions, wherein the computer instructions, when executed by a processor, implement the image adjustment method of the embodiment of the present application.

[0016] In an embodiment of the present application, an image to be converted whose image content includes an initial biological object and a reference image whose image content includes a reference biological object are obtained, wherein the reference biological image is used to adjust the initial biological object; based on the reference limb information of the reference biological object in the reference image, the limbs of the initial biological object are adjusted to obtain a first biological object; first limb attribute information of the first biological object and second limb attribute information of the reference biological object are determined; based on the second limb attribute information and the first limb attribute information, the first biological object is adjusted to obtain a target biological object; based on the target biological object, the image to be converted is converted to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold. That is, in an embodiment of the present application, the limbs, limb attributes and other information of the initial biological object in the image to be converted are adjusted using the reference biological object in the reference image to obtain the target biological object, and the image to be converted is converted based on the target biological object to obtain a biological image, thereby achieving the purpose of quickly adjusting the initial biological object in the image to be converted, thereby solving the technical problem of low matching degree of the generated animation and achieving the technical effect of improving the matching degree of the generated animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 is a flow chart of an image adjustment method according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of limb-driven skeleton point matching alignment based on a multiple correction enhancement strategy according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a matching effect according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of an image adjustment device according to an embodiment of the present application;

[0022] Figure 5 is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0023] Figure 6 It is a block diagram of an electronic device according to an image adjustment method of an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0027] Body drive refers to the use of various additional control conditions to convert static images into dynamic videos through generative models. It can be implemented based on diffusion models and can be applied to entertainment, games, virtual reality and other fields to provide more vivid interactive and visual experiences.

[0028] Diffusion Models can be a type of generative model that can generate data by gradually removing noise. The above process can be regarded as a reverse process in time steps, starting from Gaussian noise, gradually removing noise, and finally generating a clear image, thereby generating target data samples from noise (sampled from a simple distribution);

[0029] Denoising Diffusion Implicit Models (DDIM) is a generative model based on diffusion model. It can generate high-quality images by operating in latent space. Compared with traditional diffusion model, its characteristic inference distribution does not depend on the forward process definition of Markov chain, which means that it can get rid of the step-by-step dependency of Markov chain and can skip step denoising, so it can quickly decode clear images from noisy samples in latent space.

[0030] Skeleton Alignment algorithm refers to the algorithm for matching the input reference image and the target pose video skeleton in the limb driving task. This algorithm can effectively solve the problems of inconsistent body proportions and background flickering in the generated video. It helps to ensure that the character's bone structure is aligned with the reference image or template when generating animations or processing images.

[0031] According to an embodiment of the present application, an embodiment of a method for adjusting an image is provided, and the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and, although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] In this embodiment, a method for adjusting an image is proposed. The method can use a reference biological object in a reference image to adjust information such as limbs and limb attributes of an initial biological object in an image to be converted to obtain a target biological object. The image to be converted is converted based on the target biological object to obtain a biological image, thereby achieving the purpose of quickly adjusting the initial biological object in the image to be converted, thereby solving the technical problem of low matching degree of the generated animation and achieving the technical effect of improving the matching degree of the generated animation.

[0033] Figure 1 FIG. 1 is a flow chart of an image adjustment method according to an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0034] Step S101, obtaining an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object;

[0035] Step S102, adjusting the limbs of the initial biological object based on the reference limb information of the reference biological object in the reference image to obtain a first biological object;

[0036] Step S103, determining first limb attribute information of the first biological object, and second limb attribute information of the reference biological object;

[0037] Step S104, adjusting the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object;

[0038] Step S105 , transforming the image to be transformed based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0039] In this embodiment, the initial biological object and the reference biological object may be objects containing vital signs, such as animals, humans, etc. It should be noted that this is only for example, and the types of the biological objects are not specifically limited. As long as the objects contain vital signs, they should be within the scope of protection of this application. The object to be converted may be a video frame, or an image obtained from the video to be converted. The reference image may be a pre-collected image, and may include a reference biological object. The reference biological object may be a biological object in a normal state, and may be used to determine the limb length, limb position, and proportional relationship between limbs of the initial biological object. The reference limb information may include coordinate information of key points between limbs, and may include information such as limb length, limb angle, and limb proportional relationship. The first limb attribute information may be used to characterize the limb length of the first biological object. The second limb attribute information may be used to characterize the limb length of the reference biological object. The limbs may include wrists, shoulders, feet, hands, ears, upper arms, lower arms, thighs, lower legs, and other torsos. The adjustment may include operations such as reducing or expanding the length of the limbs. The similarity threshold may be a preset value. It should be noted that this is only an example, and there is no specific limitation on the limb type, adjustment method, and method of determining the similarity threshold.

[0040] Optionally, an object to be converted including an initial biological object is obtained, and a reference image including a reference biological object can be obtained. The reference biological object and the initial biological object are the same biological object, but both can be used to characterize the state of the biological object under different actions. Reference limb information of the reference biological object in the reference image is determined, and based on the reference limb information, second limb attribute information of the reference biological object can be determined. At the same time, the length of the limb of the initial biological object can be adjusted based on the reference limb information to obtain the first biological object. Based on the first limb attribute information and the second limb attribute information, the proportional relationship between the limb of the first biological object and the limb of the reference biological object can be determined, and based on the proportional relationship, the first biological object can be adjusted to obtain the target biological object. The image to be converted can be converted based on the target biological object, so as to obtain a target image, and the similarity of the limb proportion, limb length and other attributes of the biological object in the target image to the limb proportion, limb length and the reference biological object is higher than the similarity threshold.

[0041] Optionally, the above-mentioned image to be converted can be data captured from a video frame, and can include an initial biological object that needs to be adjusted in posture. The above-mentioned reference image can be a biological object that includes a target posture, that is, a reference biological object. This reference image can be used as a standard or template for posture adjustment. The image data of the biological object, that is, the visual information of the initial biological object and the reference biological object, can be read from these two images. Further, the reference limb information of the reference biological object in the reference image can be referenced, including information such as the length, angle and position of the limbs. Based on this information, the limbs of the initial biological object on the image to be converted can be adjusted so that the initial biological object matches the reference biological object. The above-mentioned adjustment can involve scaling, rotating and translating the limbs, etc., and the type of adjustment is not specifically limited here. The adjusted initial biological object can be the first biological object.

[0042] Furthermore, the limb attribute information of the first biological object and the reference biological object can be determined. The limb attribute information of the first biological object (first limb attribute information) and the limb attribute information of the reference biological object (second limb attribute information) may include but are not limited to the specific length of the limb, the outline of the muscle, the shape of the limb, etc. The second limb attribute information of the reference biological object is compared with the first limb attribute information of the first biological object to determine the difference between the two. The target biological object can be fine-tuned based on the difference to make the limb attributes of the first biological object closer to the reference biological object. This may involve fine-tuning of limb length, matching of muscle lines, optimization of limb shape, etc. After adjustment, the first biological object is optimized to be a target biological object, and its limb attributes are more consistent with the reference biological object.

[0043] Optionally, after acquiring the target biological object, the adjusted target biological object can be applied to the image to be converted to replace or modify the original biological object in the image. This process can involve texture mapping, color adjustment, shape matching and other technologies to ensure that the visual similarity between the target biological object and the reference biological object is greater than a similarity threshold. The biological object in the generated target image has a high similarity in appearance to the reference biological object.

[0044] Optionally, the above method gradually adjusts and optimizes the limbs of the initial biological object, and finally generates a target image that is highly similar to the reference biological object in both posture and appearance. This technology has broad application prospects in the fields of animation production, virtual reality, game development, image synthesis, etc., and can help users or systems create more realistic and demand-compliant biological images in digital content.

[0045] Through the above steps S101 and S105 of the present application, an image to be converted whose image content includes an initial biological object and a reference image whose image content includes a reference biological object are obtained, wherein the reference biological image is used to adjust the initial biological object; based on the reference limb information of the reference biological object in the reference image, the limbs of the initial biological object are adjusted to obtain a first biological object; the first limb attribute information of the first biological object and the second limb attribute information of the reference biological object are determined; based on the second limb attribute information and the first limb attribute information, the first biological object is adjusted to obtain a target biological object; based on the target biological object, the image to be converted is converted to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than the similarity threshold. That is, in the embodiment of the present application, the reference biological object in the reference image is used to adjust the limbs, limb attributes and other information of the initial biological object in the image to be converted to obtain the target biological object, and the image to be converted is converted based on the target biological object to obtain a biological image, thereby achieving the purpose of quickly adjusting the initial biological object in the image to be converted, thereby solving the technical problem of low matching degree of the generated animation, and achieving the technical effect of improving the matching degree of the generated animation.

[0046] The above method of this embodiment is further introduced below.

[0047] As an optional implementation, the method may further include: extracting the image to be converted to obtain initial limb information of the initial biological object, and extracting the reference biological object to obtain reference limb information; step S102, based on the reference limb information of the reference biological object in the reference image, adjusting the limbs of the initial biological object to obtain a first biological object, including: determining affine transformation parameters based on the reference limb information and the initial limb information, wherein the affine transformation parameters are used to adjust the properties of the limbs of the initial biological object; adjusting the limbs according to the affine transformation parameters to obtain the first biological object.

[0048] In this embodiment, the image to be converted can be extracted to obtain the initial limb information of the initial biological object, and the reference biological object in the reference image can be extracted to obtain the reference limb information. The initial limb information and the reference limb information may include a probability map (heatmap) of key points in the limb and a connection relationship vector field (PAF) between key points, which may be posture information. It should be noted that this is only an example, and there is no specific restriction on the content of the limb information. The affine transformation parameters can be used to adjust the properties of the limbs of the initial biological object, and can be used to adjust the size, length, proportion and other properties of the limb. It should be noted that this is only an example, and there is no specific restriction on the adjustment object of the affine transformation parameters.

[0049] Optionally, after obtaining the image to be converted and the reference image, posture recognition (for example, human body posture recognition) may be performed on the two input images to obtain initial limb information and reference limb information.

[0050] For example, when the biological object is a human, an input image can be obtained, and the input image can be an image or video frame containing a human body, as well as a reference image containing the human body. The above input image can be feature extracted through a convolutional neural network to obtain initial limb information and reference limb information, and the limb information can be used for human posture features (or information). For example, a probability map of each key point and a connection relationship vector field between key points can be generated through a neural network, and the above recognition results can be refined in an iterative manner, so that the key point position can be determined through heatmap and PAF, and the limb connection (for example, skeleton connection) can be determined, and then the output data can be obtained. The output data can be initial limb information and reference limb information, and the above limb information can be displayed through an image or other format.

[0051] Optionally, after obtaining the initial limb information (e.g., coordinates of poses to be matched) and reference limb information (e.g., coordinates of reference key points), a rough affine transformation match can be performed based on the initial limb information and the reference limb information. The rough match can be based on the relative positions of key points of the body trunk (e.g., left shoulder, right shoulder, midpoint of shoulder, left hip, right hip), so that the coordinates of each key point in the initial biological object are close to the reference key point coordinates through scaling and rotation transformation, thereby effectively avoiding the situation of uncoordinated movements caused by too large a difference in the body shape of the precise matching person.

[0052] Optionally, the affine transformation parameters may be used to determine how to adjust the initial biological object to obtain the first biological object.

[0053] Optionally, the image to be converted is analyzed by computer vision technology, such as a deep learning human posture recognition algorithm, to extract the initial limb information of the initial biological object. The initial limb information may include key limb points of the initial biological object identified and located, such as shoulders, elbows, knees, etc., and record their positions, angles and lengths in the image. At the same time, the reference biological object in the reference image may also be processed in the same way to extract reference limb information of the reference biological object. The above limb information may be used in subsequent adjustment and matching steps.

[0054] Optionally, the reference limb information and the initial limb information are compared, especially those key points that can reflect the body proportion and orientation, such as shoulders, hips, and torso center points. By calculating the relative position and orientation between these key points, the system can determine a set of affine transformation parameters, including scaling, rotation, and translation factors, etc. It should be noted that this is only an example, and there is no specific restriction on the type of the above affine transformation parameters. The above affine transformation parameters can be used to adjust the properties of the initial biological object's limbs so that they match the limb information of the reference biological object in terms of overall size, orientation, and position.

[0055] Optionally, after the affine transformation parameters are determined, the limbs of the initial biological object can be adjusted according to the affine transformation parameters. For example, the limbs can be scaled to match the size of the reference organism, the limbs can be rotated to adjust their direction, and the limbs can be translated to align with the position of the reference organism. The above adjustments ensure that the limbs of the initial biological object are preliminarily aligned with the limbs of the reference biological object in terms of size, direction, and position, thereby obtaining a first biological object. The limb attributes (such as length, angle) of the first biological object have been adjusted accordingly based on the limb information of the reference organism to obtain the first biological object.

[0056] Through the above process, the affine transformation coarse matching algorithm can effectively use the biological object posture in the reference image to coarsely match the limbs of the initial biological object, and obtain a first biological object that is initially adjusted but closer to the reference biological posture. This coarse matching strategy is achieved through the calculation and application of affine transformation parameters, which provides an important starting point for subsequent fine adjustment and attribute matching, ensuring that the generated target biological object can maintain a high degree of consistency with the reference biological object in limb proportions and posture.

[0057] As an optional implementation, step S103, determining first limb attribute information of a first biological object, and second limb attribute information of a reference biological object, includes: obtaining multiple first position information of multiple key points in the first biological object, and multiple second position information of multiple reference key points in the reference biological object; determining the first limb attribute information based on the multiple first position information, and determining the second limb attribute information based on the multiple second position information.

[0058] In this embodiment, the first limb attribute information can be determined based on multiple first position information of multiple key points in the first biological object, and can be used to determine the length of the limb in the first biological object. The first position information can be coordinate information corresponding to the key point in the first biological object, and can be used to determine the position of multiple key points in the first biological object. The second limb attribute information can be determined based on multiple second position information of multiple reference key points in the reference biological object, and can be used to determine the length of the limb in the reference biological object. The second position information can be used to determine the position of multiple key points in the reference biological object in the reference biological object, and can be coordinate information corresponding to the key points in the reference biological object. The key points can be key points in the limbs, which can be used to determine the shape of the limbs and can be used to form the limbs. It should be noted that this is only for example, and there is no specific restriction on the contents of the first limb attribute information, the second limb attribute information, the second position information, and the first position information.

[0059] Optionally, after acquiring the first biological object, based on the rough matching, the key point coordinates can be extracted from the reference frame Pi (the first frame of the posture sequence by default) and the reference image posture q0 after the rough matching from the template posture sequence P to obtain the first position information and the second position information. Based on the first position information, the length of the limb obtained by connecting the two key points can be determined. Similarly, based on the second position information, the length or angle of the limb obtained by connecting the two key points in the reference biological object can be determined, that is, the second limb attribute information can include the length information or angle information corresponding to multiple limbs.

[0060] Optionally, after the rough matching of the affine transformation, in order to further improve the similarity and detail matching between the biological object and the reference biological object in the limbs, the precise position information of the key limb points of the first biological object can be extracted after the biological object is basically aligned with the reference biological object after preliminary adjustment. The above key points may include but are not limited to shoulders, elbows, wrists, hips, knees and ankles. By using a human posture recognition algorithm, the coordinates of these key points in the image can be accurately located, and these coordinate information are multiple first position information, which will be used for the subsequent determination of the first limb attribute information. Similarly, the coordinate information of the key limb points of the reference biological object can also be extracted from the reference image to form a set of multiple second position information. This information contains the position of the key limb points of the reference biological object in the target posture, which will serve as the target standard for adjusting the limb attributes of the biological object.

[0061] Furthermore, based on the multiple first position information extracted from the key points of the first biological object, the attribute information of the limbs, such as the length, width, angle and relative position between the limbs, etc., can be calculated. These attribute information constitute the first limb attribute information of the first biological object. Similarly, based on the multiple second position information extracted from the reference key points, the attributes of the limbs in the reference biological object, such as length, width, angle and posture, can be determined. These attribute information constitute the second limb attribute information of the second biological object.

[0062] Optionally, through the above process, the system can carefully analyze and quantify the limb attribute information of the first biological object and the reference biological object, providing specific data basis for the next step of precise adjustment. This stage of processing focuses on the accurate capture and expression of limb features, which is a key step to achieve subsequent precise limb correction and proportion adjustment and ensure that the generated target biological object is highly similar to the reference biological object.

[0063] As an optional implementation, step S104, based on the second limb attribute information and the first limb attribute information, adjusts the first biological object to obtain the target biological object, including: determining the ratio between the first limb attribute information and the second limb attribute information of the same limb position; in response to the ratio not satisfying a preset range, adjusts the first biological object to obtain the target biological object.

[0064] In this embodiment, the ratio between the first limb attribute information and the second limb attribute information of the same limb position can be determined. The ratio can be a ratio, which can be used to determine the limb proportion difference between the postures of the reference biological object and the first biological object. If the ratio does not meet the preset range, the limbs of the first biological object can be adjusted to obtain the target biological object.

[0065] Optionally, based on the rough matching, the first position information and the second position information can be obtained, and based on the first position information and the second position information, the first limb attribute information and the second limb attribute information corresponding to the limb can be determined. The ratio between the first limb attribute information and the second limb attribute information at the same limb position can be determined.

[0066] For example, for each pair of connected points (i k ,j k ), calculate the Euclidean distance d between two sets of coordinate systems 1k and d 2k , and find their ratio This ratio reflects the difference in limb proportions between the postures of the reference biological object and the first biological object. Based on the ratio, it can be determined whether to adjust the first biological object.

[0067] Optionally, the first biological object and the reference biological object are compared in terms of attribute information at the same limb position, in particular, the length, width, and angle of the limb. For example, the algorithm calculates the ratio of the length of the left arm of the reference biological object to the length of the left arm of the first biological object, and similar comparisons are made for other limbs such as legs, torsos, etc. These ratios reflect the degree of difference between the two biological objects in limb size and shape. A range can be preset in advance, which can define the acceptable difference in limb attributes between the two biological objects. If the calculated ratio exceeds this preset range, the corresponding limb of the first biological object can be further adjusted. This adjustment may be achieved by a slight scaling, rotation, or translation, with the purpose of making the limb attributes of the first biological object closer to the attributes of the reference biological object. During the adjustment process, the algorithm continuously checks the ratio between the adjusted limb attributes and the reference attributes until they fall within the preset range. Through such an iterative adjustment process, the first biological object gradually evolves into a target biological object, in which the limb attributes remain consistent with the reference biological object.

[0068] For example, if the reference biological object's legs are 10% longer than the first biological object's legs, the first biological object's leg length can be adjusted based on this difference to narrow the gap between the two. After the adjustment, the algorithm will calculate the ratio again to check whether it meets the preset range. If not, it will continue to adjust until the ratio meets the requirements.

[0069] Optionally, the above steps can be a fine-tuning stage following the coarse matching of the affine transformation, which focuses on adjusting the limb attributes of the first biological object to ensure an accurate match with the reference biological object in limb proportions. Through this process, the algorithm can overcome the unnatural proportion problem common in limb-driven tasks and improve the visual realism and coherence of the generated animation. This precise attribute adjustment strategy ensures that the generated target biological object can faithfully present the body characteristics of the reference biological object, whether it is limb length, width or shape, it remains within a preset similarity range, thereby greatly improving the visual quality of the generated video or animation.

[0070] Optionally, the limb length ratio can be corrected based on the ratio: in actual testing, due to the complex and diverse postures of the reference image, especially when the body is sideways and the head has a pitch angle, the above calculated ratio has a certain deviation, so it needs to be corrected. The specific correction positions are: wrist, shoulder, foot, hand, ear, upper arm, forearm, thigh, calf, etc. When the corresponding length ratio r is calculated k , determine whether the ratio is within a preset range (for example, [α·r w ,β·r w ]) If not, the limb proportions of the first biological object are corrected to avoid interference of extreme postures in calculating the proportions.

[0071] As an optional implementation, in response to the ratio not satisfying a preset range, the first biological object is adjusted to obtain a target biological object, including: in response to the ratio not satisfying a preset range, determining angle information corresponding to the limb, wherein the angle information is used to characterize the direction of the limb; based on the angle information and the ratio, adjusting the first biological object to obtain the target biological object.

[0072] In this embodiment, in response to the ratio not satisfying the preset range, the angle information corresponding to the limb can be determined, and the angle information can be used to characterize the direction of the limb. After the first biological object is adjusted based on the ratio, the adjusted biological object can be adjusted based on the angle information to obtain the target biological object. It should be noted that in addition to adjusting the first biological object in the above manner, the first biological object can also be adjusted simultaneously using the angle information and the ratio to obtain the target biological object.

[0073] For example, for each limb segment, the angle θ between the coordinates of the two endpoints can be calculated k =arctan2(c 1jk -c 1ik ,c 2jk -c 2ik ) to obtain the limb angle, which is used to determine the orientation of the limb segment. Based on the calculated length ratio r kand angle θ k , the limb coordinates in the image pose q0 (i.e., the limb coordinates of the first biological object) can be updated to match the template pose sequence where v k is a vector representing the direction and distance of movement of the limb segment in two-dimensional space. The final new point c′ 2i The coordinates of can be obtained as follows: 2i =c 1i +v k +∈, where ∈ is an offset used to adjust the position of the alignment pose to ensure alignment with the character features in the reference image. It is obtained by calculating the offset of the clavicle position coordinates in the reference image pose to the corresponding position coordinates in the template pose sequence.

[0074] Optionally, the angle information corresponding to the limb is determined, wherein the angle information can be used to characterize the direction of the limb: based on the ratio of the limb attributes of the first biological object and the reference biological object, if this ratio exceeds a preset range, it indicates that there is a significant difference in the size or shape of the limb, and further fine-tuning is required to improve the similarity. At this time, the angle information of the limb between the two connection points is calculated. These connection points may include joints, such as shoulder to elbow, elbow to wrist, hip to knee, etc. The angle information describes the directional characteristics of the limb and is one of the key parameters for adjusting the shape and position of the limb. By identifying and quantifying these angles, the orientation and posture of the limb can be understood more accurately, providing guidance for subsequent adjustments.

[0075] Optionally, after the angle information is determined, a comprehensive adjustment can be made in combination with the limb attribute ratio. That is, the relationship between the angle information and the limb attribute ratio can be analyzed to determine how to most effectively adjust the limbs of the first biological object so that they are close to the limb attributes of the reference biological object. For example, if the length ratio of a limb deviates from the preset range, and the angle of the limb is different from the angle of the corresponding limb of the reference biological object, the system will consider length scaling and smooth rotation at the same time to adjust the length and direction of the limb at the same time, ensuring that the adjusted limb and the limb of the reference biological object reach a preset similarity range in length and direction. This adjustment strategy based on angle information and ratios can effectively overcome the unnatural phenomena caused by body shape differences and posture angle changes during limb driving, and ensure that the generated biological object is highly coordinated with the reference biological object in limb proportions and directions.

[0076] Optionally, through the above steps, the system can not only identify and quantify the differences in limb attributes, but also make precise adjustments based on these differences to improve the visual effects of the generated animation or video. This process effectively combines scale correction and direction correction to ensure that the limbs of the biological object maintain both the appearance consistency with the reference biological object and the natural and smooth dynamic effects during the generation process, thereby providing strong technical support for the generation of high-quality dynamic content.

[0077] Currently, in the field of dynamic video animation generation, how to generate dynamic and realistic video animations from static images is a key challenge. Traditional graphics techniques have been enhanced by data-driven models that leverage large visual datasets to generate more realistic details. However, these methods face challenges in pose transfer and maintaining subject identity. In addition, existing diffusion model-based methods, while able to generate visually plausible animations, have problems with appearance consistency.

[0078] In the field of character animation generation, technological developments are driving the shift from static images to dynamic video animations, a shift that has significant implications for entertainment, social media, virtual reality, and other immersive digital experience industries. The core challenge of character animation technology is how to maintain appearance consistency and fidelity in animation sequences, which is critical to the realism and overall quality of generated content. Traditional graphics techniques have been significantly improved with the assistance of data-driven models that leverage large-scale visual datasets to achieve more cost-effective solutions.

[0079] Although generative adversarial networks have shown potential in creating realistic details, they face challenges in motion transfer and maintaining subject identity across poses. In addition, although diffusion models can generate visually plausible animations, when the input character image and the character in the driving video are too different in shape, the generated realism and consistency will be poor, the limb proportions in the generated animations will be unnatural, the effect will be unsatisfactory, and due to the different positions of the characters, the background will be cluttered and flickering.

[0080] In summary, the shortcomings of the above technologies show the importance of maintaining the consistency of character appearance in dynamic video generation, and also highlight the limitations of existing technologies. Therefore, it is necessary to propose a new training-free framework and dual alignment strategy to achieve higher-quality character animation generation, which is of great significance for promoting the progress of computer graphics and computer vision.

[0081] In order to solve the above problems, a posture-guided video generation strategy that does not require training is proposed in this embodiment. This strategy solves the challenges faced by the existing technology in dealing with dynamic video animation generation by introducing an innovative dual alignment method.

[0082] Optionally, this embodiment improves the realism and overall quality of the animation by ensuring that the generated video sequence accurately reflects the motion characteristics of the pose sequence while maintaining consistency in appearance details with the reference image.

[0083] Optionally, the method proposed in this embodiment does not require training, avoids the use of large video data sets and expensive GPU resources, lowers the technical threshold and cost of video generation, and reduces dependence on large data sets and expensive computing resources.

[0084] Optionally, the above method proposed in this embodiment enhances the precise control capability of the algorithm by separating the skeleton and motion priors, and precisely controlling the transfer of skeleton data during the animation generation process, thereby ensuring that the animation can faithfully reproduce the posture of the reference character while maintaining similarity to the motion of the posture sequence.

[0085] Optionally, the method proposed in this embodiment improves the alignment efficiency of the generated animation by improving the reference image so that it can be closely aligned with the initial frame of the driving posture video, and using the information of the current diffusion model to guide the reference image to imitate the movement of the starting posture. At the same time, the method enables the algorithm to dynamically adjust the generation strategy according to the characteristics of the reference image and posture sequence, thereby improving the applicability and flexibility of the algorithm under different roles and posture conditions.

[0086] In this embodiment, a training-free framework pose adapter (PoseAdapter) is proposed. The adapter maintains the appearance consistency between the generated video sequence and the reference image through a dual alignment strategy, that is, the first body scale coefficient calculation and update are completed by updating the node vector, and then the adjustment and correction are performed within a certain range according to the specific segment scale of the character in the input image. The core of this strategy is to separate the skeleton and motion priors from the pose information itself, and to identify the basic clues in the key point representation, such as the position, length and angle changes of the skeleton, which reflect the individual's body information and motion trend. By using efficient linear matrix operations, it is possible to distinguish the identity information and motion information in the skeleton sequence, so that the skeleton data in the reference image can be transferred to the driving pose sequence while maintaining the intrinsic motion characteristics of the pose.

[0087] In this embodiment, the matching logic based on node vector update and the adaptive extreme value correction mechanism can effectively complete the mapping between reference images and action postures. By optimizing the logic of the matching algorithm, the accuracy and stability of the image-generated video results can be significantly improved.

[0088] In this embodiment, a training-free framework is proposed that maintains the appearance consistency of the generated video sequence with the reference image through a dual alignment strategy. The core of this strategy lies in separating the skeleton and motion priors from the pose information itself, by identifying basic clues in the key point representation, such as skeleton position, length and angle changes, which reflect the individual's physical information and movement trends. Using efficient linear matrix operations, it is possible to distinguish the identity information and motion information in the skeleton sequence, so that the skeleton data in the reference image can be transferred to the driving pose sequence while maintaining the intrinsic motion characteristics of the pose. In addition, in order to achieve the accuracy of conditional control, the reference image is also improved so that it can start the animation closely aligned with the initial frame of the driving pose video. This improvement utilizes the information stored in the current diffusion model to guide the reference image to imitate the movement of the starting pose. The result is an improved alignment between the reference image and the driving pose video, laying the foundation for a temporally coherent and visually unified animation sequence.

[0089] In this embodiment, a dynamic correction strategy is proposed. The limb-driven bone point matching and alignment algorithm can dynamically adjust the matching ratio according to the limb lengths of different human bodies to ensure the consistency and accuracy of the matching. This feature enables the algorithm to adapt to different human skeletons and different human postures, including complex postures such as sitting and lying, thereby ensuring the similarity of the characters in the image-generated video characters and improving the generation quality.

[0090] Optionally, the above method can generate the required motion videos according to different customer needs, generate high-quality digital humans according to given reference images, etc. And the above method can significantly improve the generation quality and accuracy of image-generated video tasks.

[0091] Optionally, the above method also has the following beneficial effects: (1) Through the multiple alignment strategy, the appearance consistency between the reference image and the generated video can be better maintained, so that the appearance of the character in the video remains consistent, avoiding common problems such as background flickering and unnatural limbs. This method effectively solves the visual inconsistency caused by body shape differences when generating images in traditional methods, thereby significantly improving the authenticity and consistency of the generated video. (2) By accurately aligning the reference image at the initial stage of video generation, it is ensured that the starting frame of the animation is highly consistent with the first frame of the driving posture video, thereby improving the fluency and quality of the generated video. This not only makes the generated video more natural, but also reduces unnecessary posture deviations, making the generated animation more professional. (3) Compared with traditional generation technologies that rely on large-scale training, the present application adopts a posture guidance method that does not require training, which significantly reduces the dependence on large data sets and expensive computing resources (such as GPUs). This non-training framework design not only lowers the technical threshold, but also accelerates the video generation process, meeting the demand for efficient generation of dynamic content. (4) By introducing a mechanism for separating bones and motion priors, the posture transitions and action details of the generated video can be more accurately controlled. The separation of skeletal drive and motion features allows the algorithm to flexibly adapt to changes in the postures of different characters, whether it is a complex sitting posture, lying posture or other complex postures, it can ensure the natural transition and realistic performance of the characters in the animation. (5) It can be dynamically adjusted according to different reference images and posture sequences, and has strong applicability and flexibility. Whether it is entertainment, virtual reality, social media or other immersive digital experience fields, this technology can meet diverse application needs, thus bringing new breakthroughs and developments to character animation generation. (6) Through innovative posture-guided generation methods, it provides a new solution for the fields of computer graphics and computer vision. This not only promotes the development of character animation technology, but also provides a solid technical foundation for future applications in more fields (such as film and television production, game development, etc.).

[0092] Figure 2 is a schematic diagram of limb-driven skeleton point matching alignment based on a multiple correction enhancement strategy according to an embodiment of the present application, such as Figure 2As shown, an image to be converted 201 and a reference image 202 are obtained, wherein the image to be converted is a static image and needs to be converted into a dynamic video through an algorithm; the reference image provides a benchmark for the target posture and appearance. A deep learning human posture recognition algorithm can be used to extract the initial limb information of the initial biological object (human body) from the image to be converted, and at the same time extract the reference limb information of the reference biological object from the reference image. This process generates key point coordinates and attributes such as limb length and angle. Further, based on the extracted limb information, the system adjusts the initial biological object through affine transformation (including scaling, rotation and translation) so that it is basically aligned with the reference biological object in the limb position to generate a first biological object. This step solves the initial deviation in limb position and direction, laying the foundation for subsequent more refined adjustments. The precise position information of multiple key points of the first biological object and the reference biological object is extracted, and the first limb attribute information and the second limb attribute information are determined based on this information, including limb length, width, angle, etc. The ratio between the first limb attribute information and the second limb attribute information is compared. If these ratios do not meet the preset range, the angle information corresponding to the limb is further determined, which helps to characterize the direction of the limb. Based on the angle information and ratio, the algorithm fine-tunes the limb attributes of the first biological object to ensure that the limbs not only match in size, but also reach a high degree of consistency in direction and proportion, thereby generating a target biological object. After obtaining a target image containing the target biological object, the posture-guided generative model can be used to generate an animation containing the initial biological object. The target image can be a target image or video sequence that is highly consistent with the reference biological object in limb proportions and postures, ensuring the naturalness and coherence of the visual effect.

[0093] Figure 3 is a schematic diagram of a matching effect according to an embodiment of the present application, such as Figure 3 As shown, the left part shows multiple pose frames extracted from the driving video, which have been aligned after algorithm processing to reflect similar poses to the skeleton of the person in the reference image. The skeleton is represented by colored lines and points (or white lines). The pose frame can clearly show the position of the key points of the human body and the connection of the limbs, providing a benchmark for the subsequent fine-tuning and generation process.

[0094] Optionally, the middle part shows a reference image used to generate the animation, including a static character image. The reference image is used to provide visual features for the generated animation, including the character's appearance, clothing details, hairstyle, etc., to ensure that the generated animation is highly consistent with the reference image visually.

[0095] Optionally, the right part shows a template image processed by a dual alignment strategy. In this image, the original reference image character has been adjusted and aligned to a skeleton that matches the pose sequence in the driving video, achieving accurate matching of posture and limb proportions. The template image also retains the appearance features of the reference image, such as clothing, facial features, etc., to ensure that the generated animation remains visually unified and aligned with the dynamic skeleton of the driving video, generating a smooth and natural dynamic video. The above template image can be a target image.

[0096] like Figure 3 As shown in the figure, from the reference image to the template image, the character's posture and body proportions are precisely adjusted to match the dynamic skeleton in the driving video, while retaining the appearance characteristics of the reference image. This process effectively overcomes the problems of unnatural body proportions and background flickering in traditional methods, improves the realism and quality of the generated animation, and provides significant improvements to image-generated video technology.

[0097] pass Figure 3 It can be seen that the algorithm can dynamically adjust the generation strategy according to the input pose sequence and reference image to achieve skeleton alignment and appearance consistency. This not only reflects the technical innovation of the algorithm, but also provides higher quality output for image-generated video applications, which is suitable for character animation generation in entertainment, social media, virtual reality and other fields.

[0098] In this embodiment, a template pose sequence (i.e., multiple frames of images to be converted): poses1 and a reference image pose: pose2 may be input. By converting the above input by the above method, an output target image may be obtained, which may include a converted pose image set kps_results2 and a converted pose array poses2.

[0099] Optionally, the image to be transformed and the reference image may be scaled according to their actual sizes to ensure that the sizes of the two images match. Using the joint coordinates and limb sequence information of the image to be transformed and the reference image, edge length ratios (edge_ratios) may be calculated.

[0100] Optionally, for each pose1 in poses1, do the following: Update the body position in pose2 using the calculated edge_ratios and pose1. Further adjust the hand position in pose2 based on the updated body position, pose1, and edge_ratios. Normalize pose2 and draw it on the canvas to generate the transformed pose image pose_image2. Add pose_image2 to kps_results2 and add pose2 to the poses2 array. End the loop for all poses in poses1. Finally, output kps_results2 and poses2.

[0101] In an embodiment of the present application, the reference biological object in the reference image is used to adjust the limbs, limb attributes and other information of the initial biological object in the image to be converted to obtain the target biological object, and the image to be converted is converted based on the target biological object to obtain the biological image, thereby achieving the purpose of quickly adjusting the initial biological object in the image to be converted, thereby solving the technical problem of low matching degree of the generated animation and achieving the technical effect of improving the matching degree of the generated animation.

[0102] According to an embodiment of the present application, an image adjustment device is also provided. It should be noted that the image adjustment device of this embodiment can be used to execute the image adjustment method of the above embodiment of the present application.

[0103] Figure 4 is a schematic diagram of an image adjustment device according to an embodiment of the present application. Figure 4 As shown, the image adjustment device 40 may include: an acquisition unit 402 , a first adjustment unit 404 , a determination unit 406 , a second adjustment unit 408 and a conversion unit 410 .

[0104] The acquisition unit 402 is used to acquire an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object.

[0105] The first adjustment unit 404 is configured to adjust the limbs of the initial biological object based on the reference limb information of the reference biological object in the reference image to obtain a first biological object.

[0106] The determination unit 406 is configured to determine the first limb attribute information of the first biological object and the second limb attribute information of the reference biological object.

[0107] The second adjustment unit 408 is used to adjust the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object.

[0108] The conversion unit 410 is configured to convert the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0109] The image adjustment device of this embodiment obtains, through an acquisition unit, an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; through a first adjustment unit, based on reference limb information of the reference biological object in the reference image, the limbs of the initial biological object are adjusted to obtain a first biological object; through a determination unit, first limb attribute information of the first biological object and second limb attribute information of the reference biological object are determined; through a second adjustment unit, based on the second limb attribute information and the first limb attribute information, the first biological object is adjusted to obtain a target biological object; through a conversion unit, the image to be converted is converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold, thereby solving the technical problem of low matching degree of the generated animation and achieving the technical effect of improving the matching degree of the generated animation.

[0110] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0111] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0112] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image adjustment method: obtaining an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; adjusting the limbs of the initial biological object based on reference limb information of the reference biological object in the reference image to obtain a first biological object; determining first limb attribute information of the first biological object, and second limb attribute information of the reference biological object; adjusting the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object; converting the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0113] Optionally, Figure 5 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Figure 5 As shown, the computer terminal 508 may include: one or more (only one is shown in the figure) processors 502 , a memory 504 and a transmission device 506 .

[0114] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image adjustment method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image adjustment method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 508 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0115] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; based on reference limb information of the reference biological object in the reference image, adjust the limbs of the initial biological object to obtain a first biological object; determine first limb attribute information of the first biological object and second limb attribute information of the reference biological object; based on the second limb attribute information and the first limb attribute information, adjust the first biological object to obtain a target biological object; convert the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0116] It can be understood by those skilled in the art that Figure 5 The structure shown is for illustration only, and the computer terminal 508 may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID for short), a PAD, or other terminal devices. Figure 5 The structure of the computer terminal 508 is not limited. For example, the computer terminal 508 may also include Figure 5 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 5 Different configurations are shown.

[0117] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0118] According to an embodiment of the present application, a computer-readable storage medium is further provided, the computer-readable storage medium comprising a stored program, wherein the program executes the image adjustment method in the above embodiment.

[0119] Optionally, in this embodiment, the computer-readable storage medium may be located in any one of the computer terminals in a computer terminal group in a computer network, or in any one of the mobile terminals in a mobile terminal group.

[0120] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: extracting the image to be converted to obtain initial limb information of the initial biological object, and extracting the reference biological object to obtain reference limb information.

[0121] Optionally, the computer-readable storage medium may also execute program code for the following steps: determining affine transformation parameters based on reference limb information and initial limb information, wherein the affine transformation parameters are used to adjust properties of the limbs of the initial biological object; and adjusting the limbs according to the affine transformation parameters to obtain a first biological object.

[0122] Optionally, the above-mentioned computer-readable storage medium can also execute program code for the following steps: obtaining multiple first position information of multiple key points in a first biological object, and multiple second position information of multiple reference key points in a reference biological object; determining first limb attribute information based on multiple first position information, and determining second limb attribute information based on multiple second position information.

[0123] Optionally, the computer-readable storage medium may also execute program code for the following steps: determining a ratio between first limb attribute information and second limb attribute information of the same limb position; in response to the ratio not satisfying a preset range, adjusting the first biological object to obtain a target biological object.

[0124] Optionally, the computer-readable storage medium may also execute program code for the following steps: in response to the ratio not satisfying a preset range, determining angle information corresponding to the limb, wherein the angle information is used to characterize the direction of the limb; and adjusting the first biological object based on the angle information and the ratio to obtain a target biological object.

[0125] In this embodiment, the reference biological object in the reference image is used to adjust the limbs, limb attributes and other information of the initial biological object in the image to be converted to obtain the target biological object, and the image to be converted is converted based on the target biological object to obtain the biological image, thereby achieving the purpose of quickly adjusting the initial biological object in the image to be converted, thereby solving the technical problem of low matching degree of the generated animation and achieving the technical effect of improving the matching degree of the generated animation.

[0126] According to an embodiment of the present application, a processor is further provided, and the processor is used to run a program, wherein when the program is run by the processor, the image adjustment method in the above embodiment is executed.

[0127] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0128] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image adjustment method: obtaining an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; adjusting the limbs of the initial biological object based on reference limb information of the reference biological object in the reference image to obtain a first biological object; determining first limb attribute information of the first biological object, and second limb attribute information of the reference biological object; adjusting the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object; converting the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

[0129] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image adjustment method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image adjustment method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] Optionally, the processor may also execute program codes of the following steps: extracting the image to be converted to obtain initial limb information of the initial biological object, and extracting the reference biological object to obtain reference limb information.

[0131] Optionally, the processor may also execute program code of the following steps: determining affine transformation parameters based on reference limb information and initial limb information, wherein the affine transformation parameters are used to adjust properties of the limbs of the initial biological object; and adjusting the limbs according to the affine transformation parameters to obtain a first biological object.

[0132] Optionally, the processor can also execute the program code of the following steps: obtaining multiple first position information of multiple key points in the first biological object, and multiple second position information of multiple reference key points in the reference biological object; determining the first limb attribute information based on the multiple first position information, and determining the second limb attribute information based on the multiple second position information.

[0133] Optionally, the processor may also execute program code of the following steps: determining a ratio between first limb attribute information and second limb attribute information of the same limb position; in response to the ratio not satisfying a preset range, adjusting the first biological object to obtain a target biological object.

[0134] Optionally, the processor may also execute program code of the following steps: in response to the ratio not satisfying a preset range, determining angle information corresponding to the limb, wherein the angle information is used to characterize the direction of the limb; and adjusting the first biological object based on the angle information and the ratio to obtain a target biological object.

[0135] By adopting the embodiment of the present application, the reference biological object in the reference image is used to adjust the limbs, limb attributes and other information of the initial biological object in the image to be converted to obtain the target biological object, and the image to be converted is converted based on the target biological object to obtain the biological image, thereby achieving the purpose of quickly adjusting the initial biological object in the image to be converted, thereby solving the technical problem of low matching degree of the generated animation and achieving the technical effect of improving the matching degree of the generated animation.

[0136] According to an embodiment of the present application, a computer program product is further provided. The computer program product includes computer instructions, wherein when the computer instructions are executed by a processor, the image adjustment method in the above embodiment is implemented.

[0137] An embodiment of the present application may provide an electronic device, which may include a memory and a processor.

[0138] Figure 6 It is a block diagram of an electronic device according to an image adjustment method of an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0139] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 to a random access memory (RAM) 603. In RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0140] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0141] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a verification method for data. For example, in some embodiments, the verification method for data may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the verification method for data described above may be executed. Alternatively, in other embodiments, the computing unit 601 may be configured to execute the data verification method in any other appropriate manner (for example, by means of firmware).

[0142] According to an embodiment of the present application, a method for adjusting an image is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0143] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0145] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0146] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or an LCD (liquid crystal display, monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a path ball) through which the user can provide input to the computer. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0147] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a Local Area Network (LAN), a Wide Area Network (WAN), and the Internet.

[0148] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0149] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0150] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0152] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0153] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0154] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0155] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for adjusting an image, characterized in that: include: Acquire an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; Based on the reference limb information of the reference biological object in the reference image, adjusting the limbs of the initial biological object to obtain a first biological object; determining first limb attribute information of the first biological object, and second limb attribute information of the reference biological object; Based on the second limb attribute information and the first limb attribute information, adjusting the first biological object to obtain a target biological object; The image to be converted is converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

2. The method according to claim 1, characterized in that: The method further comprises: Extracting the image to be converted to obtain initial limb information of the initial biological object, and extracting the reference biological object to obtain reference limb information; The step of adjusting the limbs of the initial biological object based on the reference limb information of the reference biological object in the reference image to obtain a first biological object includes: Determining affine transformation parameters based on the reference limb information and the initial limb information, wherein the affine transformation parameters are used to adjust the properties of the limb of the initial biological object; The limb is adjusted according to the affine transformation parameters to obtain the first biological object.

3. The method according to claim 1, characterized in that The determining of the first limb attribute information of the first biological object and the second limb attribute information of the reference biological object comprises: Acquire a plurality of first position information of a plurality of key points in the first biological object, and a plurality of second position information of a plurality of reference key points in the reference biological object; The first limb attribute information is determined based on the plurality of first position information, and the second limb attribute information is determined based on the plurality of second position information.

4. The method according to claim 1, characterized in that: The step of adjusting the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object includes: Determine a ratio between the first limb attribute information and the second limb attribute information at the same limb position; In response to the ratio not satisfying a preset range, the first biological object is adjusted to obtain the target biological object.

5. The method according to claim 4, characterized in that In response to the ratio not satisfying a preset range, adjusting the first biological object to obtain the target biological object includes: In response to the ratio not satisfying the preset range, determining angle information corresponding to the limb, wherein the angle information is used to characterize the direction of the limb; Based on the angle information and the ratio, the first biological object is adjusted to obtain the target biological object.

6. An image adjustment device, characterized in that: include: an acquisition unit, configured to acquire an image to be converted whose image content includes an initial biological object, and a reference image whose image content includes a reference biological object, wherein the reference biological image is used to adjust the initial biological object; A first adjustment unit, configured to adjust the limbs of the initial biological object based on the reference limb information of the reference biological object in the reference image to obtain a first biological object; a determining unit, configured to determine first limb attribute information of the first biological object and second limb attribute information of the reference biological object; A second adjustment unit, configured to adjust the first biological object based on the second limb attribute information and the first limb attribute information to obtain a target biological object; A conversion unit is used to convert the image to be converted based on the target biological object to obtain a target image, wherein the similarity between the biological object in the target image and the reference biological object is higher than a similarity threshold.

7. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method described in any one of claims 1 to 5 when run by the processor.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 5 when executed.

9. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 5 when being executed by a processor.

Images