Image processing method and device, data processing method

Abstract

The present application discloses an image processing method and device, and a data processing method. The method includes: receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is the posture of the target object after conversion from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image of the target object when converted to the target posture; using a global residual network model to process the second appearance image and the first appearance image, generating a third appearance image of the target object in the target posture, the third appearance image having more detailed features than the second appearance image. The present application solves the technical problem that there is information missing and a single texture effect in the process of posture migration due to the current model that uses a single-stage network to migrate the image of a model in a current posture to other target postures.

Description

Technical Field

[0001] The present application relates to the field of image synthesis, and more specifically, to an image processing method and device, and a data processing method. Background Art

[0002] In e-commerce scenarios, users want to see clothes in multiple poses and angles, but more and more comprehensive poses mean greater operation and time investment. To this end, a generative model is trained to transfer the image of the model in the current pose to other target poses, so as to achieve a comprehensive display of clothing and other products.

[0003] Currently, the model for migrating the image of a model in a current pose to other target poses generally adopts a deep learning-based generative adversarial model (GAN). This human pose transfer model uses a single-stage network to transfer the human pose in one step, but the appearance details such as hair, clothing texture, shoes, etc. are severely lost. This single-stage network can perform well in high-level semantics such as human pose transfer, but it has poor recovery of detailed textures of low-level semantic information.

[0004] The current model for migrating an image of a model in a current pose to another target pose uses a single-stage network. There are problems of missing information and single texture effect in the pose migration process, and no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide an image processing method and device, and a data processing method, so as to at least solve the technical problems of missing information and single texture effect in the process of posture migration caused by the current model that migrates the image of a model in a current posture to other target postures using a single-stage network.

[0006] According to one aspect of an embodiment of the present application, there is provided an image processing method, including: acquiring a first image of a target object and a target rotation angle; rotating the first image according to the target rotation angle to generate a second image; generating a target image corresponding to the target object based on the first image and the second image, the target image having more detail features than the first image, wherein both the second image and the target image include posture information of the target object after being rotated by the target rotation angle.

[0007] According to another aspect of an embodiment of the present application, another image processing method is provided, including: obtaining first multimedia data corresponding to a target object, and conversion description data for converting a target part in the first multimedia data, wherein the target part is any part of the target object; generating a first image corresponding to the target object based on the first multimedia data and the conversion description data, wherein the first image includes an image after the target part is converted according to the conversion description data; generating a second image corresponding to the target object based on the first multimedia data and the first image, wherein the second image includes an image after the target part is converted according to the conversion description data, and detail features of the target part in the second image are more than the detail features of the target part in the first image.

[0008] According to another aspect of an embodiment of the present application, another image processing method is provided, including: acquiring multiple images of a first target object acquired at adjacent sampling moments; determining posture change information of the first target object in the multiple images; acquiring a first image of a second target object; converting the first image according to the posture change information to generate a second image; and generating a target image corresponding to the second target object based on the first image and the second image, the target image having more detail features than the first image.

[0009] According to another aspect of an embodiment of the present application, another image processing method is provided, including: receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after being converted from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image of the target object when converted to the target posture; using a global residual network model to process the second appearance image and the first appearance image, generating a third appearance image of the target object in the target posture, the third appearance image having more detail features than the second appearance image.

[0010] According to another aspect of an embodiment of the present application, another image processing method is provided, including: when a target object switches from a first posture to a second posture, obtaining a first appearance image of the target object when it is in the first posture and posture information of the second posture; based on the first appearance image and the posture information of the second posture, generating a second appearance image of the target object when it is in the second posture; using a global residual network model to process the second appearance image and the first appearance image to generate a third appearance image of the target object in the second posture, the third appearance image having more detail features than the second appearance image.

[0011] According to another aspect of an embodiment of the present application, another image processing method is provided, including: generating a first appearance image of a target object, wherein the first appearance image is an appearance image of the target object in a target posture; using a global residual network model to process the first appearance image and a default appearance image of the target object to generate a second appearance image of the target object in the target posture, the second appearance image having more detail features than the first appearance image.

[0012] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining first multimedia data and conversion description data corresponding to a target object, wherein the conversion description data is used to convert the first multimedia data; generating a first image corresponding to the target object based on the first multimedia data and the conversion description data; and generating a second image corresponding to the target object based on the first multimedia data and the first image, wherein the detail features of the second image are more than the detail features of the first image; wherein the first multimedia data includes a first posture of the target object, and the first image and the second image both include a second posture of the target object.

[0013] According to another aspect of the embodiments of the present application, there is also provided an image processing device, including: a receiving module, used to receive a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is the posture of the target object after being converted from the current posture; a generating module, used to generate a second appearance image of the target object when it is converted to a target posture based on the first appearance image and the posture information of the target posture; and a processing module, used to process the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in the target posture, wherein the third appearance image has more detail features than the second appearance image.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, characterized in that the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above image processing method.

[0015] According to another aspect of the embodiment of the present application, a computing device is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after being converted from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image of the target object when it is converted to a target posture; using a global residual network model to process the second appearance image and the first appearance image, generating a third appearance image of the target object in the target posture, the third appearance image having more detailed features than the second appearance image.

[0016] In an embodiment of the present application, a first appearance image of a target object and posture information of a target posture are received, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after conversion from the current posture; based on the first appearance image and the posture information of the target posture, a second appearance image of the target object when converted to the target posture is generated; the second appearance image and the first appearance image are processed using a global residual network model to generate a third appearance image of the target object in the target posture, wherein the third appearance image has more detail features than the second appearance image, and a two-stage appearance conversion learning is adopted, wherein a rough target human appearance image is generated in the first stage, and a target human appearance with more texture details is generated from coarse to fine based on the rough appearance of the first stage, thereby achieving the technical effect of alleviating the problem of missing detailed information and single texture effect generated in the current single-stage appearance conversion model, and improving the display effect of the clothes on the model, thereby solving the technical problem of missing information and single texture effect in the process of posture migration caused by the current model that migrates the image of the model in the current posture to other target postures using a single-stage network. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method is shown;

[0019] Figure 2 is a flow chart of an image processing method according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of a training posture image migration model according to an embodiment of the present application;

[0021] Figure 4 is a schematic diagram of an application of a global residual network model according to an embodiment of the present application;

[0022] Figure 5 is a flowchart of another image processing method according to an embodiment of the present application;

[0023] Figure 6 is a flowchart of another image processing method according to an embodiment of the present application;

[0024] Figure 7 is a structural diagram of an image processing device according to an embodiment of the present application;

[0025] Figure 8 is a structural block diagram of a computer terminal according to an embodiment of the present application;

[0026] Fig. 9 is a flow chart of a data processing method according to an embodiment of the present application;

[0027] Fig.10 is a flowchart of another image processing method according to an embodiment of the present application;

[0028] Fig.11 is a flowchart of another image processing method according to an embodiment of the present application;

[0029] Fig.12 is a flowchart of another image processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0033] GAN: Deep Learning Generative Adversarial Models.

[0034] pix2pix: Image-to-Image Translation with Conditional Adversarial Networks.

[0035] pix2pixHD: High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs.

[0036] Example 1

[0037] According to an embodiment of the present application, an embodiment of a method for processing an image is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing an image processing method. Figure 1As shown, the computer terminal 10 (or mobile device 10) may include one or more (102a, 102b, ..., 102n are used to illustrate) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0039] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the image processing method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned image processing method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0043] Under the above operating environment, this application provides Figure 2 The processing method of the image shown. Figure 2 is a flowchart of an image processing method according to Embodiment 1 of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0044] Step S202 , receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after conversion from the current posture.

[0045] With reference to the e-commerce application scenario mentioned in the background technology, the target object in step S202 is a model displaying clothing.

[0046] The first appearance image is an image of the model in the current posture (for example, the current model is facing 45 degrees to the left front), and the target posture is the posture of the model facing 45 degrees to the right front. The so-called posture transfer is the posture image of the model facing 45 degrees to the left front after the deep neural network model is processed. It should be noted that 45 degrees is just an example, and other angles or postures can also be used.

[0047] Step S204: generating a second appearance image when the target object is converted into the target posture based on the first appearance image and the posture information of the target posture.

[0048] This step is the first stage of the two-stage appearance transfer learning. In this step, the first appearance image of the target object and the posture information of the target posture are input into the neural network model to generate a rough target human appearance image.

[0049] Step S206: Process the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in a target posture, wherein the third appearance image has more detailed features than the second appearance image.

[0050] Step S206 is the second stage of the two-stage appearance transfer learning. In this step, the rough target human appearance image and the first appearance image of the target object generated in the first stage are processed using the global residual network model to generate a target human appearance with more texture details from coarse to fine.

[0051] According to an optional embodiment of the present application, when executing step S206, the rough image obtained in step S204 is first processed using the global residual network model, and then the obtained processing result is superimposed with the first appearance image of the target object to generate a final target image of the target object, that is, an image of the target object in the target posture, such as Figure 3 shown.

[0052] Figure 3 It is a schematic diagram of an application of a global residual network model according to an embodiment of the present application. A first appearance image (image1) and posture information of a target posture are processed using a first neural network model, and the generated second appearance image (image2', the second appearance image is a rough image) is input into a constructed detail residual network G2 for processing to obtain a processing result, and then a third appearance image (image2") of the target object in the target posture is generated together with the first appearance image (image1).

[0053] It should be noted that the neural network model used to generate a rough image of the target object converted to the target posture in the above step S204 and the global residual network model in step S206 can be replaced by other networks. For example, the global residual network model can be replaced by image translation models such as pix2pix and pix2pixHD.

[0054] Through the above method, by adopting a two-stage appearance transfer learning, the first stage generates a rough target human appearance map, and the second stage generates a target human appearance with more texture details from coarse to fine based on the rough appearance of the first stage, thereby alleviating the problems of missing generated detail information and single texture effect in the current single-stage appearance transfer model, and improving the technical effect of improving the display effect of the clothes on the model.

[0055] According to an optional embodiment of the present application, step S204 can be implemented by the following method: using a first neural network model to process the first appearance image and the posture information of the target posture to generate a second appearance image, wherein the neural network model is trained using a training sample set.

[0056] In an optional embodiment of the present application, before using the first neural network model to process the first appearance image and the posture information of the target posture to generate the second appearance image, it is also necessary to use a training sample set to train the model parameters of the fitting network model to generate the first neural network model, wherein the first neural network model is used to generate the second appearance image of the target image to be identified.

[0057] In another optional embodiment of the present application, before using the training sample set to train the model parameters of the fitting network model, it is also necessary to construct the training sample set, wherein the training sample set includes multiple groups of training samples, wherein constructing the training sample set includes: obtaining multiple posture images; randomly extracting a first sample picture and a second sample picture with the same sample object from the multiple posture images, wherein the first sample picture is an image of the sample object before being transformed into a target posture, and the second sample picture is an image of the sample object after being transformed into the target posture; extracting a posture image of the target posture from the second sample picture; and constructing a group of training samples based on the first sample picture, the second sample picture and the extracted posture image.

[0058] Figure 4 is a schematic diagram of a training posture image migration model according to an embodiment of the present application, such as Figure 4 As shown:

[0059] First, construct a training set (i.e., the training sample set mentioned above), collect multiple pose image groups of the same model clothing, and multiple groups of images constitute the training data set. Randomly select two images of the same model clothing image group, the first sample image (image1) and the second sample image (image2), and extract 18 point poses (other numbers of point poses can also be extracted) from the second sample image (image2) as the estimated response image (pose2, the pose image of the target pose), where image1 is the input reference model image, pose2 is the target pose, and image2 is the target pose supervision image to be generated. That is, image1, image2 and pose2 are a set of training samples.

[0060] After the training sample set is constructed, the first neural network model G1 for the first stage content learning is constructed using the training sample set, and the parameters of the first neural network model G1 are trained and fitted using the training samples, and the training is performed until the network converges.

[0061] After the first neural network model G1 is trained, the first sample image (image1) and the pose image (pose2) of the target pose are input into the first neural network model G1 for processing to generate a rough target human appearance image image2'.

[0062] In another optional embodiment of the present application, before executing step S206, it is also necessary to construct a global residual network model, which step includes: using a rough image of the second sample image and the first sample image, and using the second sample image as a supervision image to fit and train the model parameters of the global residual network model until the global residual network model converges.

[0063] Construct the second-stage detail residual network G2, use image2' and image1 generated by G1 as input, and use image2 as supervision to fit and train G2 parameters to generate the final refined target human appearance image (image2"). The second stage is detail generation, using a global human residual model. The network task only needs to focus on the residual between the learning target supervision image and the target image generated in the first stage, that is, f(x)-y=r; wherein f(x) is the supervision image image2, y is the rough target human image2' generated by G1, and r is the learned detail residual image. G2 is jointly trained on the basis of G1. The above method provided in the embodiment of the present application uses a two-stage posture transfer network. The first stage focuses on learning the overall appearance action transfer of the human body and the coarse-grained limb and facial transformation; after the target human posture image image2' is obtained in the first stage, the second stage uses a global residual network to learn the appearance detail residuals of the supervision image image2 and image2'. The final overall posture learning and appearance detail learning are fully combined.

[0064] It should be noted that the open source software pytorch is used in the specific implementation of the embodiment of the present application, and only the binary program is used, and no code modification is involved. Pytorch is an open source Python machine learning library based on Torch, which is used for applications such as natural language processing.

[0065] According to an optional embodiment of the present application, extracting the posture image of the target posture from the second sample image includes: extracting a plurality of feature points located at different positions of the target object from the second sample image; connecting the plurality of feature points according to the positions of the plurality of feature points in the human body and the correlation relationship between the plurality of feature points to obtain the posture image of the target posture.

[0066] In this step, a number of feature points are extracted from the second sample image (image2), and then lines are connected according to the positions of the feature points in the human body, so as to ensure that the obtained posture image can fully reflect the contour of the human body as much as possible.

[0067] In an optional embodiment, the extracted feature points can be connected according to the correlation between the feature points. For example, the feature points on the same arm of the human body can be connected, and then connected with other feature points. This can also ensure that the obtained posture image can fully reflect the contour of the human body as much as possible.

[0068] In an optional embodiment of the present application, step S202 can be implemented by the following method: receiving a clothing matching image of the target object currently captured by the image acquisition device, and using the clothing matching image as the first appearance image, wherein the clothing matching image includes: the target object and clothing information matched with the target object; receiving posture information of the target posture input through the human-computer interaction interface of the image acquisition device.

[0069] The embodiment of the present application is applied in an e-commerce application scenario. The target object is a model showing clothes. Step S202 is performed to obtain not only the image of the model showing clothes, but also the image of the model's clothing matching. The posture information of the target posture can be input through the human-computer interaction interface of the image acquisition device.

[0070] In an optional embodiment of the present application, after step S206 is executed, feedback information on the third appearance image is received; when the feedback information is the first result, the brand to which the clothing matching image belongs is identified, and the target clothing is selected from the clothing database corresponding to the brand and input into the second neural network model, wherein the second neural network model is trained based on the historical clothing matching information of the target object; when the feedback information is the second result, the third appearance image is stored in the clothing matching database corresponding to the target object.

[0071] After generating a second appearance image of the target object in the target posture, feedback information on the second appearance image is received, and the feedback information is used to evaluate the user's satisfaction with the generated second appearance image. If the user is not satisfied with the generated second appearance image (the first result above), it is necessary to determine the brand of the clothing matching image, and then select the target clothing from the database corresponding to the brand and input it into the neural network model for processing, and use the processing result as the second appearance image, and the neural network model is trained based on the historical clothing matching information of the target object.

[0072] If the user is satisfied with the generated second appearance image, the second appearance image is stored in the clothing matching database, and when there is a subsequent clothing matching demand, it can be directly called without the need to be processed by the neural network model; alternatively, the images in the clothing matching database are used as training data to train the second neural network model, so that the clothing matching information results predicted by the second neural network model are more accurate.

[0073] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that the image processing method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0075] Example 2

[0076] Figure 5 is a flowchart of another image processing method according to an embodiment of the present application, such as Figure 5 As shown, the method comprises the following steps:

[0077] Step S502: when the target object switches from the first posture to the second posture, a first appearance image of the target object in the first posture and posture information of the second posture are obtained.

[0078] It should be noted that the above-mentioned posture information of the second posture refers to a number of point postures extracted from the image of the second posture.

[0079] Step S504: generating a second appearance image when the target object is in the second posture based on the first appearance image and the posture information of the second posture.

[0080] This step is the first stage of a two-stage appearance transfer learning. In this step, the first appearance image and the posture information of the second posture of the target object are input into the neural network model to generate a rough image of the target object in the second posture.

[0081] Step S506: Process the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in a second posture, wherein the third appearance image has more detailed features than the second appearance image.

[0082] Step S506 is the second stage of the two-stage appearance transfer learning. In this step, the rough image generated in the first stage is processed using the global residual network model to generate an appearance image of the target object in the second posture with more texture details from coarse to fine.

[0083] It should be noted that the target object mentioned above can refer to the model showing clothes mentioned in the e-commerce application scenario mentioned in the background technology. It can also be other target objects for posture transfer. In addition, Figure 5 The preferred implementation of the illustrated embodiment can be found in Figure 2 The relevant description of the illustrated embodiment will not be repeated here.

[0084] Example 3

[0085] Figure 6 is a flow chart of an image processing method according to an embodiment of the present application, such as Figure 6 As shown, the method comprises the following steps:

[0086] Step S602: Generate a first appearance image of the target object, wherein the first appearance image is an appearance image of the target object in a target posture.

[0087] Step S604: Use a global residual network model to process the first appearance image and the default appearance image of the target object to generate a second appearance image of the target object in a target posture, wherein the second appearance image has more detail features than the first appearance image.

[0088] Step S604 is the second stage of the two-stage appearance transfer learning. In this step, the rough target human appearance map generated in the first stage is processed using the global residual network model to generate a target human appearance with more texture details from coarse to fine.

[0089] According to an optional embodiment of the present application, when the target object changes from the current posture to the target posture, the second appearance image is an appearance image refined based on the first appearance image, wherein the default appearance image is a posture image before the target object is converted to the target posture.

[0090] It should be noted that Figure 6 The preferred implementation of the illustrated embodiment can be found in Figure 2 The relevant description of the illustrated embodiment will not be repeated here.

[0091] Example 4

[0092] According to an embodiment of the present application, a device for implementing the above-mentioned image processing method is also provided, such as Figure 7 As shown, the device comprises:

[0093] A receiving module 70 is used to receive a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after conversion from the current posture;

[0094] The above-mentioned target object is a model displaying clothing in the e-commerce application scenario mentioned in the reference background technology.

[0095] The first appearance image is an image of the model in the current posture (for example, the current model is facing 45 degrees to the left front), and the target posture is the posture of the model facing 45 degrees to the right front. The so-called posture transfer is the posture image of the model facing 45 degrees to the left front after the deep neural network model is processed. It should be noted that 45 degrees is just an example, and other angles or postures can also be used.

[0096] A generating module 72, configured to generate a second appearance image when the target object is converted to a target posture based on the first appearance image and the posture information of the target posture;

[0097] The generation module 72 is used to execute the steps of the first stage of the two-stage appearance conversion learning. In this step, the first appearance image of the target object and the posture information of the target posture are input into the neural network model to generate a rough target human appearance image.

[0098] A processing module 74 is used to process the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in a target posture, wherein the third appearance image has more detailed features than the second appearance image;

[0099] The processing module 74 is used to execute the steps of the second stage of the two-stage appearance transfer learning. In this step, the rough image obtained by the generation module 72 is first processed using the global residual network model, and then the processing result is superimposed on the first appearance image of the target object to generate the final target image of the target object, that is, the image of the target object in the target posture.

[0100] It should be noted that the receiving module 70, the generating module 72 and the processing module 74 correspond to steps S202 to S206 in Example 1, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the above-mentioned Example 1. It should be noted that the above-mentioned modules, as part of the device, can be run in the computer terminal 10 provided in Example 1.

[0101] Example 5

[0102] The embodiment of the present application may provide a computer terminal, which may be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0103] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0104] In this embodiment, the above-mentioned computer terminal can execute the program code of the following steps in the image processing method of the application: receiving a first appearance image of the target object and posture information of the target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is the posture of the target object after being converted from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image when the target object is converted to the target posture; using a global residual network model to process the second appearance image and the first appearance image, generating a third appearance image of the target object in the target posture, the third appearance image having more detailed features than the second appearance image.

[0105] Optionally, Figure 8 is a structural block diagram of a computer terminal according to an embodiment of the present application. Figure 8 As shown, the computer terminal 80 may include: one or more (only one is shown in the figure) processors 802, a memory 804, a radio frequency module, an audio module and a display screen.

[0106] Among them, the memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the image processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned image processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0107] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: receiving a first appearance image of the target object and posture information of the target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is the posture of the target object after being converted from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image when the target object is converted to the target posture; using a global residual network model to process the second appearance image and the first appearance image, generating a third appearance image of the target object in the target posture, the third appearance image having more detailed features than the second appearance image.

[0108] Optionally, the processor may also execute program code of the following steps: using a first neural network model to process the first appearance image and posture information of the target posture to generate a second appearance image, wherein the neural network model is trained using a training sample set.

[0109] Optionally, the processor may also execute the program code of the following steps: using the training sample set to train model parameters of the fitting network model to generate a first neural network model, wherein the first neural network model is used to generate a second appearance image of the target image to be identified.

[0110] Optionally, the processor may also execute the program code of the following steps: obtaining multiple posture images; randomly extracting a first sample image and a second sample image having the same sample object from the multiple posture images, wherein the first sample image is an image of the sample object before being transformed into a target posture, and the second sample image is an image of the sample object after being transformed into the target posture; extracting a posture image of the target posture from the second sample image; and constructing a set of training samples based on the first sample image, the second sample image and the extracted posture images.

[0111] Optionally, the processor may also execute the program code of the following steps: using a rough image of the second sample image and the first sample image, and using the second sample image as a supervision image to perform fitting training on model parameters of the global residual network model until the global residual network model converges.

[0112] Optionally, the processor may also execute the program code of the following steps: extracting multiple feature points located at different positions of the target object from the second sample image; connecting the multiple feature points according to the positions of the multiple feature points in the human body and the association relationship between the multiple feature points to obtain a posture image of the target posture.

[0113] Optionally, the processor may also execute program code for the following steps: receiving a clothing matching image of a target object currently captured by an image acquisition device, and using the clothing matching image as a first appearance image, wherein the clothing matching image includes: the target object and clothing information matched with the target object; and receiving posture information of a target posture input through a human-computer interaction interface of the image acquisition device.

[0114] Optionally, the processor may also execute program code of the following steps: receiving feedback information on the third appearance image; when the feedback information is the first result, identifying the brand to which the clothing matching image belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting the target clothing into a second neural network model, wherein the second neural network model is trained based on historical clothing matching information of the target object; when the feedback information is the second result, storing the third appearance image in a clothing matching database corresponding to the target object.

[0115] According to an optional embodiment of the present application, the above-mentioned processor can also call the information and application stored in the memory through the transmission device to perform the following steps: when the target object switches from a first posture to a second posture, obtain a first appearance image of the target object in the first posture and posture information of the second posture; based on the first appearance image and the posture information of the second posture, generate a second appearance image of the target object in the second posture; use a global residual network model to process the second appearance image and the first appearance image to generate a third appearance image of the target object in the second posture, and the third appearance image has more detail features than the second appearance image.

[0116] According to an optional embodiment of the present application, the above-mentioned processor can also call the information and application stored in the memory through the transmission device to perform the following steps: generate a first appearance image of the target object, wherein the first appearance image is an appearance image of the target object in a target posture; use a global residual network model to process the first appearance image and the default appearance image of the target object to generate a second appearance image of the target object in the target posture, and the second appearance image has more detailed features than the first appearance image.

[0117] Optionally, the processor may also execute program code of the following steps: when the target object changes from the current posture to the target posture, the second appearance image is an appearance image refined based on the first appearance image, wherein the default appearance image is a posture image before the target object is converted to the target posture.

[0118] According to an optional embodiment of the present application, the above-mentioned processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain the first multimedia data and conversion description data corresponding to the target object, wherein the conversion description data is used to convert the first multimedia data; generate a first image corresponding to the target object based on the first multimedia data and the conversion description data; and generate a second image corresponding to the target object based on the first multimedia data and the first image, the detail features of the second image are more than the detail features of the first image; wherein the first multimedia data includes a first posture of the target object, and the first image and the second image include a second posture of the target object.

[0119] According to an optional embodiment of the present application, the above-mentioned processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain a first image and a target rotation angle of the target object; rotate the first image according to the target rotation angle to generate a second image; based on the first image and the second image, generate a target image corresponding to the target object, the target image has more detail features than the first image, wherein the second image and the target image both include posture information of the target object after being rotated by the target rotation angle.

[0120] According to an optional embodiment of the present application, the above-mentioned processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain first multimedia data corresponding to the target object, and conversion description data for converting the target part in the first multimedia data, wherein the target part is any part of the target object; based on the first multimedia data and the conversion description data, generate a first image corresponding to the target object, wherein the first image includes an image after the target part is converted according to the conversion description data; based on the first multimedia data and the first image, generate a second image corresponding to the target object, wherein the second image includes an image after the target part is converted according to the conversion description data, and the detail features of the target part in the second image are more than the detail features of the target part in the first image.

[0121] Optionally, the processor may also execute program codes of the following steps: receiving a conversion instruction input from a target object, wherein the conversion instruction carries conversion description data; receiving voice information of the target object, and identifying the conversion description data from the voice information.

[0122] According to an optional embodiment of the present application, the above-mentioned processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain multiple images of the first target object collected at adjacent sampling moments; determine the posture change information of the first target object in the multiple images; obtain the first image of the second target object; convert the first image according to the posture change information to generate a second image; based on the first image and the second image, generate a target image corresponding to the second target object, and the target image has more detailed features than the first image.

[0123] By adopting the embodiment of the present application, a solution for image processing is provided. By adopting two-stage appearance conversion learning, the first stage generates a rough target human appearance map, and the second stage generates a target human appearance with more texture details from coarse to fine based on the rough appearance of the first stage, thereby alleviating the problem of missing detailed information and single texture effect in the current single-stage appearance conversion model, and improving the display effect of the clothes on the model, thereby solving the technical problem of missing information and single texture effect in the process of posture migration caused by the current model that migrates the image of the model in the current posture to other target postures using a single-stage network.

[0124] It can be understood by those skilled in the art that Figure 8 The structure shown is for illustration only, and the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, a mobile Internet device (Mobile Internet Devices, MID), a PAD, or other terminal devices. Figure 8 The structure of the electronic device is not limited. For example, the computer terminal 80 may also include Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 8 Different configurations are shown.

[0125] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0126] The embodiment of the present application further provides a storage medium. Optionally, in this embodiment, the storage medium can be used to store the program code executed by the image processing method provided in the above embodiment 1.

[0127] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0128] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after being converted from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image of the target object when it is converted to a target posture; using a global residual network model to process the second appearance image and the first appearance image to generate a third appearance image of the target object in the target posture, the third appearance image having more detailed features than the second appearance image.

[0129] Optionally, the storage medium is also configured to store program code for executing the following steps: using a first neural network model to process the first appearance image and posture information of the target posture to generate a second appearance image, wherein the neural network model is trained using a training sample set.

[0130] Optionally, the storage medium is also configured to store program code for executing the following steps: using a training sample set to train model parameters of a fitting network model to generate a first neural network model, wherein the first neural network model is used to generate a second appearance image of a target image to be identified.

[0131] Optionally, the storage medium is also configured to store program codes for executing the following steps: obtaining multiple posture images; randomly extracting a first sample image and a second sample image having the same sample object from the multiple posture images, wherein the first sample image is an image of the sample object before being transformed into a target posture, and the second sample image is an image of the sample object after being transformed into the target posture; extracting a posture image of the target posture from the second sample image; and constructing a set of training samples based on the first sample image, the second sample image and the extracted posture images.

[0132] Optionally, the storage medium is also configured to store program code for executing the following steps: using a rough image of the second sample image and the first sample image, and using the second sample image as a supervision image to perform fitting training on model parameters of the global residual network model until the global residual network model converges.

[0133] Optionally, the storage medium is also configured to store program codes for executing the following steps: extracting multiple feature points located at different positions of the target object from the second sample image; connecting the multiple feature points according to the positions of the multiple feature points in the human body and the association relationship between the multiple feature points to obtain a posture image of the target posture.

[0134] Optionally, the storage medium is also configured to store program codes for executing the following steps: receiving a clothing matching image of a target object currently captured by an image acquisition device, and using the clothing matching image as a first appearance image, wherein the clothing matching image includes: the target object and clothing information matched with the target object; receiving posture information of a target posture input through a human-computer interaction interface of the image acquisition device.

[0135] Optionally, the storage medium is also configured to store program codes for executing the following steps: receiving feedback information on the third appearance image; when the feedback information is the first result, identifying the brand to which the clothing matching image belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on historical clothing matching information of the target object; when the feedback information is the second result, storing the third appearance image in a clothing matching database corresponding to the target object.

[0136] According to an optional embodiment of the present application, the above-mentioned storage medium is also configured to store program codes for executing the following steps: when the target object switches from a first posture to a second posture, obtaining a first appearance image of the target object when it is in the first posture and posture information of the second posture; based on the first appearance image and the posture information of the second posture, generating a second appearance image of the target object when it is in the second posture; using a global residual network model to process the second appearance image and the first appearance image of the coarse image, generating a third appearance image of the target object in the second posture, the third appearance image having more detail features than the second appearance image.

[0137] According to an optional embodiment of the present application, the above-mentioned storage medium is also configured to store program code for executing the following steps: generating a first appearance image of the target object, wherein the first appearance image is an appearance image of the target object in a target posture; using a global residual network model to process the first appearance image and a default appearance image of the target object to generate a second appearance image of the target object in the target posture, the second appearance image having more detail features than the first appearance image.

[0138] Optionally, the storage medium is also configured to store program code for executing the following steps: when the target object changes from a current posture to a target posture, the second appearance image is an appearance image that is refined based on the first appearance image, wherein the default appearance image is a posture image before the target object is converted to the target posture.

[0139] According to an optional embodiment of the present application, the above-mentioned storage medium is also configured to store program codes for executing the following steps: obtaining first multimedia data and conversion description data corresponding to the target object, wherein the conversion description data is used to convert the first multimedia data; generating a first image corresponding to the target object based on the first multimedia data and the conversion description data; and generating a second image corresponding to the target object based on the first multimedia data and the first image, the second image having more detail features than the first image; wherein the first multimedia data includes a first posture of the target object, and the first image and the second image include a second posture of the target object.

[0140] According to an optional embodiment of the present application, the above-mentioned storage medium is also configured to store program codes for executing the following steps: obtaining a first image and a target rotation angle of the target object; rotating the first image according to the target rotation angle to generate a second image; based on the first image and the second image, generating a target image corresponding to the target object, the target image having more detail features than the first image, wherein the second image and the target image both include posture information after the target object is rotated by the target rotation angle.

[0141] According to an optional embodiment of the present application, the above-mentioned storage medium is also configured to store program codes for executing the following steps: obtaining first multimedia data corresponding to the target object, and conversion description data for converting a target part in the first multimedia data, wherein the target part is any part of the target object; generating a first image corresponding to the target object based on the first multimedia data and the conversion description data, wherein the first image includes an image after the target part is converted according to the conversion description data; generating a second image corresponding to the target object based on the first multimedia data and the first image, wherein the second image includes an image after the target part is converted according to the conversion description data, and the detail features of the target part in the second image are more than the detail features of the target part in the first image.

[0142] Optionally, the storage medium is also configured to store program codes for executing the following steps: receiving a conversion instruction input from a target object, wherein the conversion instruction carries conversion description data; receiving voice information of the target object, and identifying the conversion description data from the voice information.

[0143] According to an optional embodiment of the present application, the above-mentioned storage medium is also configured to store program codes for executing the following steps: acquiring multiple images of the first target object acquired at adjacent sampling moments; determining posture change information of the first target object in the multiple images; acquiring a first image of the second target object; converting the first image based on the posture change information to generate a second image; based on the first image and the second image, generating a target image corresponding to the second target object, the target image having more detail features than the first image.

[0144] Example 6

[0145] Fig. 9 is a flow chart of a data processing method according to an embodiment of the present application, such as Fig. 9 As shown, the method comprises the following steps:

[0146] Step S902: Acquire first multimedia data and conversion description data corresponding to the target object, wherein the conversion description data is used to convert the first multimedia data.

[0147] According to an optional embodiment of the present application, the target object in step S902 is a model for displaying clothing in an e-commerce application scenario, and the first multimedia data may be an image of the clothing in a current posture (for example, the current model faces 45 degrees to the left front), which is equivalent to Figure 4 The first sample image (image1) in the conversion description data is equivalent to Figure 4 The estimated response map (pose2, the pose image of the target pose) extracted from the second sample image (image2) in .

[0148] According to an optional embodiment of the present application, the first multimedia data in step S902 includes: picture or video data. The first multimedia information may also be video data, for example, video data of clothing displayed by models in an e-commerce application scenario.

[0149] Step S904: Generate a first image corresponding to the target object based on the first multimedia data and the conversion description data.

[0150] According to an optional embodiment of the present application, when executing step S904, the first multimedia data and the description conversion data are input into the neural network model to generate a first image corresponding to the target object. The first image is a rough appearance image of the target human body, which is equivalent to Figure 4 image2' in.

[0151] Step S906, based on the first multimedia data and the first image, generate a second image corresponding to the target object, the second image having more detail features than the first image; wherein the first multimedia data includes a first posture of the target object, and the first image and the second image both include a second posture of the target object.

[0152] In an optional embodiment of the present application, when executing step S906, the first multimedia data and the first image are input into the global residual network model to generate a final refined target human appearance map (equivalent to Figure 4 It should be noted that the postures of the target objects included in the first multimedia data and the first image and the second image are different, but the postures of the target objects included in the first image and the second image are the same, while the fineness of the target objects included in the first image and the second image is different, and the detail features of the second image are more than the detail features of the first image.

[0153] It should be noted that Fig. 9 The preferred implementation of the illustrated embodiment can be found in Figure 2 The relevant description of the illustrated embodiment will not be repeated here.

[0154] Example 7

[0155] Fig.10 is a flowchart of another image processing method according to an embodiment of the present application. Fig.10 As shown, the method comprises the following steps:

[0156] Step S1002: Acquire a first image of a target object and a target rotation angle.

[0157] Step S1004: Rotate the first image according to a target rotation angle to generate a second image.

[0158] Step S1006, generating a target image corresponding to the target object based on the first image and the second image, wherein the target image has more detail features than the first image, wherein both the second image and the target image include posture information of the target object after being rotated by a target rotation angle.

[0159] Steps S1002 to S1006 provide another method for processing an image, combining the e-commerce application scenario mentioned in Example 1, wherein the first image of the target object is an image of the model in the current posture (for example, the current model is facing 45 degrees to the left front), and the target rotation angle is the angle at which the first image is rotated (for example, if the model is facing 45 degrees to the right front, then the target rotation angle is rotated 90 degrees to the right).

[0160] When executing step S1004, the target rotation angle and the first image are input into the neural network model to generate a second image. It should be noted that the second image is an image of the model facing 45 degrees to the right front, but the second image is a rough image of the human body appearance.

[0161] When executing step S1006, the rough target human appearance image and the first image of the target object generated in step S1004 are processed using the global residual network model to generate a target human appearance with more texture details from coarse to fine.

[0162] It should be noted that Fig.10 The embodiments shown and Figure 2 The difference between the illustrated embodiment is that the posture information of the target posture in step S202 is replaced by the target rotation angle in step S1002. Fig.10 The preferred implementation of the illustrated embodiment can be found in Figure 2 The relevant description of the illustrated embodiment will not be repeated here.

[0163] Example 8

[0164] Fig.11 is a flowchart of another image processing method according to an embodiment of the present application. Fig.11 As shown, the method comprises the following steps:

[0165] Step S1102: acquiring first multimedia data corresponding to the target object and conversion description data for converting a target part in the first multimedia data, wherein the target part is any part of the target object.

[0166] According to an optional embodiment of the present application, the conversion description data is obtained in the following manner: receiving a conversion instruction input from the target object, wherein the conversion instruction carries the conversion description data; receiving voice information of the target object, and identifying the conversion description data from the voice information.

[0167] The first multimedia data in step S1102 may be a picture or video data. The conversion description data may refer to the conversion description data in step S902, which is equivalent to Figure 4 The estimated response map (pose2, the pose image of the target pose) extracted from the second sample image (image2) in . The target part is any body part of the model showing the clothing, such as hands, feet, face, etc.

[0168] For example, the first multimedia data is an image of a model in a current posture for displaying clothing (for example, the current model faces 45 degrees to the left front), and the conversion description data is used to convert the facial data of the model in the first multimedia data to face 45 degrees to the right front.

[0169] Preferably, the above conversion description data can be input by the user through a human-computer interaction interface, and then effective feature data is extracted from the conversion description data input by the user through natural language processing technology. It can also be obtained from the voice control instructions issued by the user through voice recognition technology.

[0170] Step S1104: generating a first image corresponding to the target object based on the first multimedia data and the conversion description data, wherein the first image includes an image obtained by converting the target part according to the conversion description data.

[0171] When executing step S1104, the first multimedia data and the conversion description data are input into the neural network model to generate a first image corresponding to the target object. It should be noted that the first image is a rough human appearance image of the model.

[0172] Step S1106, generating a second image corresponding to the target object based on the first multimedia data and the first image, wherein the second image includes an image after the target part is converted according to the conversion description data, and the detail features of the target part in the second image are more than the detail features of the target part in the first image.

[0173] When executing step S1106, the rough target human appearance image and the first multimedia data of the target object generated in the first stage are processed using the global residual network model to generate a target human appearance with more texture details from coarse to fine.

[0174] The image processing method provided in step S1102 to step S1106 can be applied to model local feature cutting, for example, generating model face image B from model face image A, or generating model hand image B from model hand image A.

[0175] It should be noted that Fig.11 The preferred implementation of the illustrated embodiment can be found in Figure 2 The relevant description of the illustrated embodiment will not be repeated here.

[0176] Example 9

[0177] Fig.12 is a flowchart of another image processing method according to an embodiment of the present application. Fig.12 As shown, the method comprises the following steps:

[0178] Step S1202: Acquire multiple images of the first target object captured at adjacent sampling moments.

[0179] Step S1204: determining posture change information of the first target object in the multiple images.

[0180] Step S1206: Acquire a first image of the second target object.

[0181] Step S1208: convert the first image according to the posture change information to generate a second image.

[0182] Step S1210: generating a target image corresponding to the second target object based on the first image and the second image, wherein the target image has more detail features than the first image.

[0183] The image processing method provided in step S1202 to step S1210 can be applied to video editing scenarios, for example, the action of the first target object is extracted and assigned to the second target object, and can be applied to video editing of fitness courses, etc.

[0184] Specifically, when executing step S1202, multiple images of the first target object captured at adjacent sampling moments are acquired to ensure that the multiple actions of the first target object captured are a series of coherent actions.

[0185] When executing step S1202, the posture change information of the first target object is determined from the acquired multiple images. Then, step S1206 is executed to acquire the first image of the second target object.

[0186] According to an optional embodiment of the present application, when executing step S1208, the posture change information of the first target object and the first image of the second target object are input into the neural network model to generate a second image of the second target object. It should be noted that the second image is a rough human appearance image of the second target object.

[0187] When executing step S1210, the rough human appearance image generated in step S1208 and the first image of the target object are processed using the global residual network model to generate a target human appearance with more texture details of the second target object from coarse to fine.

[0188] It should be noted that Fig.12 The preferred implementation of the illustrated embodiment can be found in Figure 2 The relevant description of the illustrated embodiment will not be repeated here.

[0189] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0190] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0191] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0192] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0193] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0194] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0195] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing an image, characterized in that: include: Acquire a first image of a target object and a target rotation angle; Rotate the first image according to the target rotation angle to generate a second image; Based on the first image and the second image, a target image corresponding to the target object is generated, wherein the target image has more detailed features than the detailed features of the first image, wherein both the second image and the target image include posture information of the target object after being rotated by the target rotation angle; After generating a target image corresponding to the target object, the method further includes: receiving feedback information on the target image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on the historical clothing matching information of the target object.

2. A method for processing an image, characterized in that: include: Acquire first multimedia data corresponding to a target object, and conversion description data for converting a target part in the first multimedia data, wherein the target part is any part of the target object; Based on the first multimedia data and the conversion description data, generating a first image corresponding to the target object, wherein the first image includes an image after the target part is converted according to the conversion description data; generating a second image corresponding to the target object based on the first multimedia data and the first image, wherein the second image includes an image after the target part is transformed according to the transformation description data, and detail features of the target part in the second image are greater than detail features of the target part in the first image; After generating a second image corresponding to the target object, the method further includes: receiving feedback information on the second image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on the historical clothing matching information of the target object.

3. The method according to claim 2, characterized in that The conversion description data is obtained in the following way: Receiving a conversion instruction input from the target object, wherein the conversion instruction carries the conversion description data; The speech information of the target object is received, and the conversion description data is identified from the speech information.

4. A method for processing an image, characterized in that: include: Acquire a plurality of images of the first target object acquired at adjacent sampling moments; Determining posture change information of the first target object in the multiple images; acquiring a first image of a second target object; converting the first image according to the posture change information to generate a second image; generating a target image corresponding to the second target object based on the first image and the second image, wherein the target image has more detail features than the first image; After generating a target image corresponding to the second target object, the method further includes: receiving feedback information on the target image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on the historical clothing matching information of the target object.

5. A method for processing an image, characterized in that: include: Receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after conversion from the current posture; Based on the first appearance image and the posture information of the target posture, generating a second appearance image when the target object is converted into the target posture; Processing the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in the target posture, wherein the third appearance image has more detailed features than the second appearance image; After generating a third appearance image of the target object in the target posture, the method further includes: receiving feedback information on the third appearance image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on historical clothing matching information of the target object.

6. The method according to claim 5, characterized in that Generating a second appearance image when the target object is converted to the target posture based on the first appearance image and the posture information of the target posture, comprising: The first appearance image and the posture information of the target posture are processed using a first neural network model to generate the second appearance image, wherein the neural network model is trained using a training sample set.

7. The method according to claim 6, characterized in that Before using the first neural network model to process the first appearance image and the posture information of the target posture to generate the second appearance image, the method further includes: The training sample set is used to train model parameters of a fitting network model to generate the first neural network model, wherein the first neural network model is used to generate a second appearance image of a target image to be identified.

8. The method according to claim 7, characterized in that Before using the training sample set to train the model parameters of the fitting network model, the method further includes: constructing the training sample set, wherein the training sample set includes multiple groups of training samples, wherein constructing the training sample set includes: Get multiple posture images; Randomly extracting a first sample image and a second sample image having the same sample object from the multiple posture images, wherein the first sample image is an image of the sample object before being transformed into a target posture, and the second sample image is an image of the sample object after being transformed into the target posture; Extracting a posture image of the target posture from the second sample image; A set of training samples is constructed based on the first sample image, the second sample image and the extracted posture image.

9. The method according to claim 8, characterized in that Before using the global residual network model to process the second appearance image and the first appearance image to generate the second appearance image of the target object in the target posture, the method further includes: constructing the global residual network model, which step includes: Using the rough image of the second sample image and the first sample image, and taking the second sample image as a supervision image, fitting training is performed on model parameters of the global residual network model until the global residual network model converges.

10. The method according to claim 8, characterized in that Extracting a posture image of the target posture from the second sample image includes: Extracting a plurality of feature points located at different positions of the target object from the second sample image; The plurality of feature points are connected according to their positions in the human body and the association relationship between the plurality of feature points to obtain a posture image of the target posture.

11. The method according to claim 5, characterized in that Receiving a first appearance image of a target object and posture information of a target posture, including: Receiving a clothing matching image of the target object currently captured by an image acquisition device, and using the clothing matching image as the first appearance image, wherein the clothing matching image includes: the target object and clothing matching information of the target object; Receive the posture information of the target posture input through the human-computer interaction interface of the image acquisition device.

12. The method according to claim 11, characterized in that After processing the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in the target posture, the method further includes: When the feedback information is the second result, the third appearance image is stored in a clothing matching database corresponding to the target object.

13. A method for processing an image, characterized in that: include: When the target object switches from a first posture to a second posture, obtaining a first appearance image of the target object in the first posture and posture information of the second posture; generating, based on the first appearance image and the posture information of the second posture, a second appearance image when the target object is in the second posture; Processing the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in the second posture, wherein the third appearance image has more detailed features than the second appearance image; After generating a third appearance image of the target object in the second posture, the method further includes: receiving feedback information on the third appearance image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on historical clothing matching information of the target object.

14. A data processing method, characterized in that: include: Acquire first multimedia data and conversion description data corresponding to the target object, wherein the conversion description data is used to convert the first multimedia data; generating a first image corresponding to the target object based on the first multimedia data and the conversion description data; and generating, based on the first multimedia data and the first image, a second image corresponding to the target object, wherein the second image has more detail features than the first image; The first multimedia data includes a first posture of the target object, and the first image and the second image both include a second posture of the target object; After generating a second image corresponding to the target object, the method further includes: receiving feedback information on the second image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on the historical clothing matching information of the target object.

15. The method according to claim 14, characterized in that The first multimedia data includes: picture or video data.

16. An image processing device, characterized in that: include: A receiving module, configured to receive a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after conversion from the current posture; A generating module, configured to generate a second appearance image when the target object is converted into the target posture based on the first appearance image and the posture information of the target posture; a processing module, configured to process the second appearance image and the first appearance image using a global residual network model to generate a third appearance image of the target object in the target posture, wherein the third appearance image has more detailed features than the second appearance image; The image processing device is also used to perform the following steps after generating a third appearance image of the target object in the target posture: receiving feedback information on the third appearance image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on historical clothing matching information of the target object.

17. A computer-readable storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the image processing method described in any one of claims 1 to 13 and the data processing method described in claim 14 or 15.

18. A computing device comprising: processor; as well as A memory is connected to the processor and is used to provide the processor with instructions for processing the following processing steps: receiving a first appearance image of a target object and posture information of a target posture, wherein the first appearance image is an image of the target object in a current posture, and the target posture is a posture of the target object after being converted from the current posture; based on the first appearance image and the posture information of the target posture, generating a second appearance image of the target object when it is converted to the target posture; using a global residual network model to process the second appearance image and the first appearance image to generate a third appearance image of the target object in the target posture, wherein the third appearance image has more detailed features than the second appearance image; after generating the third appearance image of the target object in the target posture: receiving feedback information on the third appearance image; when the feedback information is a first result, identifying the brand to which the clothing matching image of the target object belongs, and selecting target clothing from a clothing database corresponding to the brand and inputting it into a second neural network model, wherein the second neural network model is trained based on the historical clothing matching information of the target object.

Images