Image processing method and device

Abstract

The present invention discloses an image processing method and device, and relates to the field of computer technology. A specific implementation of the method includes: segmenting a limb area from an image to be converted from a first perspective to obtain a limb image from the first perspective; searching for a limb image template of a second perspective corresponding to the limb image from the first perspective in a template library; generating a limb image from the second perspective based on the limb image from the first perspective and the limb image template from the second perspective; based on the image to be converted from the first perspective, correcting the limb image from the second perspective to obtain a converted image from the second perspective corresponding to the image to be converted. This implementation can automatically convert the image perspective without manual intervention, solve the problems of shoulder distortion and perspective shift caused by the selfie perspective, reduce conversion time, reduce labor costs, and realize batch conversion of image perspectives.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing method and device. Background Art

[0002] In some scenarios, it is necessary to convert an image from one perspective to another, such as converting an image from a selfie perspective to an image from a third person's perspective. When a single person or a group of people take a selfie, due to the influence of arm length, angle, etc., there will be problems such as arms entering the body, body disproportion, and unnaturalness. Therefore, it is necessary to convert the image from the selfie perspective to the third person's perspective to provide a variety of options for taking photos. There are two existing solutions for converting images from the selfie perspective to images from the third person's perspective: the first solution is to remove the arms, selfie sticks, etc. that are extended due to the selfie perspective by image cropping, and obtain a shortened image that is basically from the third person's perspective; the second solution is to manually change the perspective effect of the image through post-production manual retouching methods, smear out the arms that enter the body, and use the "clone stamp" and other retouching functions to fill the background pattern into the arms and other areas to achieve the purpose of converting the selfie perspective to the third person's perspective.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] Solution 1 requires manual selection of the cropped parts, and more parts need to be cropped to remove unnatural images such as arms. It cannot solve problems such as shoulder distortion and perspective shift caused by the selfie perspective. This solution is relatively mechanical. Solution 2 requires certain post-production photo editing skills, has many steps, and takes a long time. It cannot achieve batch conversion of image perspectives in a short period of time. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides an image processing method and device, which can automatically convert the image perspective without human intervention, solve problems such as shoulder distortion and perspective shift caused by selfie perspective, reduce conversion time, reduce labor costs, and realize batch conversion of image perspective.

[0006] To achieve the above objective, according to one aspect of an embodiment of the present invention, an image processing method is provided.

[0007] An image processing method includes: segmenting a limb area from an image to be converted at a first perspective to obtain a limb image at the first perspective; searching a template library for a limb image template at a second perspective corresponding to the limb image at the first perspective; generating a limb image at the second perspective based on the limb image at the first perspective and the limb image template at the second perspective; and correcting the limb image at the second perspective based on the image to be converted at the first perspective to obtain a converted image at the second perspective corresponding to the image to be converted.

[0008] Optionally, searching for a limb image template of the second perspective corresponding to the limb image of the first perspective in a template library includes: using a nearest neighbor algorithm to search for a limb image template of the first perspective that is closest to the limb image of the first perspective in the template library; and searching for a limb image template of the second perspective corresponding to the limb image template of the first perspective based on the found limb image template of the first perspective.

[0009] Optionally, before searching for a limb image template of the second perspective corresponding to the limb image of the first perspective in the template library, the method includes: pre-establishing an image pair, the image pair including an image sample of the first perspective and an image sample of the second perspective corresponding to the image sample of the first perspective; performing limb recognition on the image samples in the image pair to obtain a limb image template of the first perspective corresponding to the image sample of the first perspective, and a limb image template of the second perspective corresponding to the image sample of the second perspective; and storing the limb image template of the first perspective and the limb image template of the second perspective in the template library in correspondence with each other according to the correspondence between the image sample of the first perspective and the image sample of the second perspective.

[0010] Optionally, the image to be converted of the first perspective is corrected on the limb image of the second perspective to obtain a converted image of the second perspective corresponding to the image to be converted, including: inputting the image to be converted of the first perspective and the limb image of the second perspective into a trained preset generation model to output a converted image of the second perspective corresponding to the image to be converted.

[0011] Optionally, the preset generation model is trained using an image sample set of a first perspective and a limb image template set of a second perspective, wherein the image sample set of the first perspective includes image samples of the first perspective, and the limb image template set of the second perspective includes limb image templates of the second perspective.

[0012] Optionally, the preset generation model is implemented based on a twin network and a generative adversarial network.

[0013] Optionally, generating the limb image of the second perspective based on the limb image of the first perspective and the limb image template of the second perspective includes: filling the limb image template of the second perspective according to the limb image of the first perspective according to various parts of the limb to generate the limb image of the second perspective.

[0014] According to another aspect of an embodiment of the present invention, an image processing apparatus is provided.

[0015] An image processing device includes: a first-perspective limb image generation module, used to segment a limb area from an image to be converted at a first perspective to obtain a limb image at the first perspective; a template search module, used to search a template library for a second-perspective limb image template corresponding to the limb image at the first perspective; a second-perspective limb image generation module, used to generate a second-perspective limb image based on the limb image at the first perspective and the limb image template at the second perspective; and a correction processing module, used to perform correction processing on the second-perspective limb image based on the image to be converted at the first perspective to obtain a converted image at the second perspective corresponding to the image to be converted.

[0016] Optionally, the template search module is also used to: use a nearest neighbor algorithm to search in the template library for a limb image template of the first perspective that is closest to the limb image of the first perspective; and based on the found limb image template of the first perspective, search for a limb image template of the second perspective that corresponds to the limb image template of the first perspective.

[0017] Optionally, it also includes a template library establishment module, which is used to: pre-establish an image pair, the image pair including an image sample of a first perspective and an image sample of a second perspective corresponding to the image sample of the first perspective; perform limb recognition on the image samples in the image pair to obtain a limb image template of the first perspective corresponding to the image sample of the first perspective, and a limb image template of the second perspective corresponding to the image sample of the second perspective; and store the limb image template of the first perspective and the limb image template of the second perspective in the template library in accordance with the correspondence between the image sample of the first perspective and the image sample of the second perspective.

[0018] Optionally, the correction processing module is also used to: input the image to be converted of the first perspective and the limb image of the second perspective into a trained preset generation model to output a converted image of the second perspective corresponding to the image to be converted.

[0019] Optionally, the preset generation model is trained using an image sample set of a first perspective and a limb image template set of a second perspective, wherein the image sample set of the first perspective includes image samples of the first perspective, and the limb image template set of the second perspective includes limb image templates of the second perspective.

[0020] Optionally, the preset generation model is implemented based on a twin network and a generative adversarial network.

[0021] Optionally, the second-viewing angle limb image generation module is further used to fill the second-viewing angle limb image template according to various parts of the limb and the limb image of the first view to generate the second-viewing angle limb image.

[0022] According to yet another aspect of the embodiments of the present invention, an electronic device is provided.

[0023] An electronic device includes: one or more processors; and a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing method provided by an embodiment of the present invention.

[0024] According to yet another aspect of an embodiment of the present invention, a computer readable medium is provided.

[0025] A computer readable medium stores a computer program, which, when executed by a processor, implements the image processing method provided by an embodiment of the present invention.

[0026] One embodiment of the above invention has the following advantages or beneficial effects: segmenting the limb area from the image to be converted from the first perspective to obtain the limb image from the first perspective; searching the limb image template of the second perspective corresponding to the limb image from the first perspective in the template library; generating the limb image from the second perspective based on the limb image from the first perspective and the limb image template from the second perspective; correcting the limb image from the second perspective based on the image to be converted from the first perspective to obtain the converted image from the second perspective corresponding to the image to be converted. The image perspective can be automatically converted without manual intervention, solving the problems of shoulder distortion and perspective shift caused by the selfie perspective, reducing conversion time, reducing labor costs, and realizing batch conversion of image perspectives.

[0027] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0029] Figure 1 is a schematic diagram of main steps of an image processing method according to an embodiment of the present invention;

[0030] Figure 2 is a schematic diagram of a process of converting an image from a self-portrait perspective to a third-party perspective according to an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of a framework of a GAN correction model according to an embodiment of the present invention;

[0032] Figure 4 is a schematic diagram of main modules of an image processing device according to an embodiment of the present invention;

[0033] Figure 5 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0034] Figure 6 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0036] Figure 1 Schematic diagram of main steps of an image processing method according to an embodiment of the present invention.

[0037] like Figure 1 As shown, an image processing method according to an embodiment of the present invention mainly includes the following steps S101 to S104.

[0038] Step S101: segmenting a limb region from an image to be converted at a first viewing angle to obtain a limb image at a first viewing angle.

[0039] Step S102: searching the template library for a limb image template of the second perspective corresponding to the limb image of the first perspective.

[0040] Searching for a second-perspective limb image template corresponding to a first-perspective limb image in a template library may include: using a nearest neighbor algorithm to search for a first-perspective limb image template closest to the first-perspective limb image in the template library; and searching for a second-perspective limb image template corresponding to the first-perspective limb image template based on the searched first-perspective limb image template. The limb image template is a limb image used as a template, and the template library may be a database storing all limb image templates.

[0041] Among them, the first perspective can be a selfie perspective, and the second perspective can be a perspective for taking photos with others.

[0042] Before searching for a limb image template of the second perspective corresponding to the limb image of the first perspective in the template library, the method may include: pre-establishing an image pair, the image pair including an image sample of the first perspective and an image sample of the second perspective corresponding to the image sample of the first perspective; performing limb recognition on the image sample of the first perspective and the image sample of the second perspective in the image pair, and obtaining a limb image template of the first perspective corresponding to the image sample of the first perspective, and a limb image template of the second perspective corresponding to the image sample of the second perspective; and storing the limb image template of the first perspective and the limb image template of the second perspective in the template library in correspondence with each other according to the correspondence between the image sample of the first perspective and the image sample of the second perspective.

[0043] Among them, limb recognition can use Dense Pose technology, which is a technology that maps all human body pixels of a 2D RGB image to a 3D model in real time.

[0044] Step S103: Generate a limb image of a second perspective according to the limb image of the first perspective and the limb image template of the second perspective.

[0045] Generating a limb image of a second perspective according to the limb image of a first perspective and a limb image template of a second perspective may include: filling the limb image template of the second perspective according to the limb image of the first perspective according to various parts of the limb to generate the limb image of the second perspective.

[0046] Step S104: Based on the image to be converted at the first perspective, the limb image at the second perspective is corrected to obtain a converted image at the second perspective corresponding to the image to be converted.

[0047] Based on the image to be converted of the first perspective, the limb image of the second perspective is corrected to obtain a converted image of the second perspective corresponding to the image to be converted, which may include: inputting the image to be converted of the first perspective and the limb image of the second perspective into a trained preset generation model to output the converted image of the second perspective corresponding to the image to be converted.

[0048] A preset generation model is trained using an image sample set of a first perspective and a limb image template set of a second perspective. The image sample set of the first perspective includes image samples of the first perspective, and the limb image template set of the second perspective includes limb image templates of the second perspective. The limb image templates of the second perspective are obtained by performing limb recognition on image samples of the second perspective corresponding to image samples of the first perspective. The training target is image samples of the second perspective corresponding to each image sample of the first perspective.

[0049] The preset generative model can be implemented based on the twin network and the generative adversarial network.

[0050] Figure 2It is a schematic diagram of a process of converting an image from a self-portrait perspective to a third-party perspective according to an embodiment of the present invention.

[0051] like Figure 2 As shown, the present invention proposes a method for converting image selfie perspective based on deep learning, which is mainly a method of synthesizing corresponding selfie images using images taken by others, using the nearest neighbor method to retrieve the selfie image closest to the given image taken by others, and then synthesizing the corresponding selfie through the GAN (Generative Adversarial Network) correction model (i.e., the preset generation model). It can effectively avoid the shortcomings of traditional conversion methods, achieve better shooting angle conversion effects, and realize automatic conversion without manual intervention. The process of the present invention is described in detail below.

[0052] A self-portrait-photographed image pair is established, and the image pair includes an image sample of a first perspective and an image sample of a second perspective corresponding to the image sample of the first perspective. In this embodiment, the first perspective is the self-portrait perspective, and the second perspective is the perspective of the photographed person. The image pair is identified with the Dense Pose technology to obtain a limb image template of the first perspective corresponding to the image sample of the first perspective, and a limb image template of the second perspective corresponding to the image sample of the second perspective. The limb image template of the first perspective and the limb image template of the second perspective are combined into a limb image template pair, and stored in a self-portrait-photographed database (i.e., a template library).

[0053] The limb area of ​​the image to be converted (i.e., the image to be converted from the first perspective) is segmented to obtain the limb image from the first perspective. The limb part of the limb image from the first perspective is recognized using Dense Pose, and the limb image taken by another person (i.e., the limb image from the first perspective) that is closest to the limb image from the established limb image template library is found using the nearest neighbor method. Figure 2 Specifically, the nearest neighbor algorithm can be used to find a first-perspective limb image template that is closest to the first-perspective limb image in the template library, and then, based on the found first-perspective limb image template, find a second-perspective limb image template corresponding to the first-perspective limb image template (that is, the other-perspective limb image in the template library that is closest to the first-perspective limb image).

[0054] According to the limb image of the first perspective, the limb image template of the second perspective detected is filled in according to each part of the human body, and a rough limb image taken by another person (the limb image of the second perspective, i.e. Figure 2 The rough image taken by someone else in the figure only contains a portrait, and further background filling and limb repair are required.

[0055] The image sample set of the first perspective and the limb image template set of the second perspective are used as training samples, and the image sample set of the second perspective corresponding to the image sample set of the first perspective is used as the label of the training sample. Specifically, the GAN correction model (i.e., the preset generation model) can be trained based on the twin network and CycleGAN (cycle consistency generative adversarial network). The image sample set of the first perspective includes image samples of the first perspective, the image sample set of the second perspective includes image samples of the second perspective, the limb image template set of the second perspective includes limb image templates of the second perspective, and the limb image template of the second perspective is obtained by performing limb recognition on the image samples of the second perspective corresponding to the image samples of the first perspective. The GAN correction model realizes the background filling of the limb image of the second perspective and the repair and refinement of the limb.

[0056] The image to be converted from the first perspective and the limb image from the second perspective are input into the trained GAN correction model to perform background filling and limb repair, so as to output a converted image taken by another person corresponding to the image to be converted (i.e., the converted image from the second perspective).

[0057] Those skilled in the art will appreciate that, in other embodiments, the first viewing angle may be a perspective taken by another person, and the second viewing angle may be a perspective taken by a selfie, so that the process of converting the perspective taken by another person to the perspective taken by a selfie can be implemented through the image processing method of an embodiment of the present invention.

[0058] Figure 3 Schematic diagram of a GAN correction model according to an embodiment of the present invention.

[0059] like Figure 3 As shown, when using the GAN correction model, the image to be converted and the rough image taken by another person (i.e., the limb image taken from another person's perspective) are taken as input, and the converted image taken by another person is obtained through a CNN (convolutional neural network) module and a generative adversarial network (such as CycleGAN).

[0060] When training the GAN correction model, the dual-input and single-output characteristics of the twin network are utilized, and the image sample set of the first perspective and the limb image template set of the second perspective are used as two inputs. After convolution processing by the CNN network module, they are connected to the generative adversarial network, and the image sample set of the second perspective corresponding to the image sample set of the first perspective is used as the target output. After repeated iterations, the trained GAN correction model is finally obtained. The image sample set of the first perspective corresponds to the image sample set of the second perspective, which means that in the image sample sets of the two perspectives, the objects (for the embodiment of the present invention, the objects are people) in the image samples of the first perspective and the image samples of the second perspective that have a corresponding relationship are the same, but the perspectives are different. For example, the image sample of the first perspective is a person in a certain posture taken from the self-portrait perspective, and the corresponding image sample of the second perspective is the person in this posture taken from the perspective of others.

[0061] Figure 4 4 is a schematic diagram of main modules of an image processing device according to an embodiment of the present invention.

[0062] like Figure 4 As shown, an image processing device 400 according to an embodiment of the present invention mainly includes: a first-view limb image generation module 401 , a template search module 402 , a second-view limb image generation module 403 , and a correction processing module 404 .

[0063] The first-view limb image generation module 401 is used to segment the limb area from the image to be converted in the first view to obtain the limb image in the first view.

[0064] The template search module 402 is used to search the template library for a body image template of the second perspective corresponding to the body image of the first perspective.

[0065] The second-viewing angle limb image generating module 403 is used to generate a second-viewing angle limb image according to the first-viewing angle limb image and the second-viewing angle limb image template.

[0066] The correction processing module 404 is used to perform correction processing on the limb image of the second perspective based on the image to be converted of the first perspective, so as to obtain a converted image of the second perspective corresponding to the image to be converted.

[0067] In one embodiment, the template search module is specifically used to: use the nearest neighbor algorithm to search in the template library for a first-perspective limb image template that is closest to the first-perspective limb image; and based on the found first-perspective limb image template, search for a second-perspective limb image template corresponding to the first-perspective limb image template.

[0068] In one embodiment, it may also include a template library establishment module, which is used to: pre-establish an image pair, the image pair including an image sample of a first perspective and an image sample of a second perspective corresponding to the image sample of the first perspective; perform limb recognition on the image samples in the image pair to obtain a limb image template of the first perspective corresponding to the image sample of the first perspective, and a limb image template of the second perspective corresponding to the image sample of the second perspective; according to the correspondence between the image sample of the first perspective and the image sample of the second perspective, store the limb image template of the first perspective and the limb image template of the second perspective in the template library accordingly.

[0069] In one embodiment, the correction processing module is specifically used to: input the image to be converted from the first perspective and the limb image from the second perspective into a trained preset generation model to output a converted image from the second perspective corresponding to the image to be converted.

[0070] In one embodiment, a preset generation model is trained using an image sample set of a first perspective and a limb image template set of a second perspective, wherein the image sample set of the first perspective includes image samples of the first perspective, and the limb image template set of the second perspective includes limb image templates of the second perspective.

[0071] In one embodiment, the preset generative model is based on a twin network and a generative adversarial network implementation model.

[0072] In one embodiment, the second-viewing angle limb image generation module is specifically used to fill the second-viewing angle limb image template according to various parts of the limb and the limb image of the first view to generate the second-viewing angle limb image.

[0073] Figure 5 An exemplary system architecture 500 to which the image processing method or image processing apparatus according to the embodiment of the present invention can be applied is shown.

[0074] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, 503, a network 504 and a server 505. Network 504 is used to provide a medium for communication links between terminal devices 501, 502, 503 and server 505. Network 504 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0075] Users can use terminal devices 501, 502, 503 to interact with server 505 through network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0076] The terminal devices 501 , 502 , and 503 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0077] The server 505 may be a server that provides various services, such as a background management server (only for example) that provides support for shopping websites browsed by users using the terminal devices 501, 502, and 503. The background management server may process the received data such as the image processing request, and feed back the processing result (such as the processed image - only for example) to the terminal device.

[0078] It should be noted that the image processing method provided in the embodiment of the present invention is generally executed by the server 505 , and accordingly, the image processing device is generally disposed in the server 505 .

[0079] It should be understood that Figure 5 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0080] Reference below Figure 6 , which shows a schematic diagram of the structure of a computer system 600 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 6 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0081] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage part 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the system 600 are also stored. The CPU 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0082] The following components are connected to the I / O interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the I / O interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage section 608 as needed.

[0083] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present invention are executed.

[0084] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0085] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0086] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a first-view limb image generation module, a template search module, a second-view limb image generation module, and a correction processing module. The names of these modules do not constitute a limitation on the modules themselves in some cases. For example, the first-view limb image generation module may also be described as "a module for segmenting a limb area from an image to be converted from a first view to obtain a limb image from a first view".

[0087] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: segmenting a limb area from the image to be converted from the first perspective to obtain a limb image from the first perspective; searching for a limb image template of the second perspective corresponding to the limb image from the first perspective in the template library; generating a limb image from the second perspective based on the limb image from the first perspective and the limb image template from the second perspective; based on the image to be converted from the first perspective, correcting the limb image from the second perspective to obtain a converted image from the second perspective corresponding to the image to be converted.

[0088] According to the technical solution of the embodiment of the present invention, the limb area is segmented from the image to be converted from the first perspective to obtain the limb image of the first perspective; the limb image template of the second perspective corresponding to the limb image of the first perspective is searched in the template library; the limb image of the second perspective is generated according to the limb image of the first perspective and the limb image template of the second perspective; based on the image to be converted from the first perspective, the limb image of the second perspective is corrected to obtain the converted image of the second perspective corresponding to the image to be converted. The image perspective can be automatically converted without human intervention, and the problems of shoulder distortion and perspective shift caused by the selfie perspective can be solved, the conversion time can be reduced, the labor cost can be reduced, and the batch conversion of the image perspective can be realized.

[0089] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Pre-establishing an image pair, wherein the image pair includes an image sample of a first perspective and an image sample of a second perspective corresponding to the image sample of the first perspective; Performing limb recognition on the image samples in the image pair to obtain limb image templates of the first perspective corresponding to the image samples of the first perspective, and limb image templates of the second perspective corresponding to the image samples of the second perspective; according to the correspondence between the image samples of the first perspective and the image samples of the second perspective, storing the limb image templates of the first perspective and the limb image templates of the second perspective in a template library in a corresponding manner; Segmenting a limb region from the image to be converted from the first perspective to obtain a limb image from the first perspective; Searching for a limb image template of a second perspective corresponding to the limb image of the first perspective in a template library, comprising: using a nearest neighbor algorithm to search for a limb image template of the first perspective that is closest to the limb image of the first perspective in the template library; searching for a limb image template of the second perspective corresponding to the limb image template of the first perspective based on the found limb image template of the first perspective; Generate a limb image at a second perspective according to the limb image at the first perspective and the limb image template at the second perspective; Based on the image to be converted of the first perspective, the limb image of the second perspective is corrected to obtain a converted image of the second perspective corresponding to the image to be converted, including: inputting the image to be converted of the first perspective and the limb image of the second perspective into a trained preset generation model to output the converted image of the second perspective corresponding to the image to be converted; wherein the preset generation model is trained using an image sample set of the first perspective and a limb image template set of the second perspective, the image sample set of the first perspective includes the image samples of the first perspective, and the limb image template set of the second perspective includes the limb image templates of the second perspective.

2. The method according to claim 1, characterized in that The preset generation model is implemented based on a twin network and a generative adversarial network.

3. The method according to claim 1, characterized in that The step of generating a limb image at a second perspective according to the limb image at the first perspective and the limb image template at the second perspective includes: According to each part of the limb, the limb image template of the second perspective is filled in according to the limb image of the first perspective to generate the limb image of the second perspective.

4. An image processing device, characterized in that: include: A template library establishment module is used to establish an image pair in advance, wherein the image pair includes an image sample of a first perspective and an image sample of a second perspective corresponding to the image sample of the first perspective; perform limb recognition on the image samples in the image pair to obtain a limb image template of the first perspective corresponding to the image sample of the first perspective, and a limb image template of the second perspective corresponding to the image sample of the second perspective; according to the correspondence between the image sample of the first perspective and the image sample of the second perspective, store the limb image template of the first perspective and the limb image template of the second perspective in the template library in correspondence; A first-view limb image generation module, used to segment a limb area from the image to be converted at the first view to obtain a limb image at the first view; A template search module, used for searching a template library for a limb image template of a second perspective corresponding to the limb image of the first perspective; It is also used to use the nearest neighbor algorithm to search in the template library for a limb image template of the first perspective that is closest to the limb image of the first perspective; and according to the found limb image template of the first perspective, search for a limb image template of the second perspective that corresponds to the limb image template of the first perspective; A second-viewing angle limb image generating module, configured to generate a second-viewing angle limb image according to the first-viewing angle limb image and the second-viewing angle limb image template; A correction processing module, configured to perform correction processing on the limb image of the second perspective based on the image to be converted of the first perspective, so as to obtain a converted image of the second perspective corresponding to the image to be converted; It is also used to input the image to be converted of the first perspective and the limb image of the second perspective into a trained preset generation model to output a converted image of the second perspective corresponding to the image to be converted; wherein the preset generation model is trained using an image sample set of the first perspective and a limb image template set of the second perspective, the image sample set of the first perspective includes image samples of the first perspective, and the limb image template set of the second perspective includes limb image templates of the second perspective.

5. The device according to claim 4, characterized in that The preset generation model is implemented based on a twin network and a generative adversarial network.

6. The device according to claim 4, characterized in that The second perspective limb image generation module is also used for: According to each part of the limb, the limb image template of the second perspective is filled in according to the limb image of the first perspective to generate the limb image of the second perspective.

7. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

8. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images