Electronic device for processing image, and operating method thereof

Abstract

The method of the present disclosure may comprise the steps of: acquiring a plurality of images including each of a plurality of persons; determining any one of the plurality of images as a base image; acquiring, from the plurality of images, a plurality of pieces of pose information related to a pose of a first person from among the plurality of persons; three-dimensionally modeling the pose of the person on the basis of the plurality of pieces of pose information so as to acquire a pose range relating to the pose of the person that can be implemented in a three-dimensional form; determining a first pose within the pose range; and synthesizing, on the base image, a body image of the first person taking the first pose so as to generate a correction image.

Description

Electronic device for processing images and method of operation thereof

[0001] The present invention relates to an electronic device for processing images and an operating method thereof, and more particularly, to an electronic device for changing a pose of a person in an image based on a plurality of images acquired through continuous shooting and an operating method thereof.

[0002] In recent years, rapid advancements in communication technology have led to the expansion of mobile device functionality, leading to the provision of increasingly diverse user interfaces (UIs). To enhance the utility of these devices and satisfy the diverse needs of users, a variety of applications capable of running on these devices are being developed.

[0003] In particular, with the growing user interest in photography and video, most mobile devices now offer digital camera functionality. At the same time, users demand high-quality photos taken with digital cameras and want them to look attractive. Methods to satisfy these preferences, such as correcting hand shake or retouching photos, are being introduced. However, these are merely supplementary methods for retouching photos, and if the user is not satisfied with the expression or pose, the photo will ultimately need to be retaken.

[0004] A method disclosed as a technical means for achieving a technical task may include a step of acquiring a plurality of images, each of which includes a plurality of persons. The method may include a step of determining one of the plurality of images as a base image. The method may include a step of acquiring a plurality of pose information regarding a pose of a first person among the plurality of persons from the plurality of images. The method may include a step of acquiring a pose range regarding a pose of the person that can be implemented in a three-dimensional form by three-dimensionally modeling the pose of the person based on the plurality of pose information. The method may include a step of determining a first pose within the pose range. The method may include a step of generating a corrected image by synthesizing a body image of the first person striking the first pose onto the base image.

[0005] An electronic device disclosed as a technical means for achieving a technical task may include an input / output interface, a memory, and at least one processor. The input / output interface may receive a user input requesting image processing, and output a processed image according to the user input. The memory may store commands for processing the image. By having at least one processor execute a program or at least one instruction stored in the memory, the electronic device may acquire a plurality of images each including a plurality of persons, determine one of the plurality of images as a base image, acquire a plurality of pose information regarding a pose of a first person among the plurality of persons from the plurality of images, and model the pose of the person in three dimensions based on the plurality of pose information, thereby acquiring a pose range regarding a pose of the person that can be implemented in three dimensions, determining a first pose within the pose range, and synthesizing a body image of the first person striking the first pose onto the base image, thereby generating a corrected image.

[0006] A computer-readable recording medium disclosed as a technical means for achieving a technical task may have stored thereon a program for executing at least one of the embodiments of the disclosed method on a computer.

[0007] FIG. 1 is a conceptual diagram illustrating a method for generating a correction image according to an embodiment of the present disclosure.

[0008] FIG. 2 is a diagram for explaining an overview of a process for generating a correction image according to one embodiment of the present disclosure.

[0009] FIG. 3 is a flowchart illustrating a method for generating a correction image according to an embodiment of the present disclosure.

[0010] FIG. 4 is a conceptual diagram illustrating a method for detecting an object in an image according to one embodiment of the present disclosure.

[0011] FIG. 5 is a conceptual diagram illustrating a method for classifying objects in an image according to one embodiment of the present disclosure.

[0012] FIG. 6 is a flowchart illustrating a method for classifying objects in an image according to one embodiment of the present disclosure.

[0013] FIG. 7 is a flowchart illustrating a method for classifying objects in an image according to one embodiment of the present disclosure.

[0014] FIG. 8 is a conceptual diagram illustrating a method for obtaining pose information of a person from an image according to one embodiment of the present disclosure.

[0015] FIG. 9 is a flowchart illustrating a method for obtaining pose information of a person from an image according to one embodiment of the present disclosure.

[0016] FIG. 10 is a conceptual diagram illustrating a method for generating a 3D model of a person's pose based on an image and pose information of the person according to one embodiment of the present disclosure.

[0017] FIG. 11 is a flowchart illustrating a method for generating a correction image according to one embodiment of the present disclosure.

[0018] FIG. 12 is a flowchart illustrating a method for determining a pose of a person in an image according to one embodiment of the present disclosure.

[0019] FIG. 13 is a conceptual diagram illustrating a method for generating a correction image according to one embodiment of the present disclosure.

[0020] FIG. 14 is a block diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

[0021] In describing this disclosure, descriptions of technical details that are well-known in the technical field to which this disclosure pertains and are not directly related to this disclosure will be omitted. This is to avoid obscuring the gist of this disclosure by omitting unnecessary explanations and to convey it more clearly. Furthermore, the terms described below are defined based on their functions in this disclosure and may vary depending on the intent or custom of the user or operator. Therefore, their definitions should be based on the contents of this specification as a whole.

[0022] For the same reason, some components in the attached drawings are exaggerated, omitted, or schematically depicted. Furthermore, the dimensions of each component do not entirely reflect its actual size. Identical or corresponding components in each drawing are assigned the same reference numbers.

[0023] The advantages and features of the present disclosure, and methods for achieving them, will become clearer with reference to the embodiments described below in detail with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms. The disclosed embodiments are provided to ensure that the disclosure of the present disclosure is complete and to fully inform those skilled in the art of the present disclosure of the scope of the disclosure. An embodiment of the present disclosure may be defined according to the claims. Like reference numerals denote like elements throughout the specification. In addition, when describing an embodiment of the present disclosure, if a detailed description of a related function or configuration is determined to unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted. In addition, the terms described below are terms defined in consideration of the functions of the present disclosure and may vary depending on the intention or custom of the user or operator. Therefore, the definitions should be made based on the contents throughout this specification.

[0024] In one embodiment, each block of the flowchart diagrams and combinations of the flowchart diagrams can be performed by computer program instructions. The computer program instructions can be installed on a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, and the instructions, when executed by the processor of the computer or other programmable data processing apparatus, can create means for performing the functions described in the flowchart block(s). The computer program instructions can also be stored in a computer-available or computer-readable memory that can direct a computer or other programmable data processing apparatus to implement the functions in a particular manner, and the instructions stored in the computer-available or computer-readable memory can also produce an article of manufacture that includes instruction means for performing the functions described in the flowchart block(s). The computer program instructions can also be installed on a computer or other programmable data processing apparatus.

[0025] Additionally, each block in the flowchart diagram may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specified logical function(s). In one embodiment, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may be executed substantially simultaneously or, depending on the function, may be executed in reverse order.

[0026] The term '~ unit' used in one embodiment of the present disclosure may represent software or a hardware component such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC), and the '~ unit' may perform a specific role. Meanwhile, the '~ unit' is not limited to software or hardware. The '~ unit' may be configured to be on an addressable storage medium and may be configured to play one or more processors. In one embodiment, the '~ unit' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided through a specific component or a specific '~ unit' may be combined to reduce the number of components or separated into additional components. In addition, in one embodiment, the '~ unit' may include one or more processors.

[0027] Below, the meanings of terms used in this disclosure are explained.

[0028] In this disclosure, the human body is divided into a face and a body. The human body may refer to any part of the body other than the face. The neck, which connects the face and the body, may be divided into two parts and included separately in the face and the body, or may be included as an integral part of the face or the body. This does not limit the technical concept of the present disclosure.

[0029] In the present disclosure, a base image may be an image selected as a base from among a plurality of images. The base image may be selected as an image requiring minimal correction from among the plurality of images. For example, a method according to one embodiment of the present disclosure may replace a portion of the base image with another image based on the base image. In particular, a portion of the base image may be replaced so that the pose of a person in the base image is changed. The term "base image" is used only to mean an image used as a basis for image processing, and may be replaced with various terms such as "reference image" or "base image."

[0030] In the present disclosure, the base face may be a facial shape or image selected from among the faces of individuals within a plurality of images. The base face may be determined for each individual among the individuals within the plurality of images. For example, the base face may be determined based on a score that quantifies the degree of completion of the photographing of the facial region. In another example, the base face may be selected by user input. The base face may be replaced with various terms such as "best face," "base face," and "reference face."

[0031] A method according to one embodiment of the present disclosure can determine a pose of a person by considering connectivity with a base face. For example, a range of poses of a person that can be implemented can be determined through 3D modeling. Within the range of poses of a person that can be implemented, the appropriateness of connectivity between the base face and various poses can be quantified, and the pose of the person can be determined based on the quantified score.

[0032] A person's pose can refer to the posture of the person's upper and lower body, excluding the person's head.

[0033] Connectivity refers to the appropriateness of the connection between a person's face and body regions. In particular, whether the neck area, where the face and body regions meet, is naturally connected can be an important factor in determining connectivity. Connectivity can refer to the degree to which the face region (e.g., the base face) and the body region to be replaced can be naturally connected for a given person.

[0034] In the present disclosure, a keypoint refers to a point within an image that is easily distinguishable or identifiable from the surrounding background, and corresponds to a key point of the body. For example, a keypoint for a hand may include points corresponding to multiple joints within the hand. A keypoint may be expressed as a three-dimensional position coordinate value, which is position information about the x-axis, y-axis, and z-axis of a key point of the body. Of course, the term "keypoint" may be replaced with various terms, such as "feature point."

[0035] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts that are not related to the description are omitted in order to clearly describe the present disclosure, and similar parts are designated with similar reference numerals throughout the specification. In addition, the reference numerals used in each drawing are only for the purpose of describing each drawing, and different reference numerals used in different drawings do not indicate different elements. The present disclosure will be described in detail below with reference to the attached drawings.

[0036] FIG. 1 is a conceptual diagram illustrating a method for generating a correction image according to an embodiment of the present disclosure.

[0037] Referring to FIG. 1, the electronic device can acquire a plurality of images (110; 111, 112, 113, 114).

[0038] The plurality of images (110) may be images captured continuously over a set period of time. The plurality of images (110) may include an image sequence, which refers to a set of images arranged in order over time. For example, the plurality of images (110) may include at least one of a set of images captured over a set period of time, a motional image composed of images captured over a set period of time, and a video.

[0039] In one embodiment, the plurality of images (110) may each represent a group photo including a plurality of individuals. The plurality of images (110) may each be images including the face and body regions of the individuals. For example, the plurality of images (110) may be images including the face and body regions of three individuals, and the number of individuals does not limit the technical concept of the present disclosure.

[0040] In one embodiment, the plurality of images (110) may include a group photo including not only a person but also a plurality of living things. The plurality of images (110) may each include images including face areas and body areas of the plurality of living things. For example, at least one of the plurality of images (110) may be an image including the face and body of a dog, and an image processing method according to one embodiment may perform an operation of replacing the body area of ​​the dog in the base image with an image of the body of the dog in another pose. However, the present invention will be described with a focus on a method of replacing the body area of ​​a person.

[0041] In one embodiment, the plurality of images (110) may be images capturing the appearance of a plurality of people as they change over time. Over time, some of the plurality of images (110) may be images capturing at least one of the plurality of people. For example, one image of the plurality of images (110) may include all of the plurality of people, while another image of the plurality of images (110) may include two people as part of the plurality of people.

[0042] Each person included in the plurality of images (110) may be captured differently for each image, such as by taking different poses or making different facial expressions over time. In particular, with respect to the body poses of the people, each person included in the plurality of images (110) may be captured in a manner such as being obscured by external objects, taking a pose that does not match with other people, or making incomplete movements. This may cause the user to be dissatisfied with the captured images.

[0043] In one embodiment, the electronic device can detect an object from a plurality of images (110). The electronic device can detect a person within the plurality of images (110). For example, the electronic device can detect a plurality of people within the plurality of images (110), and the present disclosure will focus on an operation of detecting a first person (10, 11), which is one person within the plurality of images (110).

[0044] In one embodiment, the electronic device can detect a first person (10) from a plurality of images (110). Specifically, the electronic device can detect the body of the first person (10) from the plurality of images (110). The electronic device can obtain a body region (R1) of the first person (10).

[0045] An electronic device can detect the body of a first person (10) within a plurality of images (110) using an object detection algorithm. The object detection algorithm can be implemented using various deep learning models and is not limited to the technical concepts of the present disclosure.

[0046] In one embodiment, the electronic device may acquire multiple pieces of pose information regarding the pose of the first person (10) from multiple images (110) based on the detection result. For example, the first person (10) in the multiple images (110) may be captured in various poses. The electronic device may acquire multiple pieces of pose information regarding the various poses of the first person (10) in the multiple images (110).

[0047] In one embodiment, the electronic device can obtain a 3D (3-Dimensional) model (120) that reconstructs the pose of a person in three dimensions by three-dimensionally modeling the pose of a person based on a plurality of pose information.

[0048] In Fig. 1, a 3D model (120) expressing six poses is illustrated as an example, but the poses of the person expressed by the 3D model (120) may be more diverse, and the types of poses are not limited.

[0049] In one embodiment, 3D modeling refers to depicting a real object through a 3D model of a virtual space, or creating the shape of an object in a virtual environment by modeling a physical environment. In the present disclosure, 3D modeling refers to depicting the physical posture of a real person or creating a virtual body posture by creating a virtual 3D model of the person's body posture based on a plurality of pose information.

[0050] In one embodiment, the 3D model (120) may include an image or data depicting a body posture of a person according to a plurality of pose information. The 3D model (120) may be information depicting a body posture that is identical to or similar to one of the body postures of the person according to the plurality of pose information.

[0051] In one embodiment, the electronic device may generate a 3D model (120) of a virtual body posture using an artificial intelligence model learned based on the body posture of a person. The electronic device may generate a 3D model (120) of a virtual body posture based on the body posture of the person according to a plurality of pose information.

[0052] For example, the electronic device may obtain first pose information for the first person (10) from a first image (111) among the plurality of images (110), and may obtain second pose information for the first person (11) from a second image (112) among the plurality of images (110). The first pose information may be data regarding a body posture in which both arms are lowered and attached to the sides of the torso. The second pose information may be data regarding a body posture in which both arms are lowered but separated from the torso. Based on the first pose information and the second pose information, the electronic device may generate a virtual body posture as a 3D model (120) corresponding to a body posture of the first person from a state in which the right arm is lowered and attached to the torso to a state in which the right arm is separated from the torso by a certain distance.

[0053] The first person (10) in the first image (111) and the first person (11) in the second image (112) are represented by different drawing symbols for convenience of explanation, but they mean the same person. In other words, the first person (10) and the first person (11) mean the same person, although they are included in different images.

[0054] As another example, the electronic device may obtain first pose information regarding the first person (10) from the first image (111) among the plurality of images (110). The first pose information may be data regarding a body posture in which both arms are lowered and placed next to the torso. Based on the first pose information, the electronic device may generate a virtual body posture related to the posture of the first person with both arms lowered as a 3D model (120). For specific examples, virtual body postures such as a posture in which both arms are lowered and fists are clenched, a posture in which both arms are lowered and palms are facing the floor, a posture in which one arm is lowered and the other arm is raised, etc. may be generated.

[0055] In one embodiment, an electronic device can obtain a pose range regarding a pose of a person that can be implemented in a three-dimensional form through a 3D model (120). The electronic device can obtain the pose range based on a plurality of pose information obtained from a plurality of images (110), and various pose ranges can be determined based on the plurality of pose information.

[0056] In one embodiment, an electronic device can determine a first pose regarding a body posture of a person within a pose range. The electronic device can obtain a virtual body posture of the person in the first pose through a 3D model (120). The electronic device can obtain a body image of the first person in the first pose. The body image can be an image generated based on the 3D model (120) regarding the first pose.

[0057] The operation of creating a 3D model (120) which is a virtual body posture and the operation of creating a body image from the 3D model (120) may be performed through separate models or integrated through one model, and the technical idea of ​​the present disclosure is not limited thereto.

[0058] In one embodiment, the electronic device may determine any one of the plurality of images (110) as the base image. For example, the electronic device may determine the fourth image (114) as the base image. In another example, but different from that illustrated in FIG. 1, the electronic device may also determine the first image (111) of the plurality of images (110) as the base image.

[0059] In one embodiment, the electronic device can generate a corrected image (130) by synthesizing a body image (131) of a first person according to a first pose onto a base image.

[0060] In one embodiment, the electronic device may remove a body region of the first person within the base image. For example, the body region of the first person within the base image may be deleted, erased, or blurred. The electronic device may synthesize a body image (131) of the first person generated according to the first pose onto the body region of the first person removed within the base image.

[0061] In one embodiment, the electronic device may synthesize a body image (131) of a first person based on a face (132) of the first person in a base image. The electronic device may synthesize the body image (131) so that the face (132) of the first person and the body of the first person in the body image are well connected.

[0062] For example, connectivity can be evaluated to quantify whether the face (132) of a first person and the body of the first person in the body image (131) are well connected. The electronic device can synthesize the body image (131) on the base image based on the connectivity. As a specific example, the angle and position of the body image (131) can be changed so that the neck area of ​​the face area (132) of the first person in the base image and the body part connected to the neck in the body image (131) can be well connected. To evaluate whether the neck area and the body part can be well connected, key points indicating major points of the body can be utilized. The electronic device can adjust the angle and position of the body image so that the directions of the neck area and the body part are consistent based on the key points. The electronic device can synthesize the body image (131) on the base image based on the adjusted angle and position. The electronic device can obtain a corrected image (130) by synthesizing the body image (131) on the base image.

[0063] FIG. 2 is a diagram illustrating an overview of a process for generating a correction image according to an embodiment of the present disclosure. For convenience of explanation, details that overlap with those described using FIG. 1 are simplified or omitted.

[0064] Referring to FIG. 2, in step S210, the electronic device can capture an image. The electronic device can include a camera and capture an image through the camera.

[0065] In one embodiment, the electronic device can acquire a captured image. The electronic device can acquire an image captured from an external device. The method of acquiring the image is not limited to the technical concept of the present disclosure.

[0066] In one embodiment, the acquired image may refer to a group photo including multiple individuals. The acquired image may be an image captured continuously over a set period of time. The acquired image may include the facial and body regions of the multiple individuals.

[0067] In step S220, the electronic device can detect an object within an image. The image may include multiple people. The electronic device can detect the body of the person within the image. The electronic device can detect the face and body of the person within the image, respectively. The electronic device can detect the body part, which is an object within the image, using an object detection algorithm. The object detection algorithm does not limit the technical concept of the present disclosure.

[0068] The electronic device may perform an operation of determining a base face in step S230 based on the face and body detected in step S220, and an operation of generating a pose in step S240. The electronic device may generate a face image based on the base face determined in step S230. The electronic device may generate a body image based on the pose generated in step S240.

[0069] In step S250, the electronic device may determine a base image. The electronic device may determine one of the captured images as the base image. In step S260, the electronic device may remove at least one person from the base image. The electronic device may erase at least one person from the base image. The electronic device may remove a body area of ​​at least one person from the base image. The act of removing the person may include editing the image so that the person is not visible, and the method does not limit the technical idea of ​​the present disclosure.

[0070] In step S270, the electronic device can synthesize the face image generated through step S230 and the body image generated through step S240 on the base image. The electronic device can synthesize the face image and the body image on an area in the base image from which at least one person has been erased. The electronic device can synthesize the body image of at least one person on an area in the base image from which the body area of ​​at least one person has been removed.

[0071] The pose of the person in the base image may not match the pose of the person in the synthesized face image and body image. Therefore, the area removed in the base image and the area of ​​the synthesized image may not match accurately. In step S280, the electronic device may perform an image processing technique to fill in the background area. For example, the electronic device may restore the background area through an in-painting technique. The electronic device may restore or fill in an area in the background base image where at least one person has been erased, excluding an area where the face image and body image have been synthesized, as a background area.

[0072] Inpainting technology is a technology for restoring damaged image areas or filling in deleted portions, and the technical idea of ​​the present disclosure does not limit the method of the inpainting algorithm.

[0073] In step S290, the electronic device can obtain a corrected image. The corrected image may be an image in which the pose of the person has been changed from the original image.

[0074] Below, the detailed operations of steps S230 and S240 are described in detail.

[0075] Step S230 may include steps S231 to S233.

[0076] In step S231, the electronic device can classify faces within the captured image. The electronic device can classify faces by person.

[0077] For example, clustering techniques can be used to classify faces. An electronic device can acquire multiple images, each containing multiple individuals. The electronic device can detect the faces of each individual in the images and group the detected faces by individual.

[0078] In step S232, the electronic device can obtain a face score for the acquired face data.

[0079] In the present disclosure, a face score refers to a numerical score quantifying the degree of completion in capturing a person's face. The completion level of shooting refers to the degree to which an image is captured well. From the perspective of the subject of the shooting, the completion level of shooting may refer to the degree to which a person included in the image is captured well. For example, the completion level of shooting may be determined for one person among multiple people included in the image. For one image, multiple completion levels of shooting may be determined, such as the completion level of shooting for a first person, the completion level of shooting for a second person, and the completion level of shooting for a third person. Furthermore, the completion level of shooting may be determined by considering factors related to how the person included in the image was captured. For example, the completion level of shooting may be determined not only by considering the person's facial expression, but also by considering aesthetic aspects such as whether the person included in the image has their eyes closed, whether the person's gaze is directed toward the camera, whether the person's face is shadowed, or whether the person's face is obscured by an object.

[0080] The face score is merely an example means for evaluating a face suitable as a base face among the acquired face data, and the technical idea of ​​the present disclosure is not limited thereto.

[0081] In step S233, the electronic device can determine a base face. The electronic device can determine a base face among the faces grouped for a single person based on a facial score. For example, the electronic device can determine the face with the highest facial score among the faces grouped for a single person as the base face. The electronic device can then generate a facial image based on the base face.

[0082] Step S240 may include steps S241 to S245.

[0083] In step S241, the electronic device can classify a body within the captured image. The electronic device can classify the body by person.

[0084] For example, clustering techniques can be used to classify bodies. An electronic device can acquire multiple images, each containing multiple individuals. The electronic device can detect the bodies of each individual in the images and group the detected bodies by individual and / or body type.

[0085] In step S242, the electronic device can analyze the body shape. From the detected body data, the electronic device can obtain data such as the person's body posture, body shape, and the size and length of each body part.

[0086] In one embodiment, the electronic device can match keypoints to each body part of the detected body data. The electronic device can obtain a posture of the detected body data based on the position data of the keypoints.

[0087] In step S243, the electronic device can three-dimensionally model the body of the person. The electronic device can obtain a three-dimensional model that three-dimensionally reconstructs the pose of the body of the person.

[0088] In one embodiment, an electronic device can obtain a 3D model that three-dimensionally embodies a person's pose by using multiple pieces of body data detected from multiple images. The electronic device can obtain a 3D model that three-dimensionally embodies a person's pose by using the results of analyzing the body shape or pose based on multiple pieces of body data.

[0089] In step S244, the electronic device can obtain a recommended pose. For example, the poses of a person that can be realized through a 3D model may vary. The electronic device can obtain a range of poses of the person that can be realized through the 3D model. The electronic device can determine a recommended pose from among the acquired pose range of the person.

[0090] In one embodiment, the electronic device may determine a recommended pose based on a base face. Some poses within the range of poses of a person that can be realized through a 3D model may not be smoothly connected to the placement of the base face.

[0091] For example, the base face may be a frontal view of the face, while the first pose in the character's pose range may be a rearward-facing pose, with the back facing the front. In this case, combining the base face and the first pose may result in an uneven appearance.

[0092] As another example, the base face may be tilted approximately 45 degrees to the left. In this case, the part of the first pose connected to the neck should be tilted approximately 45 degrees to the left, so that the first pose and the base face can be seamlessly connected.

[0093] Therefore, the electronic device can determine a recommended pose based on the base face.

[0094] In one embodiment, the electronic device can obtain information regarding the position and angle of the base face. The electronic device can obtain a recommended pose based on the information regarding the position and angle of the base face. For example, the electronic device can obtain a pose as a recommended pose based on the position of the base face, in which the position of the connection part with the base face matches. In another example, the electronic device can obtain a pose as a recommended pose based on the orientation of the base face, in which the orientation of the connection part with the base face matches.

[0095] In one embodiment, the electronic device may obtain user input regarding the user's preference for the determined recommended pose. For example, the electronic device may obtain user input regarding "preference" or "dispreference." The electronic device may determine the obtained recommended pose based on the user input regarding "preference." The electronic device may perform the operation according to step S245 based on the user input regarding "preference." The electronic device may delete the obtained recommended pose based on the user input regarding "dispreference." The electronic device may re-perform the operation according to step S244 based on the user input regarding "dispreference."

[0096] In step S245, the electronic device can generate a pose based on the recommended pose. The electronic device can generate the pose of the person using a 3D model. The electronic device can then generate a body image based on the generated pose.

[0097] FIG. 3 is a flowchart illustrating a method for generating a correction image according to an embodiment of the present disclosure.

[0098] For convenience of explanation, parts that overlap with those described using Figures 1 and 2 are simplified or omitted.

[0099] Referring to FIG. 3, in step S310, the electronic device can acquire multiple images, each including multiple people.

[0100] In one embodiment, an electronic device may include a camera module and may capture multiple images, each including multiple individuals, using the camera module. The electronic device may capture multiple images over a set period of time to obtain an image sequence. The electronic device may capture, for example, a video or a motion image.

[0101] In one embodiment, the electronic device can acquire multiple images through communication with a separate server. The multiple images may be multiple images captured continuously over a set period of time.

[0102] In step S320, the electronic device can determine one of the plurality of images as a base image.

[0103] In one embodiment, the electronic device may determine the last image captured among a plurality of sequentially captured images as the base image. Alternatively, the electronic device may determine the first image captured among a plurality of sequentially captured images as the base image. The electronic device may determine one image among the plurality of images as the base image.

[0104] The method for selecting a base image is not limited to the technical concepts of the present disclosure. For example, a base image may be selected based on user input, or a numerical score for the shooting completion may be measured for each image, and the base image may be determined based on the measured score.

[0105] In step S330, the electronic device can obtain multiple pose information regarding the pose of the first person from multiple images.

[0106] In one embodiment, an electronic device may detect a body of a first person within a plurality of images using an object detection algorithm. The electronic device may group image data relating to the body of the first person. The electronic device may obtain a plurality of pose information pieces from the image data relating to the body of the first person.

[0107] For example, an electronic device may obtain first pose information regarding a pose of a first person from a first image among a plurality of images. The electronic device may obtain second pose information regarding a pose of the first person from a second image among the plurality of images. The electronic device may obtain multiple pose information regarding the first person from the multiple images.

[0108] In one embodiment, an electronic device may acquire image data regarding the body of a first person and associate keypoints with body parts of the first person. The keypoints may include three-dimensional position coordinate data corresponding to the body parts of the first person. Based on the keypoints, the electronic device may acquire pose information regarding the posture of the first person.

[0109] In step S340, the electronic device can obtain a pose range for a pose of a person that can be implemented in a three-dimensional form by three-dimensionally modeling the pose of the person based on a plurality of pose information.

[0110] In one embodiment, an electronic device can obtain a 3D model that reconstructs a pose of a person in three dimensions by three-dimensionally modeling the pose of a person based on a plurality of pose information. The 3D model can be generated within a pose range based on the plurality of pose information.

[0111] For example, the plurality of images may be images related to a first person sitting on a chair. The electronic device may acquire multiple pose information from the multiple images. The plurality of pose information may also be postures related to the first person sitting on the chair. The electronic device may acquire a 3D model based on the plurality of pose information of the sitting image, and the range of poses that can be implemented three-dimensionally through the 3D model may be limited to the sitting pose.

[0112] However, there is no limitation on the number of poses used in 3D modeling, and a wider range of poses can be implemented by inputting various poses. The technical concept of the present disclosure is not limited thereto.

[0113] In step S350, the electronic device can determine a first pose within a pose range.

[0114] In one embodiment, the electronic device may select a first pose from among possible poses within a pose range. For example, the electronic device may assign a score to each pose based on user satisfaction, and select the first pose with the highest score. As a specific example, a pose frequently selected by a large number of users in the records may be assigned a high score. The method of assigning scores does not limit the technical concept of the present disclosure.

[0115] In one embodiment, the electronic device may receive a user input for selecting a first pose within a pose range. The electronic device may display example poses that are implementable within the pose range, and receive a user input for selecting one of the example poses.

[0116] In one embodiment, the electronic device may receive an image of a person striking a first pose. Based on the image of the person striking the first pose, the electronic device may select a pose within a pose range that is identical to or similar to the first pose.

[0117] In one embodiment, the electronic device can select a first pose, a second pose, and a third pose from among possible poses within a pose range. The electronic device can provide a pose selection list including the first pose to the third pose.

[0118] In step S360, the electronic device can generate a corrected image by synthesizing a body image of the first person according to the first pose onto a base image.

[0119] In one embodiment, the electronic device can erase a body area of ​​a first person within a base image. The electronic device can synthesize a body image of the first person onto the erased area within the base image.

[0120] In one embodiment, the electronic device may synthesize a body image of a first person based on the facial region of the first person in the base image. For example, the electronic device may obtain the orientation and position of the face of the first person in the base image. The electronic device may synthesize a body image of the first person so that it can be associated with the orientation and position of the face of the first person.

[0121] FIG. 4 is a conceptual diagram illustrating a method for detecting an object in an image according to one embodiment of the present disclosure.

[0122] For convenience of explanation, the explanation using FIGS. 1 and 2 is redundant and therefore is simplified or omitted. For reference, FIG. 4 is a conceptual diagram explaining steps S210 and S220 of FIG. 2.

[0123] Referring to FIG. 4, the electronic device can acquire a plurality of images (110; 111, 112, 113, 114).

[0124] In one embodiment, the electronic device can detect an object from a plurality of images (110). The electronic device can detect a person within the plurality of images (110). The electronic device can detect each of the plurality of people within the plurality of images (110).

[0125] In one embodiment, the electronic device can detect a body part of a person from a plurality of images (110). The electronic device can detect the face of the person and the body of the person, respectively, from the plurality of images (110).

[0126] The electronic device can detect body parts of a person in images using an object detection algorithm, and the object detection algorithm does not limit the technical idea of ​​the present disclosure.

[0127] In one embodiment, the electronic device can detect a face (411) of a first person and a body (412) of the first person from a first image (111). The electronic device can detect a face (421) of a second person and a body (422) of the second person from the first image (111). The electronic device can detect a face (431) of a third person and a body (432) of the third person from the first image (111).

[0128] Although the method for detecting objects in the first image (111) has been described, the method for detecting objects in the second image (112), third image (113), and fourth image (114) is also the same, so it is omitted.

[0129] FIG. 5 is a conceptual diagram illustrating a method for classifying objects in an image according to one embodiment of the present disclosure.

[0130] For convenience of explanation, parts that overlap with those described using Figures 1 and 4 are simplified or omitted.

[0131] Referring to FIG. 5, an electronic device can acquire a plurality of images (110; 111, 112, 113, 114). The electronic device can detect an object from the plurality of images (110). The electronic device can detect a person's face and a person's body, respectively, from the plurality of images (110).

[0132] In one embodiment, the first image (111) may be a group photo including a first person, a second person, and a third person. The electronic device may detect the face and the body of the first person from the first image (111), respectively. The electronic device may obtain a first face image (511) regarding the face of the first person and a first body image (512) regarding the body of the first person from the first image (111).

[0133] In one embodiment, the second image (112), the third image (113), and the fourth image (114) may also be group photos including the first person, the second person, and the third person, respectively. The electronic device may detect the face of the first person and the body of the first person from the second image (112), respectively. The electronic device may obtain a second face image (521) regarding the face of the first person and a second body image (522) regarding the body of the first person from the second image (112). The electronic device may obtain a third face image (531) regarding the face of the first person and a third body image (532) regarding the body of the first person from the third image (113). The electronic device may obtain a fourth face image (541) regarding the face of the first person and a fourth body image (542) regarding the body of the first person from the fourth image (114).

[0134] In one embodiment, the electronic device can classify body images of a body of a first person obtained from a plurality of images (110). The electronic device can group body images of a body of the first person obtained from a plurality of images (110). The method of grouping the body images can be performed through a clustering technique, and the technical idea of ​​the present disclosure is not limited thereto. For example, the electronic device can classify body images (512, 522, 532, 542) of a body of the first person from a plurality of images (110), thereby obtaining a first cluster of the body of the first person.

[0135] In one embodiment, the electronic device can classify at least one of a body image of a body of a first person obtained from a plurality of images (110) and a face image of a face of the first person. The electronic device can group at least one of a body image of a body of a first person obtained from a plurality of images (110) and a face image of a face of the first person.

[0136] In one embodiment, the electronic device can classify a facial image of a first person obtained from a plurality of images (110). The electronic device can obtain a body image of the first person's body corresponding to each classified facial image. Accordingly, the electronic device can classify the body image based on the correspondence between the facial image and the body image.

[0137] For example, a first cluster may include a first face image (511). An electronic device may obtain a first body image (512) corresponding to the first face image (511) within the first cluster. An electronic device may obtain a second body image (522) corresponding to the second face image (521) within the first cluster. Based on the correspondence between the face and body of the first person, the electronic device may obtain a first cluster that classifies body images related to the body of the first person.

[0138] In one embodiment, the electronic device can classify a body image of a first person's body and a face image of the first person's face, respectively, obtained from a plurality of images (110). The electronic device can correspond the classified body images and face images. For example, the electronic device can group the first to fourth face images (511, 521, 531, 541). The electronic device can group the first to fourth body images (512, 522, 532, 542). The electronic device can correspond the first face image (511) to the first body image (512). The electronic device can correspond the second face image (521) to the second body image (522). The electronic device can correspond the third face image (531) to the third body image (532). The electronic device can correspond the fourth face image (541) and the fourth body image (542).

[0139] In the present disclosure, a face image and a body image extracted from the same image may be expressed as corresponding to each other. For example, an electronic device may obtain a first face image (511) and a first body image (512) from a first image (111). The first face image (511) and the first body image (512) may correspond to each other.

[0140] In one embodiment, FIG. 5 illustrates that each of the plurality of images (110) includes three individuals, each including the face and body of each individual. However, unlike the illustration, the first image (111) may be an image that includes the face of the first individual but does not include the body of the first individual, and the technical concept of the present disclosure is not limited thereto.

[0141] The electronic device can classify a body image of a first person's body and a face image of the first person's face, respectively, obtained from a plurality of images (110). However, if the body of the first person is not included in the first image (111), the electronic device may not be able to extract the first body image (512) from the first image (111). The electronic device may obtain a first cluster excluding the first body image (512). The first cluster may include data corresponding to each other regarding the body and face of the first person, but may partially include only the face image (511) of the first person corresponding to the first image (111).

[0142] In one embodiment, the electronic device can match the classified body image and face image. If the first body image (512) of the body of the first person is not included in the first image (111), the electronic device can extract only the first face image (511) from the first image (111). The electronic device can generate the first body image (512) corresponding to the first face image (511). The electronic device can generate the first body image (512) based on the data within the first cluster.

[0143] The description of the action of classifying body images has been described with respect to the first person, but it can of course be performed in the same way for the second person, the third person, etc.

[0144] In one embodiment, the electronic device can obtain contextual information regarding whether the body image of a person in the image includes the entire body. For example, the contextual information may include information regarding whether only the upper body is present, the arms are covered, or the entire body is covered.

[0145] In one embodiment, an electronic device can classify body images based on contextual information. For example, the electronic device can group body images that only show the upper body among a plurality of images (110). The electronic device can acquire a cluster of body images classified among the plurality of images (110) that only show the upper body. The acquired cluster can be helpful in generating a 3D model for three-dimensionally representing only the upper body.

[0146] In one embodiment, the electronic device may select body images within a cluster based on contextual information to generate a 3D model. For example, body images including the lower body may be selected to generate a 3D model that accurately represents the lower body posture.

[0147] FIG. 6 is a flowchart illustrating a method for classifying objects in an image according to one embodiment of the present disclosure.

[0148] For convenience of explanation, parts that overlap with those described using Figure 3 are simplified or omitted.

[0149] Referring to FIG. 6, step S330 of FIG. 3 may include steps S610, S620, and S630.

[0150] In step S610, the electronic device can detect the face and body of the first person in each of the plurality of images.

[0151] In one embodiment, the electronic device may detect an object within an image using an object detection algorithm. The electronic device may detect the face and body of a person within the image.

[0152] For example, an electronic device may find a candidate region for a detection target to recognize an object of interest within an image. The electronic device may obtain the location of the detection target using a bounding box. The electronic device may determine a bounding box corresponding to the location of the detection target among the candidate regions for the detection target, and determine the location of the object of interest based on the determined bounding box. The electronic device may then classify objects within the determined bounding box.

[0153] In one embodiment, each of the plurality of images may include a first person. For example, the first image may be an image including a first person, and the second image may be an image including the first person. The first image and the second image may be images captured at different points in time, and the first person in the first image and the first person in the second image may assume different poses, but the first person refers to the same person.

[0154] The electronic device can detect the face and body of the first person from the first image, and can detect the face and body of the first person from the second image.

[0155] In step S620, the electronic device can group multiple body images of the first person extracted from multiple images based on the detection result.

[0156] In one embodiment, the electronic device may acquire a body image of the detected body based on the detection results. However, the acquisition of body data is merely exemplary, and the types of data are merely examples. For example, the electronic device may also acquire location information of key points corresponding to key points on the body as body data based on the detection results.

[0157] In one embodiment, the electronic device can group body images of a first person extracted from a plurality of images. The electronic device can classify body images of the first person extracted from the plurality of images.

[0158] In one embodiment, an electronic device may classify objects within an image using a clustering technique. The electronic device may classify a body image of a first person extracted from a plurality of images. The electronic device may classify the body image of the first person.

[0159] In step S630, the electronic device can obtain multiple pose information from multiple grouped body images.

[0160] In one embodiment, an electronic device can obtain pose information from a body image. The electronic device can obtain pose information corresponding to a posture of a body of a person included in the body image.

[0161] FIG. 7 is a flowchart illustrating a method for classifying objects in an image according to one embodiment of the present disclosure.

[0162] For convenience of explanation, parts that overlap with those described using FIGS. 3 and 6 are simplified or omitted.

[0163] Referring to FIG. 7, step S620 of FIG. 6 may include steps S710 and S720. In one embodiment, the electronic device may group facial images, thereby consequently grouping body images. This will be described in detail below.

[0164] In step S710, the electronic device can group multiple facial images of the first person extracted from multiple images based on the detection result.

[0165] In one embodiment, the electronic device may acquire a facial image of the detected face based on the detection results. However, the data acquired is merely data about the face, and the type of data is merely exemplary. For example, the electronic device may also acquire information about the direction and position of the face as facial data based on the detection results.

[0166] In one embodiment, the electronic device can group facial images of a first person extracted from a plurality of images. The electronic device can classify facial images of the first person extracted from the plurality of images.

[0167] In one embodiment, an electronic device can classify objects within an image using a clustering technique. The electronic device can classify facial images of a first person extracted from a plurality of images. The electronic device can classify facial images of the first person. The electronic device can obtain facial images of the face of the first person included in the plurality of images, and classify the facial images of the face of the first person to obtain a first cluster.

[0168] In step S720, the electronic device can group the plurality of body images by matching the grouped plurality of face images with the plurality of body images.

[0169] In one embodiment, an electronic device may acquire body images from a plurality of images. The electronic device may correspond face images included in a cluster to body images. For example, a first cluster may include face images of a face of a first person among the plurality of images. A first face image of the face of the first person acquired from the first image may correspond to a first body image of the body of the first person acquired from the first image. A second face image of the face of the first person acquired from the second image may correspond to a second body image of the body of the first person acquired from the second image.

[0170] In one embodiment, an electronic device may acquire a cluster comprising pairs of face images and body images as data. The cluster may include data pairing face images and body images of the same person acquired from a single image. For example, the electronic device may acquire a first cluster comprising data regarding a first person. The acquired first cluster may include data pairing a first face image regarding the face of the first person acquired from the first image and a first body image regarding the body of the first person acquired from the first image.

[0171] However, unlike FIG. 7, the electronic device can group face images and body images simultaneously. The electronic device can classify face images and body images simultaneously. For example, the electronic device can group face images and body images acquired from a plurality of images, respectively, by person, and acquire a first cluster including data about the face and body of a first person. The electronic device can correspond the face images and body images included in the first cluster. The electronic device can correspond the face image of the first person acquired from the first image included in the first cluster with the body image of the first person acquired from the first image.

[0172] The method of grouping face images and body images separately can help to obtain a cluster without a problem excluding the body image of the first person that should have been extracted from the second image, even if the body of the first person is not included in the second image among the multiple images.

[0173] The method of grouping body images is exemplarily explained through the embodiment of FIG. 7, and the technical idea of ​​the present disclosure is not limited to the method of grouping body images.

[0174] FIG. 8 is a conceptual diagram illustrating a method for obtaining pose information of a person from an image according to one embodiment of the present disclosure.

[0175] For convenience of explanation, parts that overlap with those described using Figures 1 and 2 are simplified or omitted.

[0176] Referring to FIG. 8, in one embodiment, an electronic device may acquire a plurality of images. The electronic device may acquire a first image (810) among the plurality of images. The plurality of images may be group photos including a plurality of individuals, and the first image (810) may also be a group photo including a plurality of individuals.

[0177] For example, the first image (810) may include three people. FIG. 8 describes an operation of acquiring pose information based on a person located within the dotted box on the right side of the first image (810).

[0178] In one embodiment, the electronic device may acquire keypoints regarding body parts of a person in a first image (810). The electronic device may designate keypoints for the person in the first image (810). The electronic device may match the keypoints to the major body points of the person in the first image (810). The electronic device may acquire keypoints corresponding to the major body points of the person in the first image (810).

[0179] In Fig. 8, a keypoint image (820) is illustrated to explain keypoints. While the electronic device can acquire the keypoint image (820) from the first image (810), it is not limited to the image format, and keypoints regarding key points of a person's body can be acquired from the first image (810). For example, the electronic device can acquire location information regarding keypoints.

[0180] In one embodiment, keypoints corresponding to key points of the body of a person in the first image (810) are displayed on the keypoint image (820). The pose of the person in the first image (810) and the body pose formed by the keypoints in the keypoint image (820) may be identical.

[0181] In one embodiment, keypoints may correspond to key locations within the face of the subject's body. For example, keypoint locations may include the eyes, nose, mouth, chin, cheekbone protrusions, neck, chest, stomach, and collarbone for determining the orientation of the upper body, hands, elbows, and shoulders for determining the posture of the upper body, and ankles, knees, and hips for determining the posture of the lower body.

[0182] In one embodiment, the keypoints may include location information regarding key points of the target person's body. The keypoints may include information regarding key points of the target person's body. For example, one of the keypoints may include information indicating that the target person's hand is positioned at a first location. The first location may be expressed as two-dimensional coordinates. As another example, another of the keypoints may include information indicating that the target person's shoulder is positioned at a second location.

[0183] Therefore, for example, a pose, including the position and orientation of a subject's arms, can be determined using three key points: the shoulder, elbow, and hand. The electronic device can acquire multiple pieces of pose information based on the key points. The pose information can include information about the subject's body posture, including the arm posture, upper body posture, and lower body posture.

[0184] Similarly, in FIG. 8, a pose information image (830) is illustrated to explain pose information. The electronic device can obtain the pose information image (830) from key points, but is not limited to the image format, and can obtain pose information about the body posture of a person from the first image (810).

[0185] FIG. 9 is a flowchart illustrating a method for obtaining pose information of a person from an image according to one embodiment of the present disclosure.

[0186] For convenience of explanation, parts that overlap with those described using FIGS. 3 and 8 are simplified or omitted.

[0187] Referring to FIG. 9, step S330 of FIG. 3 may include step S910 and step S920.

[0188] In step S910, the electronic device can obtain key points regarding a body part of a first person within a plurality of images.

[0189] In one embodiment, the electronic device may acquire keypoints corresponding to body parts of a first person within a plurality of images. The keypoints may include location information.

[0190] For example, an electronic device may detect a body part of a first person within one of multiple images. The electronic device may match a keypoint to the detected body part. The electronic device may obtain location information of the matched keypoint and information regarding the corresponding body part.

[0191] In one embodiment, the electronic device may acquire multiple keypoints for each image. For example, the electronic device may acquire a keypoint set consisting of multiple keypoints for a first image. The electronic device may acquire a keypoint set consisting of multiple keypoints for a second image. The keypoint set may include keypoints corresponding to each body part.

[0192] In step S920, the electronic device can obtain multiple pose information based on key points.

[0193] For example, an electronic device may acquire a set of keypoints for a first image. The set of keypoints may include keypoints corresponding to each body part. Based on the acquired set of keypoints, the electronic device may acquire pose information regarding the posture of a person in the first image.

[0194] In one embodiment, the electronic device can obtain a set of keypoints from each of the images, and can obtain pose information about a posture taken by a person in each of the images based on the set of keypoints.

[0195] FIG. 10 is a conceptual diagram illustrating a method for generating a 3D model of a person's pose based on an image and pose information of the person according to one embodiment of the present disclosure.

[0196] For convenience of explanation, parts that overlap with those described using FIGS. 1 and 8 are simplified or omitted.

[0197] Referring to FIG. 10, the electronic device can obtain a 3D model that three-dimensionally implements a person's pose based on a plurality of images and a plurality of pose information.

[0198] In one embodiment, an electronic device may acquire multiple images. The electronic device may group multiple body images of a single person included in the multiple images. The electronic device may acquire a clustered image (910) that is a grouped body image.

[0199] For example, the clustering image (910) may refer to a set of body images of each cluster described in FIG. 5. Specifically, the clustering image (910) may include body images (512, 522, 532, 542) included in the first cluster of FIG. 5. Alternatively, the clustering image (910) may include body images (612, 622, 632, 642) included in the second cluster of FIG. 5. Alternatively, the clustering image (910) may include body images (712, 722, 732, 742) included in the third cluster of FIG. 5. The clustering image (910) may be a set of images grouped for the body of one person within a plurality of images.

[0200] In one embodiment, the electronic device can obtain multiple pose information (920) based on a clustering image (910). The operation of obtaining multiple pose information is the same as that described in FIG. 8, and thus is omitted.

[0201] In one embodiment, the electronic device can obtain a 3D model that stereoscopically implements a pose of a person based on a plurality of images and a plurality of pose information.

[0202] In one embodiment, an electronic device can generate a three-dimensional image of a person's pose by inputting a plurality of images and a plurality of pose information into a generative artificial intelligence (AI) model. The electronic device can generate a plurality of three-dimensional images that embody various poses based on the generative AI model, and can obtain a pose range that can be derived based on the plurality of three-dimensional images. The pose range can be determined based on the plurality of pose information (920). The electronic device can obtain a three-dimensional model (930) that embodies the person's pose three-dimensionally based on the pose range.

[0203] However, it should be noted that generative AI models can perform actions that go beyond simply generating three-dimensional images of a person's posture, ultimately creating a 3D model. In this regard, the scope of actions performed by electronic devices using generative AI models is not limited to the technical concepts of the present disclosure.

[0204] In one embodiment, the electronic device can obtain a 3D model (930) modeled according to a plurality of pose information (920). The poses of a person that can be implemented by the 3D model (930) can be determined based on the plurality of pose information (920).

[0205] In one embodiment, the electronic device can determine a pose range based on a plurality of pose information (920). A pose of a person that can be implemented by a 3D model (930) can be included within the pose range.

[0206] For example, an electronic device can obtain first pose information and second pose information. The electronic device can implement the posture of a person corresponding to the first pose information through a 3D model (930), and can implement the posture of a person corresponding to the second pose information through the 3D model (930). In addition, the electronic device can implement the posture of a person having an intermediate form between the first pose information and the second pose information through the 3D model (930). If there is more pose information representing the posture of a person between the first pose information and the second pose information, the posture of the person implemented by the electronic device through the 3D model (930) can be more accurate.

[0207] As a specific example, the first pose information may be information about a posture in which the right arm is extended in the lateral direction. The second pose information may be information about a posture in which the right arm is extended upward. The electronic device can implement, based on the first pose information and the second pose information, a posture in which the right arm is extended in the right diagonal direction through the 3D model (930).

[0208] For another example, an electronic device can obtain first pose information. The electronic device can implement the posture of a person corresponding to the first pose information through a 3D model (930). The electronic device can implement a posture of a person similar to the first pose information through the 3D model (930).

[0209] As a specific example, the first pose information may be information about a victory gesture. The electronic device can implement, based on the first pose information, a posture in which the arms and legs are spread wide through the 3D model (930).

[0210] In one embodiment, the electronic device may implement a body of a person within the clustered image (910) through a 3D model (930) based on the clustered image (910). For example, each person may have different physical characteristics, clothing, etc. Various physical characteristics are possible, such as a thin body type, a fat body type, a body type with long limbs, etc., and clothing is diverse.

[0211] An electronic device can obtain information about the physical characteristics and clothing of a person from a clustered image (910). Based on the obtained information about at least one of the physical characteristics and clothing of the person, the electronic device can implement the body of the person through a 3D model (930). The 3D model (930) can be a three-dimensionally implemented model that reflects the physical characteristics and clothing of the person in the clustered image (910).

[0212] In one embodiment, the electronic device can generate a 3D model (930) regarding a person's pose by inputting a plurality of images and a plurality of pose information into a generative AI model. The electronic device can generate the 3D model (930) using a Retrieval-Augmented Generation (RAG) technique that improves the results of the generative AI model by additionally utilizing at least one of the plurality of images and the plurality of pose information already present. However, this is merely an example, and the technique for generating the 3D model (930) does not limit the technical idea of ​​the present disclosure.

[0213] In one embodiment, an electronic device may obtain pose selection information. For example, the electronic device may obtain pose selection information based on a user input specifying a specific pose. The electronic device may obtain a 3D model (930) corresponding to the body posture based on the pose selection information.

[0214] FIG. 11 is a drawing for explaining a method for generating a correction image according to one embodiment of the present disclosure.

[0215] For convenience of explanation, parts that overlap with those described using Figure 3 are simplified or omitted.

[0216] Referring to FIG. 11, step S360 of FIG. 3 may include steps S1110 and S1120.

[0217] In step S1110, the electronic device can generate a body image of a pose of a first person taking the first pose based on the first pose and the plurality of images.

[0218] In one embodiment, the electronic device can obtain a 3D model of a body posture in a first pose.

[0219] In one embodiment, an electronic device may acquire a body image corresponding to a 3D model of a body posture in a first pose based on a plurality of images. For example, the electronic device may acquire the body image as an output value by inputting the plurality of images and the 3D model of the first pose into a generative AI model. The acquired body image may be an image of a body in the first pose with the body shape and clothing of a first person in the plurality of images. The body image may be a 2D image.

[0220] In step S1120, the electronic device can generate a correction image by synthesizing a body image onto a base image.

[0221] In one embodiment, the electronic device can delete an area where a first person is located on a base image. For example, a process of erasing the person from the image can be performed. In one embodiment, the electronic device can synthesize a body image of the first person in a first pose over the area where the first person was deleted. The electronic device can obtain an image combining the base image and the body image as a corrected image.

[0222] In one embodiment, the electronic device can synthesize a body image based on a facial region of a first person in a base image. The electronic device can determine a position for synthesizing the body image so that the facial region of the first person and the body region of the first person in the body image are seamlessly connected. The electronic device can synthesize the body image at the determined position in the base image.

[0223] FIG. 12 is a flowchart illustrating a method for determining a pose of a person in an image according to one embodiment of the present disclosure.

[0224] For convenience of explanation, parts that overlap with those described using Figure 3 are simplified or omitted.

[0225] Referring to FIG. 12, step S340 of FIG. 3 may include steps S1210, S1220, and S1230.

[0226] In step S1210, the electronic device can obtain a first range of poses of a person that can be modeled in a three-dimensional form based on a plurality of pose information.

[0227] In one embodiment, an electronic device can obtain a 3D model by three-dimensionally modeling a pose of a person based on a plurality of pose information. The electronic device can implement a pose of a given person using the 3D model. The electronic device can obtain a 3D model of a given person in a given pose.

[0228] The first range may refer to a range of poses of a person that can be three-dimensionally implemented through a 3D model. For example, an electronic device may acquire a 3D model based on multiple pose information using a generative model. By inputting multiple pose information into the generative model, the electronic device may output a 3D model that expresses the pose of the person in three-dimensional form. The 3D model may implement the pose of the person within the first range based on the accuracy and diversity of the multiple pose information.

[0229] In step S1220, the electronic device can obtain a second range of poses of a person that can be connected to the face of the first person in the base image.

[0230] In the present disclosure, the degree of connection between a face and a pose may refer to the degree to which a face and a body pose are seamlessly connected. For example, the degree to which a face and a body pose are seamlessly connected may be quantified, and if the degree exceeds a threshold, the face and pose may be determined to be connected.

[0231] In one embodiment, the possibility of connection between a face and a body pose may be determined based on the connection location. That is, the possibility of connection may be determined based on whether the connection site between the face and the body pose (e.g., the neck) can be located at the same location.

[0232] In one embodiment, the possibility of connection between a face and a body pose may be determined based on the direction of connection. That is, the possibility of connection may be determined based on whether the connection site between the face and the body pose (e.g., the neck) is facing each other.

[0233] For example, the face of the first person in the base image may be a frontal view of the face. In this case, a pose that is backward-facing and has the back facing the front may produce an uneven appearance when combined with the face of the first person in the base image. The pose that has the back facing the front may be excluded from the second range. Additionally, a pose that has the chest facing the front may produce a smooth appearance when combined with the face of the first person in the base image. The pose that has the chest facing the front may be included in the second range.

[0234] In one embodiment, the electronic device can obtain a full range of poses within the database. From the full range, the electronic device can determine a second range of poses of a person that can be associated with the face of the first person within the base image.

[0235] In step S1230, the electronic device may acquire a pose range that falls within both the first range and the second range. In one embodiment, the electronic device may acquire a pose range that overlaps the first range and the second range. The acquired pose range may be a pose range of a person that can be implemented as a 3D model, and may be a pose range that can be connected to the face of the first person in the base image.

[0236] FIG. 13 is a conceptual diagram illustrating a method for generating a correction image according to an embodiment of the present disclosure. For convenience of explanation, details that overlap with those described using FIGS. 1 and 2 are simplified or omitted.

[0237] Referring to FIG. 13, in step S1310, the electronic device may acquire a base image. The electronic device may acquire multiple images and determine a base image from among the multiple images. Each of the multiple images may represent a group photo including multiple individuals.

[0238] In one embodiment, the electronic device may select a base image from among a plurality of images, as the image requiring the least amount of correction. For example, the electronic device may obtain a numerical score for the degree of completion of the photographing for the plurality of images, and select the base image based on the obtained score. In another example, the electronic device may select the last image obtained from among the plurality of images as the base image. The technical concept of the present disclosure is not limited to examples of selecting a base image.

[0239] In step S1320, the electronic device may erase a person within the base image. For example, the electronic device may delete color data within the person area within the base image. As another example, the electronic device may change the color data within the person area within the base image to white. The method for removing the person is to edit the image so that the person is not visible using image editing technology, and the method is not limited thereto.

[0240] In step S1330, the electronic device can synthesize a base face on the area from which the person has been removed within the base image. The method for obtaining the base face is the same as that described using step S230 of FIG. 2, and thus is omitted.

[0241] In one embodiment, the electronic device can synthesize a base face on an area where a person's face was located among areas where a person has been removed from a base image. The electronic device can obtain the location of the person's face in the base image, and synthesize the base face after removing the person from the base image based on the obtained location. The base face may be an image extracted from an image other than the base image. The base face may be determined through step S233 of FIG. 2.

[0242] In step S1340, the electronic device can synthesize a body image onto the base image. The operation of generating the body image is identical to that described using step S240 of FIG. 2, and thus is omitted.

[0243] In one embodiment, the electronic device can synthesize a body image on an area where the body of the person was located among areas where the person has been removed from the base image. The electronic device can obtain the location of the person's body in the base image, and synthesize the body image after removing the person from the base image based on the obtained location. The body image may be an image extracted from an image other than the base image.

[0244] In one embodiment, the electronic device can synthesize a body image on a base image based on a base face. The electronic device can synthesize the body image based on the position and orientation of the base face. The electronic device can synthesize the body image so that the position and orientation of the base face and the body of the body image are aligned, thereby seamlessly connecting the face and body.

[0245] For example, an electronic device may determine where to synthesize a body image so that the neck, which is the connection point between the base face and the body in the body image, is seamless. The electronic device may determine where to synthesize the body image by considering factors such as whether the neck, which is the connection point between the body and face, is aligned in position to ensure a seamless connection between the body and face, whether the neck, which is the connection point between the body and face, is aligned in orientation, and whether the orientation of the body and the orientation of the face form a normal human posture.

[0246] In step S1350, the electronic device may synthesize a background within the base image. The electronic device may synthesize a background image over some of the areas within the base image from which the person has been removed. The electronic device may fill in the background over some of the areas within the base image from which the person has been removed using an image-generating model.

[0247] The area where the person is removed through step S1320 and the area where the person's face and body are newly synthesized through steps S1330 and S1340 may not match each other. For example, since the pose of the person in the base image and the pose of the same person synthesized through S1340 are different, there may be a blank area (A1) where the image is not synthesized. The blank area (A1) may be a part of the base image that has been erased due to differences in the person's pose between the images. The electronic device can obtain the blank area (A1) where the image is not filled even after the base face and body are synthesized.

[0248] In one embodiment, the electronic device can synthesize a background on the blank area (A1). The electronic device can synthesize a background image on the blank area (A1) using an image generation model. The electronic device can fill the blank area (A1) with a background using the image generation model. The electronic device can correct the image using the image generation model so that the blank area (A1) of the base image after step S1340 is filled with a background. The electronic device can fill the blank area (A1) with a background using an in-painting technique.

[0249] Hereinafter, with reference to FIG. 14, the configuration of an electronic device for performing the image processing operations described so far will be described. FIG. 14 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure.

[0250] For convenience of explanation, parts that overlap with those described using Figures 1 to 13 are simplified or omitted.

[0251] Referring to FIG. 14, an electronic device (1000) according to an embodiment may include an input / output interface (1100), a memory (1200), and a processor (1300). However, the components of the electronic device (1000) are not limited to the above-described examples, and the electronic device (1000) may include more or fewer components than the above-described components. In an embodiment, some or all of the input / output interface (1100), the memory (1200), and the processor (1300) may be implemented in the form of a single chip, and the processor (1300) may include one or more processors.

[0252] The input / output interface (1100) may include an input interface (e.g., touch screen, hard button, microphone, etc.) for receiving control commands or information from a user, and an output interface (e.g., display panel, speaker, etc.) for displaying the results of execution of an operation according to the user's control or the status of the electronic device (1000).

[0253] For example, the electronic device (1000) can acquire a plurality of images based on a user's image capturing command obtained through the input / output interface (1100). The processor (1300) of the electronic device (1000) can acquire a 3D model that three-dimensionally implements a pose of a person based on the plurality of images, and perform the image processing operations described using FIGS. 1 to 13.

[0254] The memory (1200) is a configuration for storing various programs or data, and may be configured as a storage medium such as a ROM, a RAM, a hard disk, a CD-ROM, and a DVD, or a combination of storage media. The memory (1200) may not exist separately and may be configured to be included in the processor (1300). The memory (1200) may be configured as a volatile memory, a non-volatile memory, or a combination of volatile memory and non-volatile memory. Programs or instructions for performing operations according to the embodiments described with reference to FIGS. 1 to 13 may be stored in the memory (1200). The memory (1200) may also provide stored data to the processor (1300) upon request of the processor (1300).

[0255] The processor (1300) controls a series of processes to operate the electronic device (1000) according to the embodiments described with reference to FIGS. 1 to 13, and may be composed of one or more processors. One or more processors included in the processor (1300) may be circuitry such as a System on Chip (SoC), an Integrated Circuit (IC), etc. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU, a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. For example, when one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0256] The processor (1300) can write data to the memory (1200) or read data stored in the memory (1200), and in particular, process data according to predefined operation rules or artificial intelligence models by executing a program or at least one instruction stored in the memory (1200). Accordingly, the processor (1300) can perform the operations described in the embodiments described above, and the operations described as being performed by the electronic device (1000) in the embodiments described above can be regarded as being performed by the processor (1300) unless otherwise specifically described.

[0257] A method according to one embodiment may include a step of acquiring a plurality of images, each including a plurality of persons. The method may include a step of determining one of the plurality of images as a base image. The method may include a step of acquiring a plurality of pose information regarding a pose of a first person among the plurality of persons from the plurality of images. The method may include a step of acquiring a pose range regarding a pose of the person that can be implemented in a three-dimensional form by three-dimensionally modeling the pose of the person based on the plurality of pose information. The method may include a step of determining a first pose within the pose range. The method may include a step of generating a corrected image by synthesizing a body image of the first person striking the first pose onto the base image.

[0258] In one embodiment, the plurality of images may be a plurality of images captured continuously over a set period of time.

[0259] In one embodiment, the step of acquiring a plurality of pose information may include the step of acquiring key points regarding a body part of a first person within the plurality of images. The step of acquiring a plurality of pose information may include the step of acquiring a plurality of pose information based on the key points.

[0260] In one embodiment, the step of obtaining a plurality of pose information may include a step of detecting a face and a body of a first person in the plurality of images, respectively. The step of obtaining a plurality of pose information may include a step of grouping a plurality of body images of the first person extracted from the plurality of images based on the detection results. The step of obtaining a plurality of pose information may include a step of obtaining a plurality of pose information from the grouped plurality of body images.

[0261] In one embodiment, the step of grouping the plurality of body images may include the step of grouping the plurality of facial images of the first person extracted from the plurality of images based on the detection result. The step of grouping the plurality of body images may include the step of grouping the plurality of body images by matching the grouped plurality of facial images with the plurality of body images.

[0262] In one embodiment, the pose of the first person may be a pose of the upper and lower body of the first person, excluding the face.

[0263] In one embodiment, the step of generating a corrected image may include a step of generating a body image of a first person in a pose of the first person, based on the first pose and the plurality of images. The step of generating the corrected image may include a step of generating the corrected image by synthesizing the body image onto a base image.

[0264] In one embodiment, the step of acquiring a pose range may include a step of acquiring a first range of poses of a person that can be modeled in a three-dimensional form based on a plurality of pose information. The step of acquiring a pose range may include a step of acquiring a second range of poses of a person that can be connected to the face of the first person in the base image. The step of acquiring a pose range may include a step of acquiring a pose range that falls within both the first range and the second range.

[0265] In one embodiment, the step of generating the corrected image may include the step of removing a body region of the first person in the base image. The step of generating the corrected image may include the step of synthesizing the body image over the removed region.

[0266] In one embodiment, the step of generating the correction image may further include the step of synthesizing a background image over some of the removed areas.

[0267] An electronic device according to an embodiment may include an input / output interface, a memory, and at least one processor. The input / output interface may receive a user input requesting image processing, and output a processed image according to the user input. The memory may store commands for processing the image. By having at least one processor execute a program or at least one instruction stored in the memory, the electronic device may obtain a plurality of images each including a plurality of persons, determine one of the plurality of images as a base image, obtain a plurality of pose information regarding a pose of a first person among the plurality of persons from the plurality of images, and model the pose of the person in three dimensions based on the plurality of pose information, thereby obtaining a pose range regarding a pose of the person that can be implemented in three dimensions, determining a first pose within the pose range, and synthesizing a body image of the first person striking the first pose onto the base image, thereby generating a corrected image.

[0268] In one embodiment, the plurality of images may be a plurality of images captured continuously over a set period of time.

[0269] In one embodiment, the electronic device can obtain a key point regarding a body part of a first person in a plurality of images and obtain a plurality of pose information based on the key point by having at least one processor execute a program stored in a memory or at least one instruction.

[0270] In one embodiment, the electronic device can detect a face and a body of a first person in a plurality of images, respectively, by having at least one processor execute a program stored in a memory or at least one instruction, and based on the detection result, group a plurality of body images of the first person extracted from the plurality of images, and obtain a plurality of pose information from the grouped plurality of body images.

[0271] In one embodiment, the electronic device can group a plurality of face images of a first person extracted from a plurality of images based on a detection result by having at least one processor execute a program stored in a memory or at least one instruction, and group the plurality of body images by matching the grouped plurality of face images with the plurality of body images.

[0272] In one embodiment, the pose of the first person may be a pose of the upper and lower body of the first person, excluding the face.

[0273] In one embodiment, the electronic device can generate a body image of a pose of a first person striking a first pose based on a first pose and a plurality of images by having at least one processor execute a program or at least one instruction stored in a memory, and generate a corrected image by synthesizing the body image onto a base image.

[0274] In one embodiment, the electronic device can obtain a first range of poses of a person modelable in a three-dimensional form based on a plurality of pose information by executing a program or at least one instruction stored in a memory, obtain a second range of poses of a person connectable with a face of the first person in a base image, and obtain a pose range that falls within both the first range and the second range.

[0275] In one embodiment, the electronic device can remove a body region of a first person in a base image and synthesize a body image on the removed region by having at least one processor execute a program or at least one instruction stored in a memory.

[0276] A non-transitory computer-readable recording medium having recorded thereon a program for performing any one of the methods according to one embodiment of the present disclosure on a computer may be provided.

[0277] Various embodiments of the present disclosure may be implemented or supported by one or more computer programs, and the computer programs may be formed from computer-readable program code and embodied in a computer-readable medium. In the present disclosure, "application" and "program" may refer to one or more computer programs, software components, instruction sets, procedures, functions, objects, classes, instances, associated data, or portions thereof suitable for implementation in computer-readable program code. "Computer-readable program code" may include various types of computer code, including source code, object code, and executable code. "Computer-readable medium" may include various types of media that can be accessed by a computer, such as read-only memory (ROM), random access memory (RAM), a hard disk drive (HDD), a compact disc (CD), a digital video disc (DVD), or various types of memory.

[0278] Additionally, a device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, a 'non-transitory storage medium' is a tangible device and may exclude wired, wireless, optical, or other communication links that transmit temporary electrical or other signals. Meanwhile, this 'non-transitory storage medium' does not distinguish between cases where data is permanently stored in the storage medium and cases where it is temporarily stored. For example, a 'non-transitory storage medium' may include a buffer where data is temporarily stored. A computer-readable medium may be any available medium that can be accessed by a computer, and may include both volatile and non-volatile media, and removable and non-removable media. A computer-readable medium includes a medium on which data can be permanently stored and a medium on which data can be stored and later overwritten, such as a rewritable optical disk or an erasable memory device.

[0279] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0280] The above description of the present disclosure is for illustrative purposes only, and those skilled in the art will appreciate that the present disclosure can be readily modified into other specific forms without altering the technical spirit or essential characteristics of the present disclosure. For example, suitable results can be achieved even if the described techniques are performed in a different order than the described method, and / or components of the systems, structures, devices, circuits, etc. described are combined or combined in a different form than the described method, or are replaced or substituted by other components or equivalents. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined form.

[0281] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. A step of obtaining multiple images each containing multiple characters; A step of determining one of the plurality of images as a base image; A step of obtaining multiple pose information regarding a pose of a first person among the multiple people from the multiple images; A step of obtaining a pose range for a pose of a person that can be implemented in a three-dimensional form by three-dimensionally modeling the pose of the person based on the above plurality of pose information; A step of determining a first pose within the above pose range; and A method comprising the step of generating a corrected image by synthesizing a body image of the first person taking the first pose onto the base image.

2. In paragraph 1, A method wherein the above multiple images are multiple images taken continuously over a set period of time.

3. In either of paragraphs 1 and 2, The step of obtaining the above multiple pose information is A step of obtaining key points regarding a body part of the first person in the plurality of images; and A method comprising a step of obtaining the plurality of pose information based on the key points.

4. In any one of the clauses 1 to 3, The step of obtaining the above multiple pose information is A step of detecting the face and body of the first person in the plurality of images, respectively; A step of grouping a plurality of body images of the first person extracted from the plurality of images based on the detection results; and A method comprising the step of obtaining the plurality of pose information from the plurality of grouped body images.

5. In paragraph 4, The step of grouping the above multiple body images is: A step of grouping multiple facial images of the first person extracted from the multiple images based on the detection results; and A method comprising the step of grouping the plurality of body images by matching the plurality of grouped face images with the plurality of body images.

6. In any one of paragraphs 1 to 5, The step of generating the above correction image is: A step of generating the body image regarding the pose of the first person taking the first pose based on the first pose and the plurality of images; and A method comprising the step of generating a corrected image by synthesizing the body image onto the base image.

7. In any one of paragraphs 1 to 6, The step of obtaining the above pose range is: A step of obtaining a first range of poses of a person that can be modeled in a three-dimensional form based on the plurality of pose information; A step of obtaining a second range of poses of a person that can be connected to the face of the first person in the base image; and A method comprising the step of obtaining the pose range belonging to both the first range and the second range.

8. In any one of paragraphs 1 to 7, The step of generating the above correction image is: A step of removing a body area of ​​the first person in the base image; and A method comprising the step of synthesizing the body image onto the removed area.

9. An input / output interface for receiving user input requesting image processing and outputting an image processed according to the user input; Memory where commands for processing images are stored; and Contains at least one processor, The electronic device, by causing at least one processor to execute a program or at least one instruction stored in the memory, Obtain multiple images, each containing multiple characters, Decide on one of the above multiple images as a base image, From the above plurality of images, obtain multiple pose information about the pose of a first person among the plurality of people, By modeling the pose of a person in three dimensions based on the above plurality of pose information, a pose range for a pose of a person that can be implemented in three dimensions is obtained, Determine the first pose within the above pose range, An electronic device that generates a corrected image by synthesizing a body image of the first person taking the first pose onto the base image.

10. In paragraph 9, An electronic device wherein the above multiple images are multiple images taken continuously over a set period of time.

11. In any one of the clauses 9 to 10, The electronic device, by causing at least one processor to execute a program or at least one instruction stored in the memory, Detecting the face and body of the first person in the plurality of images, respectively, Based on the above detection results, grouping the plurality of body images of the first person extracted from the plurality of images, An electronic device that obtains a plurality of pose information from a plurality of grouped body images.

12. In paragraph 11, The electronic device, by causing at least one processor to execute a program or at least one instruction stored in the memory, Based on the above detection results, grouping the plurality of facial images of the first person extracted from the plurality of images, An electronic device that groups a plurality of body images by matching the plurality of grouped facial images with the plurality of body images.

13. In any one of the clauses 9 to 12, The electronic device, by causing at least one processor to execute a program or at least one instruction stored in the memory, Based on the first pose and the plurality of images, generate the body image regarding the pose of the first person taking the first pose, An electronic device that generates a corrected image by synthesizing the body image onto the base image.

14. In any one of the clauses 9 to 13, The electronic device, by causing at least one processor to execute a program or at least one instruction stored in the memory, Obtain a first range of poses of a person that can be modeled in a three-dimensional form based on the above plurality of pose information, Obtain a second range of poses of a person that can be connected to the face of the first person in the base image, An electronic device that acquires the pose range that falls within both the first range and the second range.

15. A computer-readable recording medium having recorded thereon a program for performing the method of any one of clauses 1 to 8 on a computer.

Images