Avatar generation device, clothing synthesis device, and program
The avatar generation device addresses personal information protection and remote shopping challenges by creating anonymous avatars that resemble the user's face type, enabling accurate virtual clothing try-ons.
Patent Information
- Application Number
- JP2023165276
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-09-27
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2039-09-26
AI Technical Summary
Existing avatar generation methods that resemble a user's appearance can compromise personal information protection and hinder remote shopping experiences, as users cannot accurately try on clothing items using such avatars.
An avatar generation device that includes a face image acquisition unit, face type determination, and face deformation units to create an avatar image that resembles the user's face type while maintaining anonymity, combined with a clothing synthesis system to virtually try on clothing items.
The solution allows users to try on clothing items virtually without revealing personal information, ensuring accurate recommendations based on the user's face type and preferences, enhancing the remote shopping experience.
Smart Images

Figure 0007593456000008 
Figure 0007593456000009 
Figure 0007593456000010
Abstract
Description
[Technical field]
[0001] The present invention relates to an avatar generation device, a mobile terminal, a clothing matching system, and an avatar generation method. Regarding laws and programs. [Background technology]
[0002] An avatar is used to identify a user in online communications. There is a known method for generating a target from a captured image of the user (see Patent Documents 1 and 2). ). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] International Publication No. 2012 / 170354 [Patent Document 2] JP 2016-136324 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, avatars that resemble the user's appearance may allow the user to be identified. This posed a problem in terms of protecting personal information. is not a problem when used in online games and other applications, but In remote shopping, for example, the user purchases clothing based on a remotely generated avatar. There is a problem with the browser not being able to shop properly.
[0005] The present invention has been made in view of the above, and is a method for determining how well a clothing item suits an avatar and a user. Although it is no different from the user, it is difficult to identify the user from the avatar. Avatar generation device, mobile terminal, clothing matching system, and avatar generation method The purpose of this document is to provide laws and programs. [Means for solving the problem]
[0006] In order to achieve the above object, an avatar generation device according to one aspect of the present invention is a face image acquisition unit that extracts a face portion from a whole-body image obtained by capturing the face, and generates a face image; A face type determining method for determining whether a face image is one of a plurality of face types classified in advance A group determination unit and a variable unit for setting an average face image obtained by averaging a predetermined number of face images of people based on a predetermined condition. A shape policy setting unit weights the face image and the average face image at a predetermined ratio and synthesizes the face image. a face transformation unit that generates a transformed face image by combining the transformed face image with the whole-body captured image; a whole-body image synthesis unit that generates an avatar image that can be synthesized with a clothing synthesis image obtained by capturing an image of clothing accessories; Equipped with.
[0007] In order to achieve the above object, a mobile terminal according to one aspect of the present invention includes an avatar generating device and a The avatar image or the whole-body captured image is sent to a second user who is located at a different location from the first user. The clothing synthesis device synthesizes the clothing image based on the recommended clothing image.
[0008] In order to achieve the above object, a clothing matching system according to one aspect of the present invention is a mobile terminal a first mobile terminal which is operated by the first user; and a second mobile terminal which is operated by the second user. a second mobile terminal operated by a user and capable of communicating with the first mobile terminal; The mobile terminal displays the avatar image generated by the avatar generation device on the second mobile terminal. the clothing image received from the second mobile terminal is synthesized with the whole-body image. The second mobile terminal adds the clothing image to the avatar image received from the first mobile terminal. The clothing image is then combined and transmitted to the first mobile terminal.
[0009] In order to achieve the above object, a method for generating an avatar according to one aspect of the present invention comprises: A face image acquisition step of extracting a face portion from a whole-body captured image and generating a face image; , determining whether the face image is one of a plurality of face types classified in advance; A face type determination step and an average face image obtained by averaging a predetermined number of face images of a person based on a predetermined condition. a transformation policy setting step of setting a transformation policy for the face image and the average face image at a predetermined ratio; A face transformation step of generating a transformed face image by weighting and combining the transformed face image; The clothing accessories are then combined with a whole-body image to create an avatar image that can be combined with the clothing composite image. and generating a whole body image synthesis step.
[0010] In order to achieve the above object, a program according to one aspect of the present invention includes: A face image acquisition step of extracting a face portion from the whole-body captured image and generating a face image; A face image is judged to be one of a plurality of face types classified in advance. A type determination step and an average face image obtained by averaging a predetermined number of face images of people based on a predetermined condition are performed. and overlapping the face image and the average face image at a predetermined ratio. a face transformation step of generating a transformed face image by combining the transformed face image with the whole face image; The system synthesizes the self-portrait image and the clothing composite image to generate an avatar image that can be synthesized with the clothing composite image. A whole body image synthesis step, which is executed by a computer operating as an avatar generation device. Can. Effect of the Invention
[0011] According to the present invention, the degree to which clothing and accessories look good on an avatar is the same as that on a user. It is difficult to identify a user from their butter, so avatars can be generated. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a block diagram illustrating an example of a clothing matching system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of the mobile terminal according to the embodiment. [Diagram 3] FIG. 3 is an explanatory diagram illustrating an example of a sequence of the clothing matching system according to the embodiment. [Figure 4] FIG. 4 is a block diagram showing an example of the configuration of the avatar generation device according to the first embodiment. [Diagram 5] FIG. 5 is a flowchart showing an example of processing by the avatar generation device according to the first embodiment. [Figure 6] FIG. 6 is a block diagram showing an example of the configuration of an avatar generation device according to the second embodiment. [Figure 7] FIG. 7 is a flowchart showing an example of processing by the avatar generation device according to the second embodiment. [Figure 8] FIG. 8 is a block diagram showing an example of the configuration of an avatar generation device according to the third embodiment. [Figure 9] FIG. 9 is a flowchart showing an example of processing by the avatar generation device according to the third embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of an avatar generation device according to the fourth embodiment. [Figure 11] FIG. 11 is a flowchart showing an example of processing by the avatar generation device according to the fourth embodiment. [Figure 12] FIG. 12 is a flowchart showing an example of processing by the facial part transformation setting unit according to the fourth embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of processing by the facial part transformation section according to the fourth embodiment. [Figure 14] FIG. 14 is a flowchart showing another example of the processing of the facial parts transformation section according to the fourth embodiment. [Figure 15] FIG. 15 is a flowchart showing yet another example of the processing of the facial part transformation section according to the fourth embodiment. [Figure 16] FIG. 16 is a block diagram showing an example of the configuration of an avatar generation device according to the fifth embodiment. [Figure 17] FIG. 17 is a flowchart showing an example of processing by the avatar generation device according to the fifth embodiment. [Figure 18] FIG. 18 is a diagram showing an example of an operation screen for operating a facial part transformation adjustment unit according to the fifth embodiment. [Figure 19] FIG. 19 is a block diagram showing an example of the configuration of an avatar generation device according to the sixth embodiment. [Figure 20] FIG. 20 is a flowchart showing an example of processing by the avatar generation device according to the sixth embodiment. [Figure 21] FIG. 21 is a flowchart showing an example of the facial parts replacement process according to the sixth embodiment. [Figure 22] FIG. 22 is a flowchart showing another example of the facial parts replacement processing according to the sixth embodiment. [Figure 23] FIG. 23 is a block diagram showing an example of the configuration of an avatar generation device according to the seventh embodiment. [Figure 24] FIG. 24 is a flowchart showing an example of processing by the avatar generation device according to the seventh embodiment. [Diagram 25] FIG. 25 is a flowchart showing an example of the facial parts replacement process according to the seventh embodiment. [Figure 26] FIG. 26 is a block diagram showing an example of the configuration of an avatar generation device according to a modification of the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the embodiment, and when there are multiple embodiments, each embodiment The present invention also includes configurations in which the embodiments are combined. (Embodiment) First, referring to FIG. 1, a clothing matching system according to the first to seventh embodiments of the present invention will be described. A common configuration of the clothing matching system 1 according to the embodiment will be described. FIG. 1 is a block diagram showing an example of a clothing matching system 1. In this service, consumers who wish to purchase clothing items can receive recommendations for clothing items based on their desired conditions. The clothing matching system 1 is a system that can match the user x who is in location X. and a mobile terminal 100X used by a user y located at a different location Y from the location X. and a body terminal 100Y.
[0014] User x is a person who wants to purchase clothing. User x sends clothing recommendations to user y. User y is asked to choose clothing items that suit the client's appearance, hobbies, etc., better than a typical store clerk. A user y is a person who can recommend items efficiently. For example, the user y is not a shop clerk, but a shop assistant. They sell clothing through charismatic buyers who visit the stores or through flea market apps, etc., and receive some kind of incentive. User y is a person who wants to get the benefits of using User x’s personal information such as email address and SNS account. You will only have the minimum amount of account necessary to communicate with us.
[0015] The mobile terminal 100X and the mobile terminal 100Y have similar functions and are used for generating avatars. A smartphone equipped with a clothing matching application including an application The mobile terminal 100X and the mobile terminal 100Y are terminals connected to each other via the Internet. Communication is possible via a communication network such as a landline or a mobile phone line. For example, LTE (Long Term Evolution) is used. In the case where it is not necessary to distinguish between the mobile terminals 100X and 100Y, The body terminal 100 is also referred to as a body terminal 100.
[0016] Next, the mobile terminal 100 according to the first to seventh embodiments of the present invention will be described with reference to FIG. 2 is a diagram showing an example of a configuration of the mobile terminal 100 according to the embodiment. 1 is a block diagram showing a mobile terminal 100 including an image capture unit 101, a distance measurement unit 102, and An operation unit 103, a storage unit 104, a display unit 105, a communication unit 106, and a control device 110. , clothing synthesis device 120, and avatar generation device 10, 20, 30, 40, 50, 60, 70 And, it is equipped with.
[0017] The imaging unit 101 is a camera mounted on the mobile terminal 100 and captures the surroundings of the mobile terminal 100. The imaging unit 101 is a CMOS (Complementary Metal Oxide The imaging unit 101 includes an imaging element such as a CMOS image sensor. The captured image is captured in response to the operation of the operation unit 103. The captured image may be, for example, an RGB image, a Y The captured image is an image cut out from a video. The imaging unit 101 of the mobile terminal 100X may capture, for example, the entire body of the user x at least The image capturing unit 101 of the mobile terminal 100Y captures images from one or more directions including the front. Then, the clothing accessory selected by the user y is photographed from one or more directions including at least the front. The image capturing unit 101 outputs image information representing the captured image to the image capturing information acquiring unit 111 of the control device 110. do.
[0018] The distance measurement unit 102 measures the distance from the mobile terminal 100 to the object to be imaged by the imaging unit 101. The distance information measured by the distance measurement unit 102 includes the three-dimensional coordinates of the imaging target. The distance measurement unit 102 may be, for example, a laser autofocus system included in the camera of the imaging unit 101. The distance measurement unit 102 of the mobile terminal 100X may be, for example, For example, to measure the distance to the whole body of user x from one or more directions, including at least the front, The distance measurement unit 102 of the mobile terminal 100Y may measure the distance to the clothing accessory selected by the user y. The distance measurement unit 102 measures the distance from one or more directions including at least the front. The distance information is output to the distance information acquisition unit 112 of the control device 110 .
[0019] The operation unit 103 is an interface that accepts various operations for the mobile terminal 100. The operation unit 103 is, for example, a physical switch or a touch panel provided on the display unit 105. The operation unit 103 can be realized by, for example, a touch panel. The operation unit 103 can receive operations, for example, an image captured by the imaging unit 101. Avatar images or clothing images received by the communication unit 106, the avatar generation devices 10 and 2 Avatar creation screen by 0, 30, 40, 50, 60, 70, clothing synthesis device 120 The apparatus can accept an operation to display a screen for generating a composite clothing image on the display unit 105. The unit 103, for example, performs an operation to cause the communication unit 106 to transmit an avatar image or a clothing captured image. The operation unit 103 can receive the avatar generation devices 10, 20, 30, 40, and 5. You can input various setting information to generate avatars at 0, 60, and 70. The operation unit 103 is capable of inputting various operations for causing the clothing synthesis device 120 to generate a clothing synthesis image. The operation unit 103 can receive an operation for inputting setting information. The operation signal is output to the operation information acquisition unit 113 .
[0020] The storage unit 104 stores various information. ), RAM (Random Access Memory), HDD (Hard Disk Drive), Flash memory The storage unit 104 includes a storage medium such as a non-volatile or volatile semiconductor memory. Various control programs executed by the control device 110, data used in various control processes, The storage unit 104 stores various types of data acquired by the control device 110. For example, the whole-body image captured by the imaging unit 101 and the whole-body distance information measured by the distance measurement unit 102 are The storage unit 104 stores, for example, the information of the avatar generation device input from the operation unit 103. Various setting information for generating avatars at 10, 20, 30, 40, 50, 60, and 70 The storage unit 104 stores, for example, the avatar generation devices 10, 20, 30, 40, and 50. , 60, 70 store the generated avatar images and avatar distance information. For example, communication unit 106 stores the avatar image and avatar distance information received by communication unit 106. The storage unit 104 stores, for example, the clothing image captured by the image capture unit 101 and the distance measurement unit 102. The storage unit 104 stores the clothing distance information measured by the clothing synthesis device 120. The storage unit 104 stores the clothing synthesis avatar image and the clothing synthesis avatar distance information. For example, the clothing image and clothing distance information received by the communication unit 106 are stored in the storage unit 104. The unit 104 receives, for example, the clothing synthesis image and the clothing synthesis distance information generated by the clothing synthesis device 120. The storage unit 104 stores the control information output from the storage control unit 114 of the control device 110. Based on the signal, various information is stored.
[0021] The display unit 105 displays various images, videos, text, and other information. Includes display panels using liquid crystal elements or organic EL (Organic Electro-Luminescence) elements. When the display unit 105 includes a touch panel, the display unit 105 displays the user x on the display panel. The display unit 105 has a touch sensor for detecting the position of a pointer such as a finger. For example, the display unit 105 displays an image captured by the imaging unit 101. The display unit 105 displays the received avatar image or clothing image. Avatar generation screen by avatar generation device 10, 20, 30, 40, 50, 60, 70 The display unit 105 displays, for example, a screen on which a composite clothing image is generated by the clothing synthesis device 120. The display unit 105 displays the display information output from the display control unit 115 of the control device 110. Displays various information based on the information.
[0022] The communication unit 106 is a communication unit. The mobile terminal 100Y communicates with the communication unit 106 via a communication network. The communication unit 106 of the terminal 100X may be, for example, an avatar generation device 10, 20, 30, 40, 50 , 60, 70 transmit the generated avatar image and avatar distance information to mobile terminal 100Y. The communication unit 106 of the mobile terminal 100Y transmits, for example, an avatar image and an avatar distance. The communication unit 106 of the mobile terminal 100Y receives the remote information from the mobile terminal 100X. For example, the clothing image captured by the imaging unit 101 and the clothing distance measured by the distance measurement unit 102 are The distance information is transmitted to the mobile terminal 100X. The communication unit 106 of the mobile terminal 100X may For example, the clothing captured image and the clothing distance information are received from the mobile terminal 100Y. Based on the control signal output from the communication control unit 116 of the control device 110, various communications are performed. Execute.
[0023] The control device 110 is, for example, an operating system configured with a CPU (Central Processing Unit) and the like. The control device 110 is a computational processing device (control device). The control device 110 loads the program into memory and executes the instructions contained in the program. The internal memory includes an internal memory (not shown). In this case, the control device 110 stores a program for performing the above-mentioned operations in an internal memory or the like. The functions of each part are realized by extracting and executing the programs stored in the The memory is also used for temporary storage of data in the control device 110. An imaging information acquisition unit 111 and a distance information acquisition unit 112 are connected by a bus 119. , an operation information acquisition unit 113, a storage control unit 114, a display control unit 115, and a communication control unit 11 6, an information input unit 117, and an information acquisition unit 118.
[0024] The imaging information acquisition unit 111 acquires image information representing various captured images captured by the imaging unit 101. The imaging information acquisition unit 111 acquires, for example, an image of the entire body of the user x captured by the imaging unit 101. The imaging information acquisition unit 111 acquires an image of an accessory, for example, when the imaging unit 101 captures an image of the accessory. The imaging information acquisition unit 111 acquires the captured image. The data is output to the storage control unit 114.
[0025] The distance information acquisition unit 112 indicates the distances to various imaging targets measured by the distance measurement unit 102. The distance information acquisition unit 112 acquires distance information. The distance information acquisition unit 112 acquires distance information from the front of the clothing captured by the image capture unit 101. The distance information acquisition unit 112 acquires distance information from the front of the ornament. The acquired distance information is output to the memory control unit 114.
[0026] The operation information acquisition unit 113 acquires information about operations received by the operation unit 103 from users x, y, etc. The operation information acquisition unit 113 acquires an operation signal. For example, the operation information acquisition unit 113 acquires an image capturing operation by the image capturing unit 101. a display operation signal indicating a display operation of various display information; The transmission operation signal indicating the operation of the device, the input operation signal indicating the input operation of various setting information, etc. are acquired. The information acquisition unit 113 transmits a control signal corresponding to the received operation signal to the imaging information acquisition unit 111, to the distance information acquisition unit 112, the display control unit 115, the communication control unit 116, or the information input unit 117 In this case, the imaging information acquisition unit 111, the distance information acquisition unit 112, the display control unit 115 The communication control unit 116 and the information input unit 117 execute operations according to the control signal.
[0027] The storage control unit 114 stores various information acquired by the control device 110 in the storage unit 104. The various information includes a whole-body image, whole-body distance information, and various settings for generating an avatar. information, an avatar image and avatar distance information, a clothing image and clothing distance information, A composite avatar image and clothing composite avatar distance information, a clothing captured image and clothing distance information, These include clothing synthesis images and clothing synthesis distance information.
[0028] The display control unit 115 controls the display unit 105 to display various images, videos, text, and other information. The display control unit 115 outputs a display information signal to the display unit 105. The display control unit 115 displays various information on the image captured by the image capturing unit 101. , avatar image or clothing captured image, avatar generation screen or clothing composite image generation screen The display information signal is outputted.
[0029] The communication control unit 116 controls the communication in the communication unit 106. The communication control unit 116 acquires the communication information output from the memory control unit 114. The communication control unit 116 outputs the various pieces of information received by the communication unit 106. The communication control unit outputs, for example, the avatar image and the avatar The distance information, the clothing captured image, and the clothing distance information are obtained from the storage control unit 114. The control unit may, for example, The information is output to the communication unit 106.
[0030] The information input unit 117 receives various setting information input from the operation unit 103 and the storage unit 104. The various setting information stored in the avatar generation devices 10, 20, 30, 40, 50, and 60 , 70, or the clothing synthesis device 120. It is recommended that you wear a full-body image, full-body distance information, age, gender, and the clothing you wish to purchase. Recommended seasons, locations, or facial expressions, preferences, favorite faces, average face images, celebrities A face image, a replacement avatar face part image, an avatar face image, a clothing image, and a clothing distance The information input unit 117 of the mobile terminal 100X receives the following information for generating an avatar: Various pieces of information for the purpose of the creation of the avatar are input to the avatar generation devices 10, 20, 30, 40, 50, 60, and 70. The information input unit 117 of the mobile terminal 100Y receives the following information for generating a clothing composite avatar image: Various information is input to the clothing synthesis device 120. The information input unit 117 of the mobile terminal 100X Various information required to generate a composite clothing image is input to the clothing synthesis device 120.
[0031] The information acquisition unit 118 is connected to the avatar generation devices 10, 20, 30, 40, 50, 60, 70, Alternatively, various pieces of generation information are obtained from the clothing synthesis device 120. The acquisition unit 118 acquires the avatars generated by the avatar generation devices 10, 20, 30, 40, 50, 60, and 70. Or, information about the avatar being generated is acquired. 8 is a clothing synthesis avatar image and a clothing synthesis avatar distance generated by the clothing synthesis device 120. The information acquisition unit 118 of the mobile terminal 100X acquires information generated by the clothing synthesis device 120. The information acquisition unit 118 acquires the clothing composite image and clothing composite distance information acquired. The seed generation information is output to the storage control unit 114 .
[0032] The clothing synthesis device 120 performs the art based on various setting information acquired from the information input unit 117. A composite image is generated by combining the butter image or the whole-body image with the clothing image. The clothing synthesis device 120 of the mobile terminal 100Y is connected to the avatar generation devices 10, 20, Avatar images generated by the devices 30, 40, 50, 60, and 70 and the imaging unit of the mobile terminal 100Y The clothing image captured by the mobile terminal 101 is combined with the clothing image to generate a clothing-combined avatar image. The clothing synthesis device 120 of the mobile terminal 100X uses the image of the user x captured by the image capture unit 101 of the mobile terminal 100X. The whole-body image of the mobile terminal 100Y is synthesized with a clothing image captured by the imaging unit 101 of the mobile terminal 100Y. A clothing composite image is generated.
[0033] The avatar generation devices 10, 20, 30, 40, 50, 60, and 70 receive information from the information input unit 117. The avatar is generated based on various setting information acquired from the user. The configurations of the devices 10, 20, 30, 40, 50, 60, and 70 will be described in detail later. do.
[0034] The sequence of the clothing matching system 1 according to the embodiment will be described with reference to FIG. FIG. 3 is an explanatory diagram showing an example of a sequence of the clothing matching system 1 according to the embodiment. Here, the storage unit 104 of the mobile terminal 100X stores each of the avatar generating The type setting information is stored in advance. The clothing matching system 1 is a mobile terminal 100X accepts a predetermined start operation on the operation unit 103 by the user x. Then, the process proceeds to step ST101 shown in FIG. 3 to start the process.
[0035] The avatar generation devices 10, 20, 30, 40, 50, 60, and 70 of the mobile terminal 100X Then, various setting information for generating an avatar is obtained from the information input unit 117 (step ST101). The mobile terminal 100X includes the avatar generation devices 10, 20, 30, 40, 50 , 60, 70 generate avatars based on various setting information (step ST102 The generation method of the avatar generation devices 10, 20, 30, 40, 50, 60, and 70 of the respective embodiments The method will be described in detail later. A user x who operates a mobile terminal 100X Avatar data Da including the avatar image and avatar distance information of the created avatar is The mobile terminal 100Y transmits the message (step ST103).
[0036] After the mobile terminal 100Y receives the avatar data Da from the mobile terminal 100X, the user The user y photographs the clothing accessories to be worn by the avatar using the imaging unit 101 of the mobile terminal 100Y. The mobile terminal 100Y acquires the clothing image and the clothing distance information (step ST 104). The mobile terminal 100Y transmits the avatar distance information and the outfit to the outfit synthesis device 120. The avatar image and the clothing image are synthesized based on the distance information to generate a clothing-synthesized avatar image. The mobile terminal 100Y receives the clothing composite avatar image and the clothing Based on the distance information, the clothing synthesis avatar image is modeled in 3D and displayed. The user y displays the image on the display unit 105 in a three-dimensional manner (step ST106). The user checks the avatar wearing the displayed clothing item and judges whether the clothing item suits the user. If user y judges that the clothing looks good, the user y sends a notification including the judgment result, the clothing image, and the clothing distance information. The clothing data Dd including the clothing data Dd is transmitted to the mobile terminal 100X (step ST107). If the user y judges that the hairstyle does not suit him / her, the process goes from step ST104 to step ST10 Repeat steps 1 to 6.
[0037] After the mobile terminal 100X receives the clothing data Dd from the mobile terminal 100Y, the user x The clothing synthesis device 120 of the mobile terminal 100X receives the clothing image based on the whole body distance information and the clothing distance information. Based on the result, the whole body image and the clothing image are synthesized to generate a clothing synthesized image (step ST 108). The mobile terminal 100X determines the clothing based on the clothing synthesis image and the clothing synthesis distance information. The composite image is modeled three-dimensionally and displayed stereoscopically on the display unit 105 (step ST 109). As a result, user x can see an image of himself wearing the clothing item recommended by user y. It can be seen in the image. First embodiment The avatar generation device 10 according to the first embodiment will be described with reference to FIGS. 4 and 5. The avatar generating device 10 according to the first embodiment uses a captured image of the user x himself / herself and a previously captured image of the user x. An avatar is generated using an average face image obtained by averaging a predetermined number of face images stored in the storage unit 104. FIG. 4 is a block diagram showing an example of the configuration of the avatar generation device 10 according to the first embodiment. The avatar generating device 10 includes a face image acquiring unit 11, a face type determining unit 12, and , a transformation policy setting unit 13, a face transformation unit 14, and a whole-body image synthesis unit 15.
[0038] The face image acquisition unit 11 stores the face image in the memory unit 104 via the memory control unit 114 of the control device 110. The storage unit 1 acquires setting information including a whole-body captured image of the user x and whole-body distance information. The setting information obtained from 04 further includes at least the age and gender information of user x. The setting information acquired from the storage unit 104 includes a setting that recommends wearing the clothing item to be purchased. The information may include the season, location, facial expression, and other circumstances in which the image is captured. The whole-body captured image and whole-body distance information acquired from 104 are captured from the front and and distance information from the front of the whole body of user x. The facial image and facial distance information of user x are extracted from the facial image and the whole body distance information. The face image acquisition unit 11 generates the face image and the face distance information based on the setting information. The result is output to the determination unit 12.
[0039] The face type determination unit 12 receives the setting information, the face image, and the face distance information from the face image acquisition unit 11. The face type determination unit 12 determines the face type of the user x based on the face image and face distance information. The face type determination unit 12 determines the face type Ti by using a learning image data set. It has a trained inference machine.
[0040] The face type Ti is a combination of the face and the facial parts that indicate each part of the face based on a predetermined judgment criterion. The face type is determined by evaluating the size, position and shape of each. Table 1 shows the face type T As shown in Table 1, face type Ti is divided into face types T1, T2, T3, They are classified into eight types: T1, T2, T3, T4, T5, T6, T7, and T8 (Japan Face Type Diagnosis Association). The balance item for the child face type is, for example, a round or wide face, and the facial features are in proportion to the face. The adult face type in the balance category is, for example, This type has features such as a long face and facial features that are positioned upwards relative to the face. The curved type has no bones in the face, round eyes, no nose bridge, and high cheekbones. The straight line type has the following characteristics: This type has features such as a straight nose and low cheekbones. The facial features are the eyes and nose. At least
[0041] [Table 1]
[0042] Table 2 shows the first step in determining whether a face is a child face type or an adult face type. This is a list of judgment items and judgment contents corresponding to each judgment number Ni in the judgment criteria. The parts are assigned for each judgment item of judgment numbers N1, N2, N3, N4, N5, N6, and N7. The product is judged as either Grade A or Grade B based on a certain standard. Here, the predetermined standard is the standard for each judgment item. For example, the predetermined judgment of the judgment item of judgment number N2 is The criterion is whether the chin length is below normal.
[0043] [Table 2]
[0044] Table 3 shows the results of the second criterion for face and facial feature shape in face type determination. This is a list of the judgment items and judgment contents corresponding to each judgment number Ni. Regarding the assessment items for N8, N9, N10, N11, N12, N13, N14, and N15 Based on the specified criteria, the product will be rated as either C or D.
[0045] [Table 3]
[0046] Table 4 is a table explaining the classification method of face type Ti. As shown in Table 4, face type T i is the number of A-ratings for the grading items corresponding to each grading number Ni in the first grading criteria shown in Table 2 and the number of C judgments for the judgment items corresponding to each judgment number Ni in the second judgment criteria shown in Table 3. They are classified according to the size of their eyes.
[0047] [Table 4]
[0048] The face type determination unit 12 determines the face type of each of the determination items N1 to N7 for the acquired face image. The face type determination unit 12 determines whether the acquired face image is an A type or a B type. Each of the judgment items from N8 to N15 is judged as C or D. 2 shows the frontal images of each judgment item of judgment numbers N1, N2, N3, N6, and N7. The face type determination unit 12 performs the determination using the front image. The face type determination unit 12 performs the determination using a side image captured from the side. The face type determination unit 12 determines the predetermined items using an image captured from directly above. The judgment for each judgment item from judgment number N1 to N15 is based on the judgment criteria for each judgment item in advance. The method uses a machine-learned inference machine to make a binary choice decision based on the image data for learning. For example, we use training data images with the correct answers labeled for each judgment item as a dataset. For each face type Ti, about 100 images are prepared. The learning method of the inference machine is, for example, Deepler. It is not easy to quantify features like the judgment number N1. Deep learning is a method that can identify objects by providing training data images. It is preferable to use machine learning such as the above. The present invention is not limited to the embodiment, and for example, A threshold is set for each judgment item to select between two options, and the judgment criteria is whether the value is above the threshold or not. It may be judged as follows.
[0049] The face type determination unit 12 counts the number of each of the determination results A, B, C, and D. Based on the number of A and C judgements and the size of the eyes, a face type Ti is determined according to Table 4. For example, for the face image of user x, if the number of A judgments is 6 and the number of C judgments is 4, In this case, the user x is a face type T3. The face type Ti and the judgment results for each judgment item used to judge the face type Ti are used as a transformation policy. The face type determination unit 12 outputs the determination numbers N1 to N15 to the face type setting unit 13. The judgment of each judgment item is made by using an inference machine that makes a binary choice judgment based on the judgment criteria for each judgment item in advance. However, the present invention is not limited to this. For example, the correct face type Ti is labeled. We prepared about 100 images of the scanned image data for each face type Ti to create a learning data set. Using this learning dataset, we train an inference machine to generate face tags directly from face images. In this case, the judgment of each judgment item may be made based on the learning data set. This can be done at the time of generation.
[0050] The transformation policy setting unit 13 receives the setting information, the face type Ti, and The determination result for each determination item used to determine the face type Ti is obtained. 3 shows the face type Ti, the judgment results for each judgment item used to judge the face type Ti, and the setting information. The transformation policy is set based on the information stored in the storage unit 104. Among the average facial images of the groups, the average facial image of one of the groups is used for generating the avatar. Each group is classified based on age, gender, and face type Ti, etc. For example, if user x is a woman in her twenties and has face type T3, the transformation policy The sea setting unit 13 sets the transformation policy as the average face of the group of women in their twenties with face type T3. The transformation policy setting unit 13 sets the image to be used. The shape policy is output to face transformation unit 14.
[0051] The face transformation unit 14 acquires the transformation policy and setting information from the transformation policy setting unit 13. The face transformation unit 14 performs the transformation via the memory control unit 114 of the control device 110 in accordance with the transformation policy. The average face image of the desired group is acquired by storing the average face image in the storage unit 104. The average face image data for each group is classified based on age, gender, and face type Ti. The database is stored in the database. The database can be periodically updated via a communication network. The average face image for each group is a face image obtained by averaging a predetermined number of face images. The policy and classification of each group are based on race, area of residence, season when the image was taken, and the The classification may include classification according to the location where the image was taken, or the situation such as facial expression. The average face distance information, including the distance to the target and the three-dimensional coordinates of the target, is linked to the The face transformation unit 14 transforms the face image and face distance information of the user x based on the transformation policy. The face transformation unit 14 transforms the transformation polygon to generate a transformed face image and transformed face distance information. Based on the policy, generate an average face image of the group to which user x belongs, including the face image of user x. More specifically, the face transformation unit 14 transforms the face image of the user x and the average face image The face images of user x are weighted and synthesized in a predetermined ratio to generate a deformed face image. The predetermined ratio of the average face image is, for example, 1:10, but is not limited to this, and may be, for example, 0:1. If the weight of the average face image is increased, it becomes difficult to identify the user x. If the weight of the image is reduced, it becomes easier to identify the user x. For example, If the number of transactions of user x and user y is small, the weight of the average face image is increased. The weight of the average face image may be decreased as the number of times increases. In this way, the degree to which it is easy to identify user x is adjusted according to the reliability of user x and user y. The face transformation unit 14 can transform the face of the user x into a facial image including the facial distance information of the user x based on the transformation policy. The average face distance information of the group to which the user x belongs is generated and used as the deformed face distance information. Specifically, the face transformation unit 14 weights the face distance information of the user x and the average face distance information at a predetermined ratio. The face transformation unit 14 generates transformed face distance information by combining the average face image Adjust the size of the average face image so that the size of the average face image matches the size of the face in the face image (hereinafter, "scaling"). The face transformation unit 14 performs the process of transforming the face image based on the setting information, the generated transformed face image, and the transformed face distance. The information is output to the whole body image synthesis unit 15. The average face image is adjusted so as to match the size of the face in the face image. The size of the face in the face image may be adjusted, or the size of the face in the face image may be adjusted to match the size of the face in the average face image. The size may be adjusted.
[0052] The whole-body image synthesis unit 15 receives the setting information, the transformed face image, and the transformed face distance information from the face transformation unit 14. The whole-body image synthesis unit 15 synthesizes the deformed face image with the whole-body captured image to generate a deformed whole-body image. The whole-body image synthesis unit 15 synthesizes the transformed face distance information into the whole-body distance information. The whole-body image synthesis unit 15 synthesizes the distance information of the avatar, which is the transformed whole-body distance information. The setting information, the avatar image, and the avatar distance information are transmitted to the information acquisition unit 11 of the control device 110. Output to 8.
[0053] FIG. 5 is a flowchart showing an example of the process of the avatar generation device 10 according to the first embodiment. The avatar generating device 10 receives the operation information via the operation information acquisition unit 113 of the control device 110. When the unit 103 receives a predetermined start operation, the process proceeds to step ST11 shown in FIG. to start processing.
[0054] The avatar generating device 10 acquires a face image from a captured image (step ST11). Specifically, the avatar generating device 10 first receives a facial image captured by the imaging unit 101 from the face image acquisition unit 11. The avatar generation device 10 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 11 acquires face parts from the whole body image and the whole body distance information. The avatar generation device 10 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST12.
[0055] The avatar generating device 10 determines the face type Ti from the face image (step ST12). More specifically, the avatar generating device 10 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generation device 10 then causes the face type determination unit 1 2 determines the face type Ti based on the determination results for each determination item. 10 proceeds to step ST13.
[0056] The avatar generating device 10 sets a transformation policy (step ST13). In the transformation policy setting unit 13, the avatar generating device 10 sets face type Ti and face type T Based on the judgment results for each judgment item used in judging i and the setting information, a transformation policy is set. The transformation policy is set based on a predetermined group divided by age, sex, face type Ti, etc. The average face image of each group is used to generate an avatar. This is policy information. The avatar generation device 10 proceeds to step ST14.
[0057] The avatar generating device 10 averages a plurality of face images including the face image of the user x to generate an average face image. As a result, the avatar generating device generates a deformed face image (step ST14). First, the facial transformation unit 14 is notified of a group to be used for generating an avatar based on the transformation policy. The avatar generating device 10 then performs face transformation. An average face image of a group to which user x belongs is generated in a form unit 14, the average face image including the face image of user x; The avatar generating device 10 also controls the face transforming unit 14 to transform the face distance of the user x into a transformed face image. The average face distance information of the group to which user x belongs, including the distance information, is generated and used as the deformed face distance information. Avatar generation device 10 proceeds to step ST15.
[0058] The avatar generating device 10 generates an avatar image by synthesizing a captured image with a deformed face image ( Step ST15). More specifically, the avatar generating device 10 controls the whole-body image synthesis unit 15 to The deformed face image is then combined with a whole-body captured image to generate an avatar image, which is a deformed whole-body image. The avatar generating device 10 also transmits the deformed face distance information to the whole-body image synthesis unit 15 as whole-body distance information. The avatar generating device 1 generates avatar distance information, which is transformed whole-body distance information, by synthesizing the avatar distance information with the avatar image. 0 ends the processing of the flowchart shown in FIG.
[0059] As described above, the avatar generation device 10 of the first embodiment generates a face image for the face type Ti of the user x. In order to prevent the face type of the avatar image from changing, the face and the facial parts that indicate each part of the face are The size, position and shape of each can be changed. This allows you to create a It is possible to generate an avatar that is difficult to identify as user x even though it has the same face type. If they are one, it can be assumed that the clothing, hairstyle, accessories, and other accessories that suit them are the same.
[0060] Clothing matching using avatar images generated by the avatar generation device 10 of the first embodiment The shopping system 1 is a system for allowing a user x to virtually try on clothing items during remote shopping, etc. Then, the clothing item is tried on by an avatar having an average face image of the same face type as user x. This allows user x to know the identity of user y without allowing user y to identify the personal information of user x. It is possible to get recommendations on clothing that suit user x. In addition, personal information of user x, especially face The image is displayed in a way that suits user x but that user x himself cannot choose, without user y knowing more than necessary. User y can recommend unknown clothing items that he or she may not have seen before.
[0061] In addition, when collecting face images of the same face type as user x, Acquiring information such as the season in which the image was taken, the place where the image was taken, or the circumstances of the image, such as facial expressions Depending on the season, place or situation in which the clothing item you want to buy is recommended to be worn, It is possible to get highly accurate recommendations of clothing items. In addition, depending on the facial expression of the face image of user x, In this way, it is possible to get recommendations on clothing items that would suit user x.
[0062] On the other hand, user x uploads the clothing image of the clothing item recommended by user y to a full-body image of user x. Allowing a person to try on clothing items on an avatar that can be identified as the person by synthesizing the clothing items with a captured image In other words, user x can see an image of himself wearing the clothing item recommended by user y. Of course, user x can check the clothing recommended by user y on his / her avatar. You can also see images of the characters wearing the items. (Modification of the first embodiment) In the first embodiment, the entire face of the face image is transformed into an average face. Alternatively, only certain facial features of the average face image may be used, and only those certain facial features may be deformed. Here, it is assumed that the facial parts to be transformed are included in the transformation policy. As shown in Table 5, the facial parts to be transformed are the eyes, , mouth, nose, face shape, and ears. The priority is determined by the degree of impression when the facial features are changed. The smaller the number, the higher the priority. For example, from among the candidate facial features, a face that transforms the eyes and mouth, which are the top two facial features with the highest priority, is selected. You can change the facial features by changing the shape of the face. You can change the facial features by changing the shape of the face. It can be a combination of three or more parts. There may be one or more facial features, and all features may be combined.
[0063] [Table 5]
[0064] The face transformation unit 14 obtains an average face image based on the transformation policy, and transforms the face of the face image of the user x into an average face image. The facial features are replaced with the facial features of the average face image to generate a deformed face image. 4 is scaled so that the size of the average face image matches the size of the face in the face image, and then the face Replace parts.
[0065] As described above, the avatar generation device 10 according to the modification of the first embodiment can change the facial features. Therefore, it is possible to generate an avatar that retains some of the characteristics of user x. In the modified example, the facial parts with the highest priority are automatically selected, but We may select which facial features to transform based on the characteristics of user x. The priority of the parts may be dynamically changed and set, and the priority is not limited to a predetermined value. For example, The priority of the facial parts is determined based on the facial parts of user x that have a low correlation with the facial parts of the average face image. Here, the degree of correlation between the facial features of user x and It is calculated by the sum of absolute average differences of the pixel values of the facial parts of the average face image. When there are only a few facial features to be transformed, the facial features with the largest difference are transformed first. However, it is possible to generate an avatar that resembles user x but is difficult to identify as user x. Cut. Second Embodiment The avatar generation device 20 according to the second embodiment will be described with reference to FIGS. 6 and 7. The avatar generation device 20 according to the second embodiment generates a predetermined number of people who have similar tastes to the user x. The avatar is generated using the average face image obtained by averaging the face images of the same person. Alternatively, an average face image obtained by averaging a plurality of images may be used. 2 is a block diagram showing an example of the configuration of an avatar generation device 20. The avatar generation device 20 An image acquisition unit 21, a face type determination unit 22, a similar preference face image collection unit 23, and a transformation policy setting unit The facial image acquisition unit 21 includes a face determination unit 24, a face transformation unit 25, and a whole-body image synthesis unit 26. The configuration and operation of the whole-body image synthesis unit 26 are the same as those of the whole-body image synthesis unit 15 in the first embodiment. The configuration and operation of the face type determination unit 22 are the same as those of the face type determination unit 21. The operation is to send the judgment result, etc. to the similar preference face image collecting unit 23 instead of the transformation policy setting unit 13. The configuration and operation of the face type determination unit 12 in the first embodiment are the same as those of the face type determination unit 12 in the first embodiment, except that the face type determination unit 12 outputs the face type. Therefore, the description will be omitted.
[0066] The similar preference face image collecting unit 23 receives the setting information, the face image and the face type from the face type determining unit 22. The distance information, the face type Ti, and the judgment result for each judgment item used to judge the face type Ti are obtained. The similar preference face image collecting unit 23 collects the stored face images via the storage control unit 114 of the control device 110. The user x's preference information stored in the unit 104 is acquired. The preference information includes information on clothing and interior design. The preference information includes the preferences of users, purchase history, browsing history, posting history, etc. The setting information may be included in advance in the setting information acquired by the unit 21, or may be included in the operation information acquired by the control device 110. The user x may input the information from the operation unit 103 via the unit 113, or the shopping site It may be obtained from the database, browsing history of the web, SNS, etc. The collection unit 23 extracts, for example, a purchase history for a predetermined period from the acquired preference information of the user x. The similar preference face image collecting unit 23 collects face images of a predetermined number of people who have similar preferences to the user x. Similar facial images are collected. The similar facial images are obtained by taking into account the distance to the image capture target and the facial image of the image capture target. Similar preference face distance information including 3D coordinates is linked. People with similar preferences For example, the product may have a purchase history that matches or is similar to the purchase history of user x by a certain number or more. In other words, people who have a high correlation with the purchase history of user x are considered to have similar tastes. Facial images are collected from, for example, shopping site databases, WEB, The purchase history for a given period and the number of matches of the purchase history are calculated by the shopping site. The facial image of similar taste can be acquired from browsing history of websites, websites, SNS, etc. 3 is a set of setting information, face image and face distance information, and collected similar preference face images and similar preference The face distance information is output to the deformation policy setting unit 24.
[0067] The transformation policy setting unit 24 receives the setting information, the face image, and the like from the similar preference face image collecting unit 23. The facial distance information and the similar preference face image and the similar preference face distance information are acquired. The policy setting unit 24 sets a transformation policy based on the face image of the user x and the collected similar preferences. The transformation policy setting unit 24 sets the average image of the face images to be used for generating an avatar. , setting information, face image and face distance information, similar preference face image and similar preference face distance information, , and the set transformation policy are output to face transformation unit 25.
[0068] The face transformation unit 25 receives the setting information, the face image, and the face distance information from the transformation policy setting unit 24. The face transformation unit acquires the similar-preference face image, the similar-preference face distance information, and the transformation policy. 25 generates a transformed face image and transformed face distance information based on the transformation policy. The unit 25 converts the face image of the user x and the collected multiple similar preference faces based on the transformation policy. The image is averaged to generate an average face image, which is used as a deformed face image. Based on this, the face distance information of user x and the collected multiple similar preference face distance information are averaged to The face transformation unit 25 generates average face distance information and sets it as transformed face distance information. The deformed face image and the deformed face distance information are output to the whole body image synthesis unit 26 .
[0069] FIG. 7 is a flowchart showing an example of the process of the avatar generation device 20 according to the second embodiment. The avatar generating device 20 receives the operation information via the operation information acquisition unit 113 of the control device 110. When the unit 103 receives a predetermined start operation, the process proceeds to step ST21 shown in FIG. to start processing.
[0070] The avatar generating device 20 acquires a face image from a captured image (step ST21). Specifically, the avatar generating device 20 first inputs an image captured by the imaging unit 101 to the face image acquisition unit 21. The avatar generation device 20 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 21 acquires the face part from the whole body image and the whole body distance information. The avatar generation device 20 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST22.
[0071] The avatar generating device 20 determines the face type Ti from the face image (step ST22). More specifically, the avatar generating device 20 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generation device 20 then causes the face type determination unit 2 2 determines the face type Ti based on the determination results for each determination item. 20 proceeds to step ST23.
[0072] The avatar generation device 20 detects similar preferences based on purchase history, browsing history, posting history, etc. Similar preference face images, which are face images of people who resemble each other, are collected (step ST21). First, the avatar generating device 20 collects the similar-preference face image of the user x from the similar-preference face image collecting unit 23. The preference information may include purchase history, browsing history, or investment history of clothing and interior goods. The avatar generating device 20 then collects similar-preference face images from the similar-preference face image collecting unit 23. From the acquired preference information, for example, a purchase history for a predetermined period is extracted. Next, the similar preference face image collecting unit 23 collects face images of people who have similar preferences to the user x. A person with similar tastes is a person with similar tastes. For example, For example, a person whose purchase history matches a certain number of purchase histories of user x. The apparatus 20 proceeds to step ST24.
[0073] The avatar generation device 20 sets a transformation policy (step ST24). In this case, the avatar generating device 20 inputs the following transformation policy to the transformation policy setting unit 24: The face image of user x and the average image of the collected similar face images are used to generate an avatar. Avatar generation device 20 then proceeds to step ST25.
[0074] The avatar generating device 20 averages the face image and a plurality of similar preference face images to generate an average face image. The avatar generating device 2 generates a deformed face image (step ST25). 0, the face transformation unit 25 averages the face image and the collected multiple similar preference face images to generate a user Generate an average face image of x and multiple people who have similar tastes to user x, and use it as a deformed face image. In addition, the avatar generating device 20 transmits the face distance information and the collected multiple categories to the face transforming unit 25. The similar-preference face distance information is averaged to obtain the average face distance between user x and multiple people who have similar preferences to user x. The avatar generating device 20 generates face distance information and converts it into transformed face distance information. Transition to 26.
[0075] The avatar generation device 20 generates an avatar image by combining a captured image with a deformed face image ( Step ST26). More specifically, the avatar generating device 20 first includes a whole-body image synthesizing unit 26, acquires the whole-body image of the user x captured by the imaging unit 101 and whole-body distance information. The avatar generating device 20 then inputs the whole-body image synthesis unit 26 to synthesize the deformed face image with the whole-body captured image. The avatar generating device 20 generates an avatar image, which is a transformed whole-body image, by synthesizing the avatar image with the avatar image. The whole-body image synthesis unit 26 synthesizes the deformed face distance information with the whole-body distance information to generate the deformed whole-body distance information. The avatar generation device 20 generates avatar distance information according to the flowchart shown in FIG. This ends the port processing.
[0076] As described above, the avatar generation device 20 of the second embodiment The average face image of a certain number of people who have similar purchasing histories of clothing accessories or interior goods is Generally, people purchase clothing that suits them, so The average face generated from multiple people with a purchasing history that tends to It is conceivable that it would suit
[0077] Clothing matching using avatar images generated by the avatar generation device 20 of the second embodiment The shopping system 1 is a system for allowing a user x to virtually try on clothing items during remote shopping, etc. Then, we try on clothing items to user x and an avatar with an average face image that has similar tastes to user x. This allows user y to identify user x's personal information without It is possible to get recommendations for clothing that suit user x without any hassle.
[0078] In the second embodiment, the correlation degree of the purchase history is used. However, for example, the correlation degree of the posting history may be used. In this case, the similar preference face image collecting unit 23 collects the posting history for a predetermined period or until the posting history matches. or similar numbers uploaded to SNS, etc., rather than from shopping sites, etc. Alternatively, the face image may be acquired by extracting clothing accessories from a photographic image. 3) Extracting clothing-related keywords from text uploaded to SNS etc. It may also be obtained by Third embodiment An avatar generation device 30 according to a third embodiment will be described with reference to FIGS. 8 and 9. The avatar generating device 20 according to the third embodiment generates a predetermined number of images of a user x, such as a favorite celebrity. We generate an avatar using an average face image obtained by averaging the facial images of the favorite faces. Famous people are those who are famous for their face, fashion, personality, speaking skills, music, paintings, videos, and other artistic works. There are many reasons why you may like someone, such as their looks, personality, or lifestyle. From a fashion perspective, it is preferable for the person to be a celebrity you like, especially one whose face you like. FIG. 8 shows one example of the configuration of an avatar generation device 30 according to the third embodiment. The avatar generating device 30 includes a face image acquiring unit 31 and a face type determining unit 32. A determination unit 32, a favorite face image collection unit 33, a transformation policy setting unit 34, and a face transformation unit 35. and a whole-body image synthesis unit 36. The configuration and operation of the face image acquisition unit 11 and the whole-body image synthesis unit 15 in the first embodiment are the same as those in the first embodiment. The configuration and operation of the face transformation unit 35 are the same as those of the face transformation unit 35. The configuration and operation of the face transformation unit 25 in the second embodiment are the same as those of the face transformation unit 25 in the second embodiment, so the description will be omitted. In addition, the configuration and operation of the face type determination unit 32 are such that the determination result and the like are processed on behalf of the transformation policy setting unit 13. The face type determination unit 31 outputs the face type image to the favorite face image collection unit 33 instead. Since the configuration and operation are the same as those of the determination unit 12, the description will be omitted.
[0079] The favorite face image collecting unit 33 receives the setting information, the face image, and The face distance information, the face type Ti, and the judgment result for each judgment item used to judge the face type Ti are The favorite face image collecting unit 33 acquires the favorite face image via the storage control unit 114 of the control device 110. Acquire information on celebrities having favorite faces of user x stored in the storage unit 104. The person information may be included in advance in the setting information acquired by the face image acquisition unit 31, or may be stored in the control device. Alternatively, the user x may input the information from the operation unit 103 via the operation information acquisition unit 113 of 110 . The celebrity information includes information on a predetermined number of celebrities. The celebrities may be, for example, models, entertainers, sportspeople, etc. The favorite face image collecting unit 33 collects the favorite face images of the athletes, politicians, and writers. Favorite face images, which are face images of a large number of celebrities, are collected from the web, SNS, etc. The favorite face image includes the distance to the target and the three-dimensional coordinates of the target. The favorite face image collecting unit 33 is configured to collect face distance information from the favorite face image collecting unit 33. and face image and face distance information, and collected favorite face image and favorite face distance information. and output the result to the modification policy setting unit 34.
[0080] The transformation policy setting unit 34 receives the setting information, the face image, and the like from the favorite face image collection unit 33. and face distance information, and the collected favorite face image and favorite face distance information are acquired. The transformation policy setting unit 34 sets the transformation policy based on the face image of the user x and the collected Set the average image of the favorite face images to be used for avatar generation. The unit 34 stores setting information, face image and face distance information, favorite face image and favorite The face distance information and the set transformation policy are output to face transformation section 35.
[0081] The face transformation unit 35 receives the setting information, the face image, and the face distance information from the transformation policy setting unit 34. Then, the favorite face image, the favorite face distance information, and the transformation policy are acquired. The shaper 35 generates a transformed face image and transformed face distance information based on the transformation policy. The transformation unit 35 transforms the face image of the user x and the collected favorite images based on the transformation policy. The face transformation unit 35 averages the input face images to generate an average face image, which is used as a transformed face image. Based on the policy, the face distance information of user x and the collected multiple favorite face distance information are The face transformation unit 35 generates average face distance information as transformed face distance information. The generated deformed face image and deformed face distance information are output to the whole body image synthesis unit 36 .
[0082] FIG. 9 is a flowchart showing an example of the process of the avatar generation device 30 according to the third embodiment. The avatar generating device 30 receives the operation information via the operation information acquisition unit 113 of the control device 110. When the unit 103 receives a predetermined start operation, the process proceeds to step ST31 shown in FIG. to start processing.
[0083] The avatar generating device 30 acquires a face image from a captured image (step ST31). Specifically, the avatar generating device 30 first inputs the face image captured by the imaging unit 101 to the face image acquisition unit 31. The avatar generation device 30 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 31 acquires the face part from the whole body image and the whole body distance information. The avatar generation device 30 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST32.
[0084] The avatar generating device 30 determines the face type Ti from the face image (step ST32). More specifically, the avatar generating device 30 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generation device 30 then causes the face type determination unit 3 2 determines the face type Ti based on the determination results for each determination item. 30 proceeds to step ST33.
[0085] The avatar generating device 30 collects favorite face images, which are face images of celebrities who have favorite faces. More specifically, the avatar generating device 30 first collects the favorite The input face image collecting unit 33 acquires celebrity information of the favorite face of the user x. The information includes information on a predetermined number of celebrities. The avatar generating device 30 then selects the favorite face image. The collection unit 33 stores a favorite face image and a favorite face image which are the acquired face images of a predetermined number of celebrities. The face distance information is collected from the web, SNS, etc. Move on to step ST34.
[0086] The avatar generating device 30 sets a transformation policy (step ST34). The avatar generating device 30 sets the following transformation policy in the transformation policy setting unit 34: Use the face image of the user and the average image of the user's favorite face images to generate an avatar. The avatar generation device 30 then proceeds to step ST35.
[0087] The avatar generating device 30 averages the face image and the favorite face images to generate an average face image. As a result, the avatar generating device generates a deformed face image (step ST35). 30, the face transformation unit 35 averages the face image and the collected multiple favorite face images to generate a user An average face image of user x and multiple celebrities is generated and used as a transformed face image. In addition, the avatar generating device 30 transmits the face distance information and the collected favorites to the face transforming unit 35. The average face distance information between user x and multiple celebrities is calculated by averaging the face distance information. The avatar generating device 30 then proceeds to step ST36. do.
[0088] The avatar generation device 30 generates an avatar image by synthesizing the transformed face image with the captured image ( (Step ST36) More specifically, the avatar generating device 30 first includes a whole-body image synthesizing unit 36, to acquire the whole-body image of the user x captured by the imaging unit 101 and whole-body distance information. The avatar generating device 30 then inputs the whole-body image synthesis unit 36 to synthesize the deformed face image with the whole-body captured image. The avatar generating device 30 generates an avatar image, which is a transformed whole-body image, by synthesizing the avatar image with the avatar image. The whole-body image synthesis unit 36 synthesizes the deformed face distance information with the whole-body distance information to generate the deformed whole-body distance information. The avatar generation device 30 generates avatar distance information according to the flowchart shown in FIG. This ends the port processing.
[0089] As described above, the avatar generation device 30 of the third embodiment detects the user x and the user x's Generate an avatar image of the average face image of a celebrity with a favorite face. Since people have a desire to become like celebrities, It can be assumed that the average face image will look good in the clothing that user x wants to wear.
[0090] Clothing matching using avatar images generated by the avatar generation device 30 of the third embodiment The shopping system 1 is a system for allowing a user x to virtually try on clothing items during remote shopping, etc. Then, an avatar with the average face image of user x and a celebrity with user x's favorite face is created. This allows user y to try on clothing items. It is possible to have clothing items that suit user x recommended without having to specify a specific style. (Fourth embodiment) An avatar generation device 40 according to a fourth embodiment will be described with reference to FIGS. 10 to 15. The avatar generating device 40 according to the fourth embodiment performs a predetermined The avatar is generated by automatically transforming the selected facial features using the transformation method. FIG. 13 is a block diagram showing an example of a configuration of an avatar generation device 40 according to the fourth embodiment. The generating device 40 includes a face image acquiring unit 41, a face type determining unit 42, and a face part transformation setting unit 43. The facial image acquisition unit 41 includes a facial part transformation unit 44 and a whole body image synthesis unit 45. The configuration and operation are similar to those of the face image acquisition unit 11 in the first embodiment. The configuration and operation of the face type determination unit 42 are as follows: The first embodiment is the same as the first embodiment except that the facial part transformation setting unit 43 outputs the same to the facial part transformation setting unit 43 instead of the policy setting unit 13. The configuration and operation of the face type determination unit 12 are the same as those of the face type determination unit 12, so the description will be omitted. The configuration and operation of the whole body image synthesis unit 45 is as follows: 4, except that the whole body image synthesis of the first embodiment is performed using the facial part transformation unit 44. Since the configuration and operation are similar to those of the generating unit 15, the description thereof will be omitted.
[0091] The face part transformation setting unit 43 receives the setting information, the face image and the face distance from the face type determination unit 42. Obtain the separation information, face type Ti, and the judgment result for each judgment item used to judge the face type Ti. The facial part transformation setting unit 43 determines the facial part of the face type Ti from the facial image based on the face type Ti. The facial parts that do not affect the judgment result are set as the facial parts to be transformed. The parts are classified according to the judgment numbers N1, N2, N3, N4, N5, N6, and N7. The facial part transformation setting unit 43 generates a facial part list including the facial parts to be transformed. The facial part transformation setting unit 43 determines the facial type Ti of the user x and the facial parts to be transformed. The transformation method is set based on the result. The transformation method is set by setting the upper limit value corresponding to the face part to be transformed. The face feature deformation setting unit 43 includes a deformation allowable range including a lower limit value Vmax and a lower limit value Vmin. Setting information, face image, face distance information, generated face part list and transformation method The signal is output to unit 44.
[0092] The facial part transformation unit 44 receives the setting information, the face image, and the face distance information from the facial part transformation setting unit 43. The facial part transformation unit 44 acquires the generated facial part list and the transformation method. The transformation setting value is set based on the judgment number Ni corresponding to each face part. The deformation setting value is the ratio of two predetermined points set for each judgment item. The two predetermined positions are shown in Table 7 below. The facial part transformation unit 44 transforms the facial parts The facial part image to be transformed is extracted from the list. A transformed face obtained by transforming the facial part image to be transformed based on the transformation method and transformation setting value set The facial part transformation unit 44 generates a facial part image by dividing the facial part image registered in the facial part list based on the face distance information. The facial part transformation unit 44 extracts the facial part distance information of the transformation target. A transformed face part obtained by transforming the distance information of the face part to be transformed based on a transformation method and a transformation setting value. The facial part transformation unit 44 generates distance information. The facial part transformation unit 44 transforms the facial part image by combining the transformed facial part image with the facial image. A facial part transformation unit 44 combines the transformed facial part distance information with the facial distance information to generate a facial image. The facial part transformation unit 44 generates transformed face distance information based on the setting information, the generated transformed face image, and The transformed face distance information is also output to the whole body image synthesis unit 45.
[0093] FIG. 11 is a flowchart showing an example of the process of the avatar generation device 40 according to the fourth embodiment. The avatar generating device 40 receives operation information via the operation information acquiring unit 113 of the control device 110. When the operation unit 103 receives a predetermined start operation, step ST41 shown in FIG. and start processing.
[0094] The avatar generating device 40 acquires a face image from a captured image (step ST41). Specifically, the avatar generating device 40 first receives the face image captured by the imaging unit 101 and stores it in the face image acquisition unit 41. The avatar generation device 40 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 41 acquires the face part from the whole body image and the whole body distance information. The avatar generation device 40 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST42.
[0095] Avatar generating device 40 determines face type Ti from the face image (step ST42). More specifically, the avatar generating device 40 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generation device 40 then causes the face type determination unit 4 2 determines the face type Ti based on the determination results for each determination item. 40 proceeds to step ST43.
[0096] The avatar generating device 40 sets the facial parts to be transformed and the transformation method (step S More specifically, the avatar generating device 40 first causes the facial part transformation setting unit 43 to Based on the face type Ti, a face pattern that does not affect the determination result of the face type Ti from the face image is generated. The avatar generation device 40 then sets the facial part to be transformed. The shape setting unit 43 generates a face part list including the face parts to be transformed. The generating device 40 then inputs the face type Ti and the transformation of the face parts of the user x to the facial part transformation setting unit 43. The transformation method is set based on the result of the determination of the target facial features. Then, proceed to step ST44.
[0097] The avatar generating device 40 transforms the facial features to be transformed based on the set transformation method. To be more specific, the avatar generation device 4 0 first causes the facial part transformation unit 44 to set transformation setting values based on the transformation method. The butter generating device 40 then inputs to the face part transforming unit 44 the face part list from the face image. The avatar generating device 40 extracts a facial part image to be transformed. The face part to be transformed is input to the face transformation unit 44 based on the set transformation method and transformation setting value. The avatar generation device 40 then generates a transformed facial part image by transforming the image. The face transformation unit 44 receives the face part distance information from the face distance information and the face part distance information registered in the face part list. The avatar generating device 40 also extracts the separation information from the face part transformation unit 44. The facial feature distance information of the transformation target is transformed based on the transformation method and transformation setting value. The avatar generating device 40 then causes the facial feature transforming unit 44 to generate distance information for the facial features. The avatar generating device 4 generates a deformed face image by synthesizing the deformed face part image with the face image. 0, the facial part transformation unit 44 combines the transformed facial part distance information with the face distance information to generate the transformed face distance The avatar generation device 40 then proceeds to step ST45.
[0098] The avatar generation device 40 generates an avatar image by synthesizing the transformed face image with the captured image ( Step ST45). More specifically, the avatar generating device 40 controls the whole-body image synthesis unit 45 to The deformed face image is then combined with a whole-body captured image to generate an avatar image, which is a deformed whole-body image. The avatar generating device 40 also transmits the deformed face distance information to the whole-body image synthesis unit 45 as whole-body distance information. The avatar generating device 4 generates avatar distance information, which is transformed whole-body distance information, by synthesizing the avatar distance information with the avatar image. 0 ends the process of the flowchart shown in FIG.
[0099] The following describes in detail how to set the transformation method for the facial parts to be transformed. It includes one or more facial parts to be transformed. Table 6 shows a list of facial parts corresponding to face type Ti. 6 is a table showing an example of restrictions on facial parts to be transformed that are included in the table. For all face types Ti, the facial three-dimensionality of the facial feature corresponding to judgment number N5 is subject to transformation. In the example shown in Table 6, face types T2 and T6 are excluded from the judgment number N6. The size of the eyes, which are facial features that correspond to the shape of the face, is excluded from the transformation.
[0100] [Table 6]
[0101] Table 7 shows the faces to be transformed corresponding to the judgment numbers N1, N2, N3, N4, N5, N6, and N7. 1 is a table showing the thresholds, upper limit values, and lower limit values for the ratio of each part that is judged as A. The judgment result for each judgment number Ni corresponding to the face part is the judgment value V indicated by the ratio. If the judgment value V is less than the threshold value, it is judged as A, and if the judgment value V is greater than the threshold value, it is judged as B. The upper and lower limits indicate the deformation tolerance range that combines the values that result in an A rating and the values that result in a B rating. In other words, the upper limit shown in Table 7 is the upper limit of the range that is classified as B. The value is the lower limit of the range for which the evaluation is A. The lower limit of the range for which the evaluation is B is is the same as the threshold value, but in the present invention, it may be larger than the threshold value. is the same as the threshold value in the fourth embodiment, but may be smaller than the threshold value in the present invention.
[0102] [Table 7]
[0103] FIG. 12 is a flowchart showing an example of the processing of the facial part transformation setting unit 43 according to the fourth embodiment. The facial part transformation setting unit 43 performs, for example, the facial part transformation shown in step ST43 of FIG. When the process of the type determination unit 42 is completed, the process proceeds to step ST201 shown in FIG. More specifically, the process of the face type determination unit 42 starts with, for example, setting information and , face image and face distance information, face type Ti, and judgment items used to judge face type Ti The process ends by outputting the judgment result of (a) to facial part transformation setting unit 43.
[0104] The facial part transformation setting unit 43 determines the face type Ti of the user x determined by the face type determination unit 42 as In the fourth embodiment, the face type Ti is obtained as shown in Table 1. Face type: T1, T2, T3, T4, T5, T6, T7, T8. Face parts The modification setting unit 43 proceeds to step ST202.
[0105] The facial part transformation setting unit 43 determines the facial parts to be transformed based on the acquired face type Ti. In the fourth embodiment, the facial parts to be transformed are set as shown in Table 2. They are classified according to the judgment numbers N1, N2, N3, N4, N5, N6, and N7 shown in the table below. The target facial parts are set as shown in Table 6. The facial part transformation setting unit 43 performs the steps Transition to ST203.
[0106] The facial part transformation setting unit 43 sets the determination result for each determination number Ni corresponding to the facial part to be transformed. The result of the determination of the facial part to be transformed is obtained as shown in Table 2. Then, the result is either A or B. The facial part transformation setting unit 43 proceeds to step ST204. do.
[0107] The facial part transformation setting unit 43 determines whether the face type Ti acquired in step ST201 is a face type It is determined whether the face is any one of T1, T2, T3, and T4 (step ST204). The types T1, T2, T3, and T4 are face types Ti with a large number of A judgments. If the type Ti is any of the face types T1, T2, T3, and T4 (step ST20 4; Yes), the facial part transformation setting unit 43 proceeds to step ST205. If i is not one of the face types T1, T2, T3, or T4 (step ST204; No ), facial part transformation setting section 43 proceeds to step ST206.
[0108] If the face type Ti is any of face types T1, T2, T3, and T4 (step ST 204; Yes), the facial part transformation setting unit 43 uses the transformation target acquired in step ST203 It is determined whether the determination result of the facial part is A or not (step ST205). If the result of the judgment of the part is judgment A (step ST205; Yes), the face part transformation setting The determination unit 43 proceeds to step ST207. If not (step ST205; No), the facial part transformation setting unit 43 goes to step ST20 Move to 8.
[0109] If the face type Ti is not any of face types T1, T2, T3, or T4 (step ST 204; No), the facial part transformation setting unit 43 performs the transformation target acquired in step ST203. It is determined whether the facial part determination result is A determination (step ST206). If the result of the judgment of the character is judgment A (step ST206; Yes), the face part transformation setting If the determination result of the facial part to be transformed is not determination A, the processing unit 43 proceeds to step ST208. If not (step ST206; No), the facial part transformation setting unit 43 proceeds to step ST209. transition to.
[0110] If the face type Ti is any of face types T1, T2, T3, and T4 (step ST 204; Yes), and the determination result of the facial part to be transformed is A (step S T205; Yes), the facial part deformation setting unit 43 sets the deformation allowable range of the facial part to be deformed. , the range in which the face part transformation is determined to be A (step ST207). 43 sets the deformation tolerance range so that the number of facial parts that are judged as A does not change or increases. The facial part transformation setting unit 43 ends the process of the flowchart shown in FIG.
[0111] If the face type Ti is any of face types T1, T2, T3, and T4 (step ST 204; Yes), and if the determination result of the facial part to be transformed is not A (step S T205; No), the facial part deformation setting unit 43 sets the deformation allowable range of the facial part to be deformed as The range is set to be A or B (step ST208). The deformation setting unit 43 sets the allowable deformation range so that the number of face parts that are judged as B remains the same or decreases. Also, if face type Ti is not one of face types T1, T2, T3, or T4, (Step ST204; No), and the determination result of the facial part to be transformed is A (Step ST206; Yes), the facial part transformation setting unit 43 performs transformation of the facial part to be transformed. The shape tolerance range is set to a range that results in either A or B judgment (step ST208). That is, the facial part transformation setting unit 43 changes the number of facial parts that are judged as A to be unchanged or decreased. The face part deformation setting unit 43 sets the deformation allowable range. End the process.
[0112] If the face type Ti is not any of face types T1, T2, T3, or T4 (step ST 204; No), and the determination result of the facial part to be transformed is not A (step ST 206; No), the facial part deformation setting unit 43 sets the deformation allowable range of the facial part to be deformed as B In other words, the facial part transformation setting unit 43 sets the range to be judged (step ST209). Set the deformation tolerance range so that the number of facial parts that are judged as B does not change or increases. The part transformation setting unit 43 ends the processing of the flowchart shown in FIG.
[0113] FIG. 13 is a flowchart showing an example of processing by the facial part transformation unit 44 according to the fourth embodiment. The facial part transformation unit 44 performs, for example, the facial part transformation shown in step ST44 of FIG. When the process of the setting unit 43 is completed, the process proceeds to step ST251 shown in FIG. In more detail, the process of the face part transformation setting unit 43 starts with setting information, a face image, The face distance information, the face part list, and the transformation method are output to the face part transformation unit 44. And that's it.
[0114] The facial part transformation unit 44 performs a transformation for the facial part to be processed based on the acquired facial part list. The facial part transformation unit 44 acquires the corresponding judgment number Ni (step ST251). Transition to step ST252.
[0115] The facial part transformation unit 44 acquires the judgment value V corresponding to the judgment number Ni (step ST2 52). The judgment value V is the ratio value of each judgment item shown in Table 7. Then, the process proceeds to step ST253.
[0116] The facial part transformation unit 44 acquires the judgment result corresponding to the judgment number Ni (step ST2 53) The judgment result for the facial part to be transformed is either A or B, as shown in Table 7. The facial part transformation section 44 proceeds to step ST254.
[0117] The facial part transformation unit 44 performs transformation within the permissible range set by the flowchart shown in FIG. Based on this, the upper limit value Vmax corresponding to the judgment number Ni and the judgment result is obtained (step For example, if the deformation tolerance range is set to a range that results in an A rating, the upper limit V max is the upper limit of the range that is rated A. For example, the range in which the deformation tolerance is rated B When the upper limit Vmax is set to , the upper limit Vmax is the upper limit Vmax shown in Table 7. Then proceed to step ST255.
[0118] The facial part transformation unit 44 performs transformation within the permissible range set by the flowchart shown in FIG. Based on this, the judgment number Ni and the lower limit value Vmin corresponding to the judgment result are obtained (step For example, if the deformation tolerance range is set to a range that results in an A rating, the upper limit V The max is the lower limit shown in Table 7. For example, if the deformation tolerance range is set to the range that results in a B rating, In this case, the upper limit value Vmax is the lower limit value of the range in which the face part transformation unit 44 is judged as B. Then proceed to step ST256.
[0119] The facial part transformation unit 44 performs step S The value obtained by subtracting the lower limit value Vmin obtained by T255 and dividing the result by 2 is equal to or less than the judgment value V. More specifically, it is determined whether or not (upper limit value Vmax-lower limit value Vmin) / 2 ≦ judgment value V is determined. If the value of (in) / 2 is equal to or smaller than the judgment value V (step ST256; Yes), The forming unit 44 proceeds to step ST257. (upper limit value Vmax-lower limit value Vmin) / 2 If the value of is greater than the judgment value V (step ST256; No), the facial part transformation unit 44 Proceed to step ST258.
[0120] If the value of (upper limit value Vmax-lower limit value Vmin) / 2 is equal to or less than the judgment value V (step S T256; Yes), the facial part transformation unit 44 sets the transformation setting value to the lower limit value Vmin ( Step ST257) That is, when the judgment value V is closer to the lower limit value Vmin than the upper limit value Vmax, If the value is not less than the lower limit, the facial feature transformation unit 44 sets the transformation setting value to the lower limit Vmin. The art deformation section 44 proceeds to step ST259.
[0121] If the value of (upper limit value Vmax-lower limit value Vmin) / 2 is greater than the judgment value V (step S T256; No), the facial part transformation unit 44 sets the transformation setting value to the upper limit value Vmax (step Step ST258) In other words, the judgment value V is closer to the upper limit value Vmax than the lower limit value Vmin. If the value is the upper limit value Vmax, the facial feature transformation unit 44 sets the transformation setting value to the upper limit value Vmax. The transformation unit 44 proceeds to step ST259.
[0122] The facial part transformation unit 44 stores the transformation setting value corresponding to the judgment number Ni (step ST 259). The facial feature transformation unit 44 ends the process of the flowchart shown in FIG. Steps ST252 to ST259 are executed for each judgment number Ni. This process may be repeated until a transformation set value corresponding to the number Ni is set.
[0123] As described above, in the avatar generation device 40 of the fourth embodiment, in the example shown in FIG. For each face part, the amount of deformation is maximized within the allowable range. By increasing the amount of deformation for the face image of user x, the personal information of user x can be expressed by the avatar image. This can improve the confidentiality of information, etc.
[0124] FIG. 14 is a flowchart showing another example of the processing of the facial part transformation unit 44 according to the fourth embodiment. The facial part transformation unit 44 transforms, for example, the facial part shown in step ST44 of FIG. When the processing of the transformation setting unit 43 is completed, the process proceeds to step ST301 shown in FIG. More specifically, the process of the face part transformation setting unit 43 starts with setting information, a face image, The image, face distance information, the generated face part list, and the transformation method are output to the face part transformation unit 44. The process ends by:
[0125] The facial part transformation unit 44 acquires a random number value R (step ST301). For example, the random number pattern may be initialized based on the current time. The random number R is set in the range of 0 to 1. The transformation unit 44 proceeds to step ST302.
[0126] The processes from step ST302 to step ST303 shown in FIG. 14 are the same as those shown in FIG. Since the process is similar to the process from step ST251 to step ST252, the explanation will not be given. The facial part transformation unit 44 proceeds to step ST304. The process from step ST304 to step ST305 is the same as that from step ST254 shown in FIG. Since the process is the same as the process from step ST255 to step ST255, the description will be omitted. The molding section 44 proceeds to step ST306.
[0127] The facial feature deformation unit 44 determines, based on the random number R, the upper limit value Vmax, and the lower limit value Vmin, A transformation setting value is calculated (step ST306). More specifically, the transformation setting value=random value R The formula shown below is calculated: × (upper limit value Vmax-lower limit value Vmin) + lower limit value Vmin. The transformation setting value is set to the upper limit value Vmax and the lower limit value Vmin. The unit 44 proceeds to step ST307.
[0128] The facial part transformation unit 44 stores the transformation setting value corresponding to the judgment number Ni (step ST 307). The facial part transformation unit 44 ends the process of the flowchart shown in FIG. Steps ST303 to ST307 are executed for each judgment number Ni. Step ST30 may be repeated until the transformation set value corresponding to the number Ni is set. 1 may be executed at any time before step ST306 and is set for each judgment number Ni. This is also fine.
[0129] As described above, in the avatar generation device 40 of the fourth embodiment, another example shown in FIG. Then, for each judgment number Ni corresponding to a face part, we randomly set the deformation setting value within the deformation tolerance range. By changing the amount of deformation for each facial part of the face image of user x, the avatar This improves the confidentiality of personal information of user x by using a single image. Since the transformation settings change depending on the user's input, multiple patterns of avatar images can be generated. This can be done.
[0130] FIG. 15 is a flow chart showing yet another example of the processing of the facial part transformation unit 44 according to the fourth embodiment. The facial part transformation unit 44 transforms, for example, the facial part shown in step ST44 of FIG. When the processing of the part transformation setting unit 43 is completed, step ST351 shown in FIG. In more detail, the process of the face part transformation setting unit 43 starts with the setting information The facial image, the facial distance information, the generated facial part list, and the transformation method are input to the facial part transformation unit 44. The command terminates by outputting to
[0131] Facial part transformation unit 44 acquires an increase / decrease rate value P (step ST351). In this case, the facial part transformation unit 44 receives the facial part transformation data from the memory unit 10 via the memory control unit 114 of the control device 110. The increase / decrease rate value P stored in advance in 4 is obtained. The increase / decrease rate value P is greater than or equal to -1.0 and less than or equal to 1.0. The facial feature transformation unit 44 proceeds to step ST352. do.
[0132] The processes from step ST352 to step ST356 shown in FIG. 15 are the same as those shown in FIG. The process is the same as that from step ST251 to step ST255, so no explanation will be given here. The facial part transformation section 44 proceeds to step ST357.
[0133] The facial part transformation unit 44 determines the judgment value V, the increase / decrease rate value P, the upper limit value Vmax, and the lower limit value Vmi Based on n, a modified set value is calculated (step ST357). Calculate the formula expressed as fixed value = judgment value V + increase / decrease rate value P × (upper limit value Vmax - lower limit value Vmin). The facial part transformation section 44 proceeds to step ST358.
[0134] The facial part transformation unit 44 determines whether the transformation setting value calculated in step ST357 is equal to or smaller than the lower limit value Vmin. If the deformation set value is equal to or less than the lower limit Vmin, the determination is made as to whether or not the deformation set value is equal to or less than the lower limit Vmin (step ST358). If so (step ST358; Yes), the facial part transformation unit 44 proceeds to step ST359. If the transformation setting value is greater than the lower limit Vmin (step ST358; No), the face pattern is The art deformation section 44 proceeds to step ST360.
[0135] If the transformation setting value is equal to or less than the lower limit Vmin (step ST358; Yes), The face transformation unit 44 resets the transformation setting value to the lower limit value Vmin (step ST359). The part transformation section 44 proceeds to step ST362.
[0136] If the transformation setting value is greater than the lower limit Vmin (step ST358; No), the facial part The transformation unit 44 determines whether the transformation set value calculated in step ST357 is equal to or greater than the upper limit value Vmax. If the deformation set value is equal to or greater than the upper limit value Vmax (step ST360), If the answer is "Yes" in step ST360, the facial part transformation unit 44 proceeds to step ST361. If the set value is smaller than the upper limit value Vmax (step ST360; No), the facial part transformation unit 44 proceeds to step ST362.
[0137] If the transformation setting value is equal to or greater than the upper limit value Vmax (step ST360; Yes), The face transformation unit 44 resets the transformation setting value to the upper limit value Vmax (step ST361). The part deformation unit 44 proceeds to step ST362. If the deformation set value is significantly smaller than the upper limit value Vmax, the deformation set value is The value remains the same.
[0138] The facial part transformation unit 44 stores the transformation setting value corresponding to the judgment number Ni (step ST 362). The facial feature transformation unit 44 ends the process of the flowchart shown in FIG. Steps ST353 to ST362 are executed for each judgment number Ni. This process may be repeated until a transformation set value corresponding to the number Ni is set.
[0139] As described above, in the avatar generation device 40 of the fourth embodiment, the further In one example, based on the balance of the judgment value V for each judgment number Ni corresponding to the face part, The transformation setting value is set by increasing or decreasing the facial features by a specified increase or decrease ratio value P. By changing the amount of deformation with the same increase / decrease ratio value P, the balance of the face is maintained. Therefore, it is possible to generate an avatar image that resembles user x. Also, by changing the increase / decrease rate value P, By changing the settings, the transformation values change, allowing you to generate multiple patterns of avatar images. It is possible.
[0140] In the examples shown in FIGS. 13 to 15, the facial part transformation unit 44 performs the determination numbers N1, N2, N 3. I set the face part transformation setting values corresponding to N4, N5, N6, and N7, but the judgment number is N 8. Regarding face parts for N9, N10, N11, N12, N13, N14, and N15 The facial part transformation unit 44 transforms the facial parts to be transformed based on the transformation setting value. The facial part image obtained by transforming the target face image and the facial part distance information of the target face image are transformed. The facial part transformation unit 44 synthesizes the transformed facial part image with the facial image. The facial part transformation unit 44 converts the transformed facial part distance information into face distance information. The facial part transformation unit 44 combines the setting information and the generated transformed facial distance information. The image and the transformed face distance information are output to a whole-body image synthesis unit 45 . Fifth embodiment An avatar generation device 50 according to a fifth embodiment will be described with reference to FIGS. The avatar generating device 50 according to the fifth embodiment generates a predetermined The facial parts to be transformed are transformed by the transformation setting values input by the user x's operation. FIG. 16 shows an avatar generation device 50 according to the fifth embodiment. 5 is a block diagram showing an example of the configuration of an avatar generating device 50. The avatar generating device 50 includes a face image acquiring unit 51 and A face type determination unit 52, a face part transformation setting unit 53, a face part transformation adjustment unit 54, and a whole body image The face image acquisition unit 51 and the face type determination unit 52 are configured as follows: The configuration and operation of the face image acquisition unit 41 and face type determination unit 42 of the fourth embodiment are the same as those of the first embodiment. The configuration and operation of the facial part transformation setting unit 53 are the same as those in The facial parts list and the transformation method are input to the facial parts transformation adjustment unit 44 instead of the facial parts transformation unit 44. The configuration and operation of the facial part transformation setting unit 43 of the fourth embodiment, except for outputting the facial part transformation setting unit 43 to the 54, The configuration and operation of the whole body image synthesis unit 55 are the same as those of the first embodiment. The face part deformation adjustment unit 54 transfers the face image and deformation distance information, etc. to the face part deformation adjustment unit 54 instead of the face part deformation unit 44. The configuration and operation are the same as those of the whole-body image synthesis unit 45 of the fourth embodiment, except that the image is acquired from Therefore, the description will be omitted.
[0141] The facial part transformation adjustment unit 54 receives the setting information, the face image, and the face distance from the facial part transformation setting unit 53. The facial part transformation adjustment unit 54 acquires the information, the generated facial part list, and the transformation method. Extract the facial part images to be transformed from the facial image registered in the facial part list. The transformation adjustment unit 54 adjusts the transformation method set and the transformation input by the operation of the user x. Based on the set value, a transformed face part image is generated by transforming the face part image to be transformed. The part transformation adjustment unit 54 determines the face to be transformed that is registered in the face part list based on the face distance information. The facial feature transformation adjustment unit 54 extracts the feature distance information based on the set transformation method and the user's Based on the facial part transformation setting value input by the user x, the distance between the facial parts to be transformed is The facial part deformation adjustment unit 54 generates deformed facial part distance information by deforming the distance information. The facial feature image is synthesized with the facial image to generate a deformed facial image. The facial feature distance information is combined with the face distance information to generate the deformed face distance information. The unit 54 transmits the setting information, the generated deformed face image, and the deformed face distance information to the whole body image synthesis unit 55. Output.
[0142] FIG. 17 is a flowchart showing an example of the process of the avatar generation device 50 according to the fifth embodiment. The avatar generating device 50 receives operation information via the operation information acquisition unit 113 of the control device 110. When the operation unit 103 receives a predetermined start operation, step ST51 shown in FIG. and start processing.
[0143] The avatar generating device 50 acquires a face image from a captured image (step ST51). Specifically, the avatar generating device 50 first receives a facial image captured by the imaging unit 101 and stores it in the facial image acquisition unit 51. The avatar generation device 50 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 51 acquires the face part from the whole body image and the whole body distance information. The avatar generation device 50 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST52.
[0144] The avatar generating device 50 determines the face type Ti from the face image (step ST52). More specifically, the avatar generating device 50 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generation device 50 then causes the face type determination unit 5 2 determines the face type Ti based on the determination results for each determination item. 50 proceeds to step ST53.
[0145] The avatar generating device 50 sets the facial parts to be transformed and the transformation method (step S More specifically, the avatar generating device 50 first inputs to the facial part transformation setting unit 53 Based on the face type Ti, a face pattern that does not affect the determination result of the face type Ti from the face image is generated. The avatar generating device 50 then sets the facial part to be transformed. The shape setting unit 53 generates a face part list including the face parts to be transformed. The generating device 50 then inputs the face type Ti and the transformation of the face parts of the user x to the facial part transformation setting unit 53. The transformation method is set based on the result of the determination of the target facial features. Then, proceed to step ST54.
[0146] The avatar generating device 50 performs the transformation according to the input transformation setting values within the range based on the set transformation method. Therefore, the facial parts to be transformed are transformed to generate a transformed facial image (step ST54). Specifically, the avatar generating device 50 first causes the facial feature transformation adjustment unit 54 to Extract the facial part images to be transformed from the parts list. The face part transformation adjustment unit 54 receives the facial distance information and stores the facial part information registered in the facial part list. The avatar generating device 50 then extracts distance information of the facial parts to be transformed. The transformation adjustment unit 54 receives the set transformation method and the transformation input by the operation of the user x. Based on the set value, a transformed face part image is generated by transforming the face part image to be transformed. The avatar generating device 50 also transmits the set transformation method and Based on the facial part transformation setting value input by user x, the facial part to be transformed is The avatar generating device 50 generates transformed facial part distance information by transforming the facial part distance information. A facial part transformation adjustment unit 54 combines the transformed facial part image with the face image to generate a transformed face image. The avatar generating device 50 also controls the facial part deformation adjusting unit 54 to adjust the deformation facial part distance The avatar generating device 50 synthesizes the information with the face distance information to generate modified face distance information. Proceed to step ST55.
[0147] The avatar generation device 50 generates an avatar image by synthesizing the transformed face image with the captured image ( More specifically, the avatar generating device 50 controls the whole-body image synthesis unit 55 to The deformed face image is then combined with a whole-body captured image to generate an avatar image, which is a deformed whole-body image. The avatar generating device 50 also transmits the deformed face distance information to the whole-body image synthesis unit 55 as whole-body distance information. The avatar generating device 5 generates avatar distance information, which is transformed whole-body distance information, by synthesizing the avatar distance information with the avatar image. 0 ends the processing of the flowchart shown in FIG.
[0148] FIG. 18 shows an example of an operation screen for operating the facial part transformation adjustment unit 54 according to the fifth embodiment. The operation screen is an application displayed on the display unit 105 of the mobile terminal 100X of the user x. The application window 150 is displayed on the avatar image display area. The image includes a parameter display area 160, a parameter display area 170, and a confirmation button 180. The display area 160 displays a face image of the user x or a transformed face image transformed by the operation of the user x. The image is displayed.
[0149] The parameter display area 170 includes an item name display area 171 and a parameter bar for each face part. - 172, an upper limit value display area 173, a lower limit value display area 174, and an adjustment knob 175, Includes a plus button 176, a minus button 177, and a transformation setting value display area 178. In the item name display area 171, the name of the judgment item of the judgment number Ni corresponding to the face part is displayed. In the fifth embodiment, the parameter bar 172 is a horizontally long rectangle. On the tab 172, there is a rectangular adjustment knob 175 that can be slid along the longitudinal direction. In the upper limit display area 173, the face part transformation setting unit 53 sets the upper limit for each face part. The upper limit value Vmax set for each face part is displayed in the lower limit display area 174. The lower limit value Vmin set by the shape setting unit 53 is displayed.
[0150] The adjustment knob 175 is slid along the longitudinal direction on the parameter bar 172 for each facial feature. The adjustment knob 175 is used to set the transformation in accordance with the position on the parameter bar 172. For example, when the adjustment knob 175 is at the left end, the transformation setting value is at the lower limit V min, and when it is at the right end, the deformation setting value is adjusted to the upper limit value Vmax. The plus button 176 receives a predetermined operation via the operation information acquisition unit 113 of the control device 110. By attaching the adjustment knob 175, the adjustment knob 175 for each face part is moved to the right by a predetermined amount on the operation screen. The minus button 177 is operated by a predetermined operation via the operation information acquisition unit 113 of the control device 110. By accepting the operation, the adjustment knob 175 for each face part is moved to the left of the operation screen in a predetermined direction. In the transformation setting value display area 178, the position of the adjustment knob 175 for each face part is The corresponding transformation settings are displayed.
[0151] The facial part transformation adjustment unit 54 displays the parameter bar 172 of the facial part that is not the transformation target, The knob 175, plus button 176 and minus button 177 are adjusted by user x. The face part transformation adjustment unit 54, for example, as shown in FIG. In the face, the parameter bar 172, adjustment knob 175, and parameter The plus button 176 and the minus button 177 are displayed as a non-transformable display area 179 in gray. It may be displayed as out.
[0152] As described above, the avatar generation device 50 of the fifth embodiment determines the transformation target according to the face type determination. The ratio, upper limit and lower limit of each facial part are used for the facial parts that are targeted. In this way, an input method is provided for user x that accepts deformation within the allowable range of deformation. Based on the face image of user x, the face parts to be transformed are manually manipulated by user x using a predetermined transformation method. It is possible to generate a deformed avatar image based on the transformation settings input by the user. This allows user x to transform the facial features of the target face, resulting in a highly convincing result. You can generate avatar images. Sixth Embodiment An avatar generation device 60 according to a sixth embodiment will be described with reference to FIGS. The avatar generating device 60 according to the sixth embodiment generates a facial profile based on a captured image of the user x. For each face, an avatar is generated by replacing the face parts of another avatar. FIG. 1 is a block diagram showing an example of a configuration of an avatar generation device 60 according to the present invention. 0 includes a face image acquisition unit 61, a face type determination unit 62, a face part acquisition unit 63, and a face part placement unit. The face image acquisition unit 61 includes a face image conversion unit 64 and a whole-body image synthesis unit 65. Since the configuration and operation of the face image acquisition unit 11 are the same as those of the face image acquisition unit 11 in the first embodiment, the description thereof will be omitted. The configuration and operation of the face type determination unit 62 is to input the determination result and the like to the transformation policy setting unit 1. 3, except that the face type determination unit 62 outputs the face type information to the face part acquisition unit 63 instead of the face type determination unit 62 in the first embodiment. The configuration and operation of the whole body image synthesis unit 65 are the same as those of the whole body image synthesis unit 65. The configuration and operation of the face transforming unit 14 is to use the transformed face image and transformation distance information, etc., instead of the face transforming unit 14. The configuration and structure of the whole body image synthesis unit 15 of the first embodiment are the same as those of the first embodiment, except that the image is acquired from the image replacement unit 64. Since the operation is the same as that of the first embodiment, the description will be omitted.
[0153] The face part acquisition unit 63 receives the setting information, the face image, and the face distance information from the face type determination unit 62. The face type Ti and the judgment result for each judgment item used to judge the face type Ti are acquired. The facial part acquisition unit 63 notifies the user x to select a classification of the facial part to be replaced. The control device 110 displays the information, the selection information, and the like on the display unit 10 via the display control unit 115. The face part acquisition unit 63 acquires the face part via the operation information acquisition unit 113 of the control device 110. Based on the input information received from the operation unit 103, a replacement avatar for the selected classification is generated. Multiple facial part images are randomly acquired. The avatar facial part images are based on the distance to the subject. Avatar face part distance information including distance and 3D coordinates of the image capture target is linked The storage unit 104 stores a database of avatar face part images for replacement of each face part. is saved.
[0154] The database contains avatar face images for replacement of each facial part and avatar face distance information. The avatar face images stored in the database are classified into face types Ti. The avatar face images stored in the database are assigned a judgment number. The judgment result is set for each Ni. The avatar face images stored in the database are The facial features are stored separately so that the individual cannot be identified. The classification items include, for example, face shape, eyebrows, eyes, nose, and mouth. The face shape is classified into judgment numbers N1, N2, , N5, N8, and N9. Eyebrows correspond to judgment number N13. Eyes correspond to judgment number N 3, N6, N10, N11, and N12. The nose corresponds to judgment numbers N4, N7, and N14. The database corresponds to the judgment number N15. The facial part acquisition unit 63 acquires the facial image and the acquired avatar facial parts. The image selection information is displayed on the display unit 105 via the display control unit 115 of the control device 110. To make.
[0155] The facial part replacement unit 64 receives the operation information from the operation unit 103 via the operation information acquisition unit 113 of the control device 110. Based on the input information received from, the facial part image of the face image is converted into a replacement avatar face part The facial part replacement unit 64 replaces the facial part image with a face part image to generate a deformed facial part image. Based on the input information received from the operation unit 103 via the operation information acquisition unit 113, The distance information of the facial parts in the distance information is replaced with the distance information of the facial parts of the replacement avatar to transform the facial part. The distance information is generated for the replacement avatar face part image by user x. The facial part replacement unit 64 displays the facial image being replaced on the display unit 105. The facial part replacement unit 64 receives a replacement operation input by the user x. The facial part replacement unit 64 synthesizes the image with the facial image to generate a deformed facial image. The facial part replacement unit 64 generates modified facial distance information by combining the distance information with the facial distance information. The facial image synthesizer 65 outputs the information, the generated deformed face image, and the deformed face distance information to the whole body image synthesizer 65. The facial part acquisition unit 63 and the facial part replacement unit 64 generate a face type of the transformed face image. Let T i be restricted so that it is not different from the face type T i of the face image of user x.
[0156] FIG. 20 is a flowchart showing an example of the process of the avatar generation device 60 according to the sixth embodiment. The avatar generating device 60 receives operation information via the operation information acquisition unit 113 of the control device 110. When the operation unit 103 receives a predetermined start operation, step ST61 shown in FIG. and start processing.
[0157] The avatar generating device 60 acquires a face image from a captured image (step ST61). Specifically, the avatar generating device 60 first inputs the face image captured by the imaging unit 101 to the face image acquisition unit 61. The avatar generation device 60 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 61 acquires the face part from the whole body image and the whole body distance information. The avatar generation device 60 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST62.
[0158] The avatar generating device 60 determines the face type Ti from the face image (step ST62). More specifically, the avatar generating device 60 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generation device 60 then causes the face type determination unit 6 2 determines the face type Ti based on the determination results for each determination item. 60 proceeds to step ST63.
[0159] The avatar generating device 60 obtains avatar facial part images for the selected facial part classification. More specifically, the avatar generating device 60 first extracts face parts. The classification of the face parts to be replaced is transmitted to the acquiring unit 63 via the operation information acquiring unit 113 of the control device 110. The face part acquisition unit 103 receives an input operation by a user x to select the face part. 63 is information for informing the selection of a classification of face parts to be replaced, and option information, etc. The control device 110 causes the display unit 105 to display the image. Next, the device 60 transmits to the face part acquisition unit 63 via the operation information acquisition unit 113 of the control device 110 Based on the input information received from the operation unit 103, a replacement avatar for the selected classification is generated. The avatar generation device 60 randomly acquires a plurality of facial part images. Move to 4.
[0160] The avatar generating device 60 replaces the selected avatar face part image (step ST More specifically, the avatar generating device 60 first controls the facial part replacing unit 64 to A replacement avatar face part image is selected via the operation information acquisition unit 113 of the device 110. The input operation by the user x to the operation unit 103 is accepted. The avatar generation device 60 Next, the face part replacement unit 64 receives the operation information of the operation unit 1 via the operation information acquisition unit 113 of the control device 110. Based on the input information received from 03, the facial part images of the face image are converted into avatar faces for replacement. The avatar generating device 60 generates a transformed face part image by replacing the facial part image with the facial part image. The facial part replacement unit 64 receives the facial part from the operation unit 103 via the operation information acquisition unit 113 of the control device 110. Based on the input information received from the The facial part distance information is replaced with the facial part distance information to generate transformed facial part distance information. Next, the facial part replacement unit 64 synthesizes the transformed facial part image with the facial image to generate a transformed facial image. The avatar generating device 60 also generates the facial part distance information in the facial part replacement unit 64. The avatar generating device 60 synthesizes the facial distance information with the facial distance information to generate modified facial distance information. Move to step ST65.
[0161] The avatar generation device 60 generates an avatar image by synthesizing the transformed face image with the captured image ( Step ST65). More specifically, the avatar generating device 60 causes the whole-body image synthesis unit 65 to The deformed face image is then combined with a whole-body captured image to generate an avatar image, which is a deformed whole-body image. The avatar generating device 60 also outputs the transformed face distance information to the whole body image synthesis unit 65 as the whole body distance information. The avatar generating device 6 generates avatar distance information, which is transformed whole-body distance information, by synthesizing the avatar distance information with the avatar image. 0 ends the processing of the flowchart shown in FIG.
[0162] FIG. 21 is a flowchart showing an example of the facial parts replacement process according to the sixth embodiment. In the facial part replacement process, steps ST401 to ST404 shown in FIG. The process in FIG. 2 is executed by the facial part acquisition unit 63. The processes from step ST405 to step ST411 shown in FIG. 1 are performed by the facial part replacement unit 64. The facial part acquisition unit 63 executes, for example, the process shown in step ST62 of FIG. When the process of the face type determination unit 62 is completed, the process proceeds to step ST401 shown in FIG. In more detail, the face type determination unit 62 performs the process by determining the face type based on the setting information and the face type. Image and face distance information, face type Ti, and judgment results for each judgment item used to judge face type Ti The determination result is output to face parts acquisition unit 63, and the process ends.
[0163] The face part acquisition unit 63 outputs the face part to the display unit 105 via the display control unit 115 of the control device 110. The face image of the user x is displayed (step ST401). The process proceeds to ST402.
[0164] The face part acquisition unit 63 acquires the face type Ti of the user x determined by the face type determination unit 62. In the sixth embodiment, the face type Ti is the face type shown in Table 1. The type is T1, T2, T3, T4, T5, T6, T7, or T8. The unit 63 proceeds to step ST403.
[0165] The facial part acquisition unit 63 notifies the user x to select a classification of the facial part to be replaced. The control device 110 displays the information, the selection information, and the like on the display unit 10 via the display control unit 115. The facial part acquisition unit 63 determines whether or not the classification of the facial part to be replaced has been selected. More specifically, the face part acquisition unit 63 determines whether the control device 1 A classification of a face part to be replaced is selected from the operation unit 103 via the operation information acquisition unit 113 of 10. If the classification of the face part to be replaced is not selected, If so (step ST403; No), the facial part acquisition unit 63 determines whether the classification of the facial part to be replaced has been selected. Step ST403 is repeated at a predetermined interval until the face part to be replaced is selected. If the category of "s" is selected (step ST403; Yes), the face part acquisition unit 63 Proceed to step ST404.
[0166] The face part acquisition unit 63 acquires the face part from the memory unit 104 via the memory control unit 114 of the control device 110. In step ST403, the avatar face part images of the face part classification selected are randomly selected. The face part acquisition unit 63 acquires a plurality of face parts via the display control unit 115 of the control device 110. The acquired avatar facial part images are displayed on display unit 105 (step ST404). A face image and a plurality of avatar face part images are displayed on the display unit 105. The generation device 60 proceeds to step ST405.
[0167] The facial part replacement unit 64 prompts the user to change the facial part of the face to be replaced while displaying the face image being replaced on the display unit 105. The replacement operation is accepted by inputting x. The avatar face part image to be replaced is The facial part replacement unit 64 selects an avatar facial part image to be replaced. More specifically, the facial part replacement unit 6 4 is a replacement avatar from the operation unit 103 via the operation information acquisition unit 113 of the control device 110. The system determines whether an operation to select an avatar face part image to be replaced has been accepted. If no character image is selected (step ST405; No), the facial part replacement unit 64 Step ST405 is repeated at every predetermined cycle. If an image is selected (step ST405; Yes), the facial parts replacement unit 64 performs step Transition to ST406.
[0168] The facial part replacement unit 64 replaces the facial part image of the face image with the replacement avatar facial part image. The facial part replacement unit 64 replaces the facial part distance information of the facial distance information to generate a deformed facial image. The facial part distance information is replaced with the replacement avatar facial part distance information to generate transformed facial distance information. The conversion unit 64 displays the face part image on the display unit 105 via the display control unit 115 of the control device 110. The transformed face image in which the original face image is replaced with the avatar face part image is displayed (step ST406). Steps ST405 and ST406 are performed for all the facial parts to be replaced. The process may be repeated until the avatar face part image is replaced, or step ST405 In step ST406, all the facial parts to be replaced are added to the avatar facial part image In these cases, the process returns to step ST403 and the facial part classification is Even if the process from selection to replacement with the avatar face part image in step ST406 is repeated, The facial parts replacement section 64 proceeds to step ST407.
[0169] The facial part replacement unit 64 determines the facial type of the transformed facial image after replacement (step More specifically, the facial part replacement unit 64 receives the transformed face image and the transformed face distance information. The facial type determination unit 62 outputs the information to the facial type determination unit 62. Based on the deformed face distance information, a face type is determined for the deformed face image after replacement. The face type determination unit 62 outputs the determination result to the facial part replacement unit 64. The method of determining the face type Ti of the face image of the user x is the same as that of the face type Ti of the user x, so the description will be omitted. The facial parts replacement section 64 proceeds to step ST408.
[0170] The facial part replacement unit 64 determines whether the face type determined from the transformed face image after replacement is the face type of the user x. It is determined whether or not the face type is the same as the face type Ti determined from the image (step ST408). If the face type of the face image is the same as the face type Ti of the user x (step ST408; If yes, the facial parts replacing unit 64 proceeds to step ST409. If the face type is not the same as the face type Ti of the user x (step ST408; No), The .vertline. substitution unit 64 proceeds to step ST410.
[0171] The facial part replacement unit 64 transmits the facial part to the display unit 105 via the display control unit 115 of the control device 110. The confirmation button is displayed when the user x presses the operation unit 10 (step ST409). 3. By performing a predetermined operation, it is confirmed that the transformed face image after replacement is used as an avatar image. The control device 110 outputs an operation signal for setting the face part to the operation information acquisition unit 113. The .vertline. replacement unit 64 proceeds to step ST411.
[0172] The facial part replacement unit 64 transmits the facial part to the display unit 105 via the display control unit 115 of the control device 110. Predetermined warning information expressed by characters, symbols, colors, etc. is displayed (step ST41 0). The avatar generation device 60 proceeds to step ST403.
[0173] The facial part replacement unit 64 determines whether or not to confirm the use of the transformed facial image in the avatar image. More specifically, the facial part replacement unit 64 determines whether the control device 1 The transformed face image after replacement is input to the operation unit 103 via the operation information acquisition unit 113 of the avatar It is determined whether an operation to confirm that the deformed face image is to be used as an avatar is received. If it is not confirmed that the image will be used (step ST411; No), the avatar generation device The apparatus 60 proceeds to step ST403. When the face parts replacement unit 64 determines whether the face parts are to be replaced (step ST411; Yes), the face parts replacement unit 64 determines whether the face parts are to be replaced (step ST412; Yes), The flow chart processing ends.
[0174] As described above, the avatar generation device 60 of the sixth embodiment generates a user avatar based on the face image of the user x. An avatar in which each facial feature is replaced with the facial features of another avatar at the discretion of user x. This allows the user x to deform the facial features of the target face by himself / herself. By doing this, a highly convincing avatar image can be generated. In this case, in the example shown in FIG. By checking whether the face type of the transformed face image after the replacement is the same as the face type Ti of user x, Therefore, it is possible to generate an avatar image that resembles user x.
[0175] FIG. 22 is a flowchart showing another example of the facial part replacement process according to the sixth embodiment. In the facial part replacement process, steps ST451 to ST45 shown in FIG. The process up to step 4 is executed by the facial part acquisition unit 63. In the facial part replacement process, The process from step ST455 to step ST456 shown in FIG. 22 is carried out by the facial part replacement unit The facial part acquisition unit 63 executes, for example, step ST62 of FIG. When the processing of the face type determination unit 62 shown in FIG. 22 is completed, step ST45 shown in FIG. In more detail, the process of the face type determination unit 62 is performed based on the setting information and , face image and face distance information, face type Ti, and judgment items used to judge face type Ti The process ends by outputting the judgment result of (a) to face parts acquisition unit 63.
[0176] The processes from step ST451 to step ST453 shown in FIG. 22 are the same as those shown in FIG. The process is similar to the process up to step ST401 and step ST403, so the explanation will be omitted. The facial parts obtaining section 63 proceeds to step ST454.
[0177] The face part acquisition unit 63 acquires the face part from the memory unit 104 via the memory control unit 114 of the control device 110. In step ST453, the avatar face part images of the face part classification selected are randomly selected. The facial part acquisition unit 63 acquires a plurality of avatar facial parts. Assume that the judgment result of the judgment item corresponding to user x is the same as that of user x. If the selected facial feature is the "eyes", the judgement numbers are N3, N6, N10, N11, and N12. Only the avatar face part images whose judgment result is the same as that of user x are acquired. The acquisition unit 63 displays the acquired complex on the display unit 105 via the display control unit 115 of the control device 110. The number of avatar facial part images is displayed on the display unit 105 (step ST454). The avatar generating device 60 displays a step image and a plurality of avatar face part images. Switch to step ST455.
[0178] The process of step ST455 shown in FIG. 22 is the same as the process of step ST405 shown in FIG. Since this is the same process, the description will be omitted. Migrate.
[0179] The facial part replacement unit 64 replaces the facial part image of the face image with the replacement avatar facial part image. The facial part replacement unit 64 replaces the facial part distance information of the facial distance information to generate a deformed facial image. The facial part replacement unit 6 replaces the facial part distance information of the avatar with the facial part distance information for replacement to obtain the transformed facial distance information. 4 displays the facial part image on the display unit 105 via the display control unit 115 of the control device 110 as an avatar. The transformed face image is displayed on the screen (step ST456). Step ST455 and step ST456 are carried out to generate avatars for all the facial parts to be replaced. - The process may be repeated until the face part image is replaced, or only step ST455 may be performed. Repeat this process, and in step ST456, all the facial parts to be replaced are combined into an avatar facial part image. In these cases, the process returns to step ST453 and the selection of the face part classification is completed. The process may be repeated up to step ST456 where the image is replaced with an avatar face part image. The facial parts replacement unit 64 ends the process of the flowchart shown in FIG.
[0180] As described above, in the avatar generation device 60 of the sixth embodiment, in the example shown in FIG. Then, the replacement avatar face part image is made to be the same as the judgment result of the corresponding face part of user x. After limiting the scope of the transformation to the facial features of the user x, the user x transforms the facial features to be transformed. It is possible to generate an avatar image that resembles User X and is highly convincing. Seventh embodiment An avatar generation device 70 according to a seventh embodiment will be described with reference to FIGS. The avatar generating device 70 according to the seventh embodiment generates a facial profile based on a captured image of the user x. For each character, an avatar is generated by replacing the facial features of the celebrity avatar. FIG. 1 is a block diagram showing an example of a configuration of an avatar generation device 70 according to an embodiment. The device 70 includes a face image acquisition unit 71, a face type determination unit 72, an avatar acquisition unit 73, and a face type determination unit 74. The face image acquisition unit 71 and the whole body image synthesis unit 75 are included. The configuration and operation of the image synthesis unit 75 are the same as those of the face image acquisition unit 61 and the whole body image synthesis unit 62 of the sixth embodiment. The configuration and operation of the face type determination unit 7 are the same as those of the face type determination unit 65, so the description thereof will be omitted. The configuration and operation of 2 are the same as those of the facial part acquisition unit 63, but the determination result is sent to the avatar acquisition unit 73. The configuration and operation of the face type determination unit 62 of the sixth embodiment are the same as those of the face type determination unit 62 of the sixth embodiment, except that the face type determination unit 62 outputs the image to the Therefore, the description will be omitted.
[0181] The avatar acquisition unit 73 receives the setting information, the face image, and the face distance information from the face type determination unit 72. The face type Ti and the judgment result for each judgment item used to judge the face type Ti are acquired. The avatar acquisition unit 73 selects an avatar face image of a celebrity having the facial features to be replaced. The display control unit 110 controls the notification information to be notified to the user x, the selection information, etc. The avatar acquisition unit 73 displays the avatar on the display unit 105 via the control device 110. Based on the input information received from the operation unit 103 via the operation information acquisition unit 113, The avatar face image is used as a replacement image, and the distance to the image capture target and The avatar face distance information including the three-dimensional coordinates of the image capture target is linked to the avatar face distance information. A database of replacement avatar face images is stored at 104 .
[0182] The database contains avatar face images and avatar face distance information of celebrities. The avatar face images stored in the database are assigned the face type Ti. The avatar face images stored in the database are judged by the judgment number Ni. The avatar face images stored in the database are sorted by face part. The database is periodically updated via a communication network. The avatar acquisition unit 73 can update the face image and the option of the acquired avatar face image. The information is displayed on the display unit 105 via the display control unit 115 of the control device 110.
[0183] The facial part replacement unit 74 receives the operation information from the operation unit 103 via the operation information acquisition unit 113 of the control device 110. Based on the input information received from the The facial part image is replaced with a celebrity avatar facial part image to generate a transformed facial part image. The conversion unit 74 receives the command from the operation unit 103 via the operation information acquisition unit 113 of the control device 110. Based on the input information, the face part distance information of the selected face part is The facial part distance information is replaced with that of the celebrity's avatar to generate transformed facial part distance information. The facial parts to be replaced by the avatar face image are selected by user x. The replacement unit 74 displays the face image being replaced on the display unit 105 and receives the input from the user x. The facial part replacement unit 74 receives a replacement operation for synthesizing the transformed facial part image with the facial image. A facial part replacement unit 74 generates a deformed facial image by matching the deformed facial part distance information to the facial distance information. The facial part replacement unit 74 generates the transformed facial distance information based on the setting information and the generated transformed facial image. The avatar acquisition unit 73 and the transformed face distance information are output to the whole-body image synthesis unit 75. The face type Ti of the transformed face image generated by the face part replacement unit 74 is the face type of the user x. It is assumed that the face type is restricted so as to be the same as the face type Ti in the image.
[0184] FIG. 24 is a flowchart showing an example of the process of the avatar generation device 70 according to the seventh embodiment. The avatar generating device 70 receives operation information via the operation information acquiring unit 113 of the control device 110. When the operation unit 103 receives a predetermined start operation, step ST71 shown in FIG. and start processing.
[0185] The avatar generating device 70 acquires a face image from a captured image (step ST71). Specifically, the avatar generating device 70 first inputs a facial image captured by the imaging unit 101 to the face image acquisition unit 71. The avatar generation device 70 acquires a whole-body image and whole-body distance information of the user x. Next, the face image acquisition unit 71 acquires the face part from the whole body image and the whole body distance information. The avatar generation device 70 extracts the face image and face distance information of the user x and generates the face image and face distance information of the user x. Proceed to step ST72.
[0186] The avatar generating device 70 determines the face type Ti from the face image (step ST72). More specifically, the avatar generating device 70 first inputs a face image and Based on the face distance information, the judgement is made as A, B, C or the like for each judgement item corresponding to the judgement number Ni. The avatar generating device 70 then causes the face type determining unit 7 2 determines the face type Ti based on the determination results for each determination item. 70 proceeds to step ST73.
[0187] The avatar generation device 70 acquires the selected avatar face image (step ST73). More specifically, the avatar generating device 70 first transmits to the avatar acquiring unit 73 the control device 1 10 via the operation information acquisition unit 113, the avatar face image of the celebrity having the facial part to be replaced is input. The input operation by the user x to select an avatar is accepted by the operation unit 103. The unit 73 prompts the user x to select an avatar face image of a celebrity having a facial part to be replaced. Notification information to be notified, option information, etc. are displayed via the display control unit 115 of the control device 110. The avatar generation device 70 then causes the avatar acquisition unit 73 to display the control Based on input information received from the operation unit 103 via the operation information acquisition unit 113 of the control device 110, The avatar generating device 70 acquires the selected replacement avatar face image based on the selected face image. Proceed to step ST74.
[0188] The avatar generating device 70 generates the facial part image of the face image for the selected facial part as the avatar. The target face image is replaced with an avatar face part image (step ST74). The avatar generating device 70 first transmits to the face part replacing unit 74 an operation information acquisition signal for the control device 110. A user x selects a face part to be replaced through the operation unit 113 and inputs the selected face part to the operation unit 103. The avatar generating device 70 then causes the facial part replacement unit 74 to receive the image. Based on input information received from the operation unit 103 via the operation information acquisition unit 113 of 110 For the selected facial part, the facial part image of the face image is converted to a celebrity avatar facial part image. The avatar generating device 70 also generates a deformed facial part image by replacing the facial part with the The control unit 110 receives the operation information from the operation unit 103 via the operation information acquisition unit 113 and converts it to the conversion unit 74. Based on the input information, the face part distance information of the selected face part is The facial part distance information of the celebrity's avatar is replaced with the facial part distance information of the celebrity's avatar to generate the transformed facial part distance information. The target generating device 70 then controls the facial part replacing unit 74 to synthesize the transformed facial part image with the face image. The avatar generating device 70 also controls the facial part replacing unit 74 to generate a deformed facial image. The facial feature distance information is combined with the face distance information to generate the deformed face distance information. The apparatus 70 proceeds to step ST75.
[0189] The avatar generation device 70 generates an avatar image by synthesizing the transformed face image with the captured image ( More specifically, the avatar generating device 70 causes the whole-body image synthesis unit 75 to The deformed face image is then combined with a whole-body captured image to generate an avatar image, which is a deformed whole-body image. The avatar generating device 70 also outputs the transformed face distance information to the whole body image synthesis unit 75 as the whole body distance information. The avatar generating device 7 generates avatar distance information, which is transformed whole-body distance information, by synthesizing the avatar distance information with the avatar image. 0 ends the processing of the flowchart shown in FIG.
[0190] FIG. 25 is a flowchart showing an example of the facial parts replacement process according to the seventh embodiment. In the facial part replacement process, steps ST501 to ST504 shown in FIG. The process in FIG. 2 is executed by the avatar acquisition unit 73. The processes from step ST505 to step ST511 shown in FIG. 2 are performed by the facial part replacement unit 74. The avatar acquisition unit 73 executes, for example, the process shown in step ST72 of FIG. When the processing of the face type determination unit 72 is completed, the process proceeds to step ST501 shown in FIG. In more detail, the face type determination unit 72 performs the process by determining the face type based on the setting information and the face type. Image and face distance information, face type Ti, and judgment results for each judgment item used to judge face type Ti The process ends by outputting the determination result to the avatar acquisition unit 73.
[0191] The avatar acquisition unit 73 transmits the avatar to the display unit 105 via the display control unit 115 of the control device 110. The face image of the user x is displayed (step ST501). The process proceeds to ST502.
[0192] The avatar acquisition unit 73 acquires the face type Ti of the user x determined by the face type determination unit 62. In the sixth embodiment, the face type Ti is the face type shown in Table 1. The type is T1, T2, T3, T4, T5, T6, T7, or T8. Get Avatar The unit 73 proceeds to step ST503.
[0193] The avatar acquisition unit 73 selects an avatar face image of a celebrity having the facial features to be replaced. The display control unit 110 controls the notification information to be notified to the user x, the selection information, etc. The avatar acquisition unit 73 displays the replacement face part on the display unit 105 via the display unit 115. It is determined whether or not an avatar face image of a celebrity that has the camera is selected (step ST503). More specifically, the avatar acquisition unit 73 receives the operation information via the operation information acquisition unit 113 of the control device 110. and selecting an avatar face image of a celebrity having a face part to be replaced from the operation unit 103. A celebrity avatar face image having the facial part to be replaced is selected. If no face has been selected (step ST503; No), the avatar acquisition unit 73 Step ST is repeated every predetermined period until an avatar face image of a celebrity having the part is selected. 503 is repeatedly executed. An avatar face image of a celebrity having a face part to be replaced is selected. If so (step ST503; Yes), the avatar obtaining unit 73 performs the process in step ST504. transition to.
[0194] The avatar acquisition unit 73 transmits the avatar to the display unit 105 via the display control unit 115 of the control device 110. The selected avatar face image is displayed (step ST504). The image and the selected avatar face image are displayed. The avatar generation device 70 performs the steps Then proceed to ST505.
[0195] The facial part replacement unit 74 prompts the user to change the facial part of the face to be replaced while displaying the face image being replaced on the display unit 105. Accepts replacement operations by inputting x. Facial parts to be replaced with the facial image and avatar facial image is selected by the user x. The facial part replacement unit 74 receives the selected facial part to be replaced. More specifically, the facial parts replacement unit 74 determines whether or not the facial parts have been replaced (step ST505). A face part to be replaced is input from the operation unit 103 via the operation information acquisition unit 113 of the control device 110. It is determined whether a selection operation has been accepted. If no face part to be replaced has been selected (Step ST505; No), the facial part replacement unit 74 performs step ST5 at predetermined intervals. 05 is repeatedly executed. When a face part to be replaced is selected (step ST505; es), the facial parts replacement unit 74 proceeds to step ST506.
[0196] The facial part replacement unit 74 replaces the facial part image of the face image with the avatar for the selected facial part. The facial part replacement unit 7 replaces the target facial image with an avatar facial part image to generate a transformed facial image. 4. For the selected face part, the face part distance information of the face distance information is compared with the avatar face image The facial part replacement unit 74 replaces the facial part distance information of the avatar with the facial part distance information of the facial part replacement unit 74 to generate transformed facial distance information. The face part image is displayed on the display unit 105 via the display control unit 115 of the control device 110 as an avatar. The transformed face image with the replaced face part image is displayed (step ST506). Step ST505 and step ST506 are carried out to replace all the facial parts of the avatar. The process may be repeated until the face part image is replaced, or only step ST505 may be repeated. Then, in step ST506, all the facial parts to be replaced are compiled into an avatar facial part image. The facial parts replacing section 74 proceeds to step ST507.
[0197] The facial part replacement unit 74 determines the facial type of the transformed facial image after replacement (step More specifically, the facial part replacement unit 74 receives the transformed face image and the transformed face distance information. The facial type determination unit 72 outputs the information to the facial type determination unit 72. Based on the deformed face distance information, a face type is determined for the deformed face image after replacement. The face type determination unit 72 outputs the determination result to the facial part replacement unit 74. The method of determining the face type Ti of the face image of the user x is the same as that of the face type Ti of the user x, so the description will be omitted. The facial parts replacement section 74 proceeds to step ST508.
[0198] The facial part replacement unit 74 determines whether the face type determined from the transformed face image after replacement is the face type of the user x. It is judged whether or not the face type is the same as the face type Ti judged from the image (step ST508). If the face type of the face image is the same as the face type Ti of the user x (step ST508; If yes, the facial parts replacing unit 74 proceeds to step ST509. If the face type is not the same as the face type Ti of the user x (step ST508; No), The .vertline.sub.2 replacing unit 74 proceeds to step ST510.
[0199] The facial part replacement unit 74 transmits the facial part replacement image to the display unit 105 via the display control unit 115 of the control device 110. The confirmation button is displayed when the user x presses the operation unit 10 (step ST509). 3. By performing a predetermined operation, it is confirmed that the transformed face image after replacement is used as an avatar image. The control device 110 outputs an operation signal for setting the face part to the operation information acquisition unit 113. The .vertline.sub.2 replacing unit 74 proceeds to step ST511.
[0200] The facial part replacement unit 74 transmits the facial part replacement image to the display unit 105 via the display control unit 115 of the control device 110. Predetermined warning information expressed by characters, symbols, colors, etc. is displayed (step ST51 0). The avatar generation device 60 proceeds to step ST503.
[0201] The facial part replacement unit 74 determines whether or not to confirm the use of the transformed facial image in the avatar image. More specifically, the facial part replacement unit 74 determines whether the control device 1 The transformed face image after replacement is input to the operation unit 103 via the operation information acquisition unit 113 of the avatar It is determined whether an operation to confirm that the deformed face image is to be used as an avatar is received. If it is not confirmed that the image will be used as the avatar image (step ST511; No), The apparatus 60 proceeds to step ST503. When the face parts replacement unit 74 determines whether the face parts are to be replaced (step ST511; Yes), the face parts replacement unit 74 determines whether the face parts are to be replaced (step ST512; Yes), The flow chart processing ends.
[0202] As described above, the avatar generation device 70 of the seventh embodiment generates a user avatar based on the face image of the user x. An avatar in which each facial feature is replaced with the facial features of another avatar at the discretion of user x. This allows the user x to deform the facial features of the target face by himself / herself. By doing this, a highly convincing avatar image can be generated. In this case, in the example shown in FIG. By checking whether the face type of the transformed face image after the replacement is the same as the face type Ti of user x, Therefore, it is possible to generate an avatar image that resembles user x. (Modification of the fourth embodiment) An avatar generation device 40A according to a modification of the fourth embodiment will be described with reference to FIG. The avatar generation device 40A according to the modified example generates a predetermined transformation based on a captured image of the user x himself / herself. An avatar is generated by automatically transforming the selected facial and body parts using a shape method. FIG. 26 is a block diagram showing an example of the configuration of an avatar generation device 40A according to a modification of the fourth embodiment. Avatar generation device 40A is a block diagram of an avatar generation device 40 according to a fourth embodiment. A body image acquisition unit 46, a skeletal type determination unit 47, and a body part transformation setting unit 48, The avatar generating device 40A differs from the avatar generating device 40A in that it further includes a body part transforming unit 49. In comparison with the avatar generating device 40 of the fourth embodiment, instead of the whole body image synthesis unit 45, 13. It differs in that it includes a body image synthesis unit 45A.
[0203] The body image acquisition unit 46 stores the body image in the memory unit 104 via the memory control unit 114 of the control device 110. The setting information including the whole body image and whole body distance information of the user x stored in the database is acquired. In the description, the whole-body captured image and whole-body distance information acquired from the storage unit 104 are taken from the front. The captured image and the distance information from the front of the whole body of the user x are taken as the body image acquisition unit 4. 6 extracts body parts from the whole-body captured image and the whole-body distance information, and calculates the body part of user x. The body image acquisition unit 46 generates a body image and body distance information. The body image and the body distance information are output to the skeletal type determination unit 47 .
[0204] The skeletal type determination unit 47 receives the setting information, the body image, and the body distance from the body image acquisition unit 46. Based on the body image and the body distance information, the skeletal type determination unit 47 determines Determine the skeletal type of user x. Skeletal type is a set of body parts that indicate the body and each part of the body. The bone structure is determined by assessing the size, position and shape of each of the bones. The determination unit 47 determines a plurality of training data images as a learning image data set, and The training data images are, for example, 10 for each skeletal type. The learning method of the inference machine is, for example, machine learning such as deep learning. .
[0205] The body part transformation setting unit 48 receives the setting information, the body image, and the The body distance information, the skeletal type, and the judgment result for each judgment item used to judge the skeletal type are The body part transformation setting unit 48 acquires the skeleton type from the body image based on the skeleton type. The body parts that do not affect the result of the group are set as the body parts to be transformed. The part transformation setting unit 48 generates a body parts list including the body parts to be transformed. The body part transformation setting unit 48 determines the skeletal type of the user x and the body parts to be transformed. Based on the result of the determination, a transformation method is set. The transformation method corresponds to the body part to be transformed. The body part transformation setting unit 48 sets the setting information , body image, body distance information, generated body parts list and transformation method are The signal is output to unit 49.
[0206] The body part transformation unit 49 receives the setting information, the body image, the body The body parts transformation unit 49 acquires the distance information, the generated body parts list, and the transformation method. Extracts the body part images to be transformed that are registered in the body parts list from the body image. The body part transformation unit 49 transforms the body part image to be transformed based on the set transformation method. The body part transformation unit 49 generates a transformed body part image by transforming the body part from the body distance information. Extract the distance information of the body parts to be transformed from the body parts list. The body part transformation unit 49 transforms the body part distance information of the transformation target based on the set transformation method. The body part transformation unit 49 generates transformed body part distance information. The body part transformation unit 49 combines the body part transformation unit 49 with the body image to generate a transformed body image. The distance information is combined with the body distance information to generate transformed body distance information. The setting information, the generated deformed body image, and the deformed body distance information are sent to the whole body image synthesis unit 45A. Output.
[0207] The whole body image synthesis unit 45A receives the setting information, the transformed face image, and the transformed face image from the face part transformation unit 44. The whole body image synthesis unit 45A acquires face distance information from the body part transformation unit 49. The whole-body image synthesis unit 45A acquires a deformed face image, a deformed body image, and deformed body distance information. The body image and the deformed body image are synthesized with the whole-body image to generate an avatar image, which is a deformed whole-body image. The whole-body image synthesis unit 45A converts the deformed face distance information and the deformed body distance information into whole-body distance information. The whole-body image synthesis unit 45A synthesizes the whole-body image to generate avatar distance information, which is transformed whole-body distance information. The setting information, the avatar image, and the avatar distance information are transmitted to the information acquisition unit 11 of the control device 110. Output to 8.
[0208] As described above, the avatar generation device 40A according to the modification of the fourth embodiment can generate not only face images but also It is also possible to generate an avatar image with a modified body image. This makes it possible to further improve the confidentiality of personal information of user x using an avatar image. In the above, the configuration example in which an avatar image in which a body image is also transformed is generated is described as the modified example of the fourth embodiment. Although this has been described as an example, the avatar image in which the body image is also modified may be applied to other embodiments. stomach.
[0209] The present invention is not limited to the above-mentioned embodiments. For example, the face type may be changed in accordance with the above-mentioned embodiments. In this mode, the face is identified based on the size, position, and shape of the face and the facial features that represent each part of the face. The face types are classified into eight types according to the face type, but the present invention is not limited to these. , skin color or eye color, etc.
[0210] The avatar generation devices 10, 20, 30, 40, 50, 60, and 70 shown in the figure, and the mobile terminal 1 The components of the 00 and the clothing matching system 1 are functional concepts and are not necessarily The avatar generating devices 10 and 20 do not have to be physically configured as shown in the figures. , 30, 40, 50, 60, 70, mobile terminal 100 and clothing matching system 1 The specific form is not limited to the one shown in the figure, and may be changed in whole or in part depending on the processing load, usage status, etc. Parts may be functionally or physically distributed or integrated in any unit.
[0211] Avatar generation devices 10, 20, 30, 40, 50, 60, 70 and mobile terminal 100 The configuration of the above may be implemented, for example, as software by a program loaded into memory. In the above embodiment, the above hardware and software are implemented in cooperation with each other. The functional blocks are described as follows: In various forms, using only hardware, only software, or a combination of both It can be achieved. [Explanation of symbols]
[0212] 1. Clothing Matching System 10, 20, 30, 40, 40A, 50, 60, 70 Avatar Generator 11, 21, 31, 41, 51, 61, 71 Face image acquisition unit 12, 22, 32, 42, 52, 62, 72 Face type determination section 13, 24, 34 Transformation policy setting section 14, 25, 35 Facial deformity 15, 26, 36, 45, 45A, 55, 65, 75 Whole body image synthesis section 23 Similar Preference Face Image Collection Department 33 Favorite Face Image Collection Department 43, 53 Face part transformation setting section 44 Facial part transformation section 46 Body Image Acquisition Department 47 Bone Type Determination Section 48 Body Parts Transformation Setting Section 49 Body Parts Transformation 54 Facial part deformation adjustment section 63 Face Parts Acquisition Department 64, 74 Face Part Replacement 73 Avatar Acquisition Department 100, 100X, 100Y Mobile Terminals 120 Clothes Synthesizer 150 App Window 160 Avatar image display area 170 Parameter display area 171 Item name display area 172 Parameter Bar 173 Upper limit display area 174 Lower limit display area 175 Adjustment knob 176 Plus button 177 Minus button 178 Transformation setting value display area 179 Untransformable Display Area 180 Confirm button x,y user X, Y Location Da Avatar Data Dd Clothing Data
Claims
1. a face image acquisition unit for generating a face image of the first user; a face type determination unit that determines whether the face image corresponds to any one of a plurality of face types that are previously classified; a transformation policy setting unit that sets an average face image obtained by averaging face images of a predetermined number of people based on predetermined conditions; a face transformation unit that generates a transformed face image by weighting the face image and the average face image at a predetermined ratio and synthesizing the weighted face image and the average face image; a whole-body image synthesis unit that synthesizes the deformed face image with a whole-body captured image of the first user to generate an avatar image that can be synthesized with a clothing image obtained by capturing an image of clothing accessories; An avatar generation device comprising:
2. the face transformation unit extracts face parts for each predetermined judgment item from the average face image, and replaces at least one of the face parts for each predetermined judgment item of the face image with the face part extracted from the average face image to generate the transformed face image. The avatar generating device according to claim 1 .
3. the face transformation unit assigns a priority to a face feature for each predetermined judgment item; A facial part to be replaced is selected based on the priority. The avatar generating device according to claim 2 .
4. The priority is set so that the lower the correlation between the facial part of the first user and the facial part of the average face image, the higher the priority is set. The avatar generating device according to claim 3 .
5. a face image acquisition unit for generating a face image of the first user; a transformation policy setting unit that sets an average face image obtained by averaging face images of a predetermined number of people based on predetermined conditions; a similar preference face image collecting unit that collects similar preference face images, which are face images of a predetermined number of people who have similar preferences for clothing and accessories to the first user; a face transformation unit that generates a transformed face image by weighting the face image and the average face image at a predetermined ratio and synthesizing the weighted face image and the average face image; a whole-body image synthesis unit that synthesizes the deformed face image with a whole-body captured image of the first user to generate an avatar image that can be synthesized with a clothing image obtained by capturing an image of clothing accessories; Equipped with The predetermined condition includes a similarity in taste of clothing accessories to that of the first user. Avatar generator.
6. a face image acquisition unit for generating a face image of the first user; a transformation policy setting unit that sets an average face image obtained by averaging face images of a predetermined number of people based on predetermined conditions; a favorite face image collection unit that collects favorite face images, which are face images of a predetermined number of celebrities having favorite faces of the first user; a face transformation unit that generates a transformed face image by weighting the face image and the average face image at a predetermined ratio and synthesizing the weighted face image and the average face image; a whole-body image synthesis unit that synthesizes the deformed face image with a whole-body captured image of the first user to generate an avatar image that can be synthesized with a clothing image obtained by capturing an image of clothing accessories; Equipped with The predetermined condition includes having a favorite face of the first user. Avatar generator.
7. The avatar image is acquired from an avatar generation device according to any one of claims 1 to 6, A clothing synthesis device generates a clothing-synthesized avatar image by synthesizing the avatar image with a clothing image obtained by capturing clothing accessories.
8. Determining whether the face image of the first user is one of a plurality of face types classified in advance; An average face image is set by averaging face images of a predetermined number of people based on a predetermined condition; generating a modified face image by weighting the face image of the first user and the average face image at a predetermined ratio and synthesizing the weighted face image and the average face image; obtaining an avatar image by combining the deformed facial image with a whole-body image of the first user; A clothing synthesis device generates a clothing-synthesized avatar image by synthesizing the avatar image with a clothing image obtained by capturing clothing accessories.
9. Transmitting the avatar image to another device; receiving a clothing image obtained by capturing an image of a clothing item from the other device; synthesizing the clothing image with the whole-body image; The avatar generation device according to any one of claims 1 to 8.
10. A computer that operates as an avatar generation device includes: a facial image acquisition step of generating a facial image of a first user; a face type determination step of determining whether the face image corresponds to any one of a plurality of face types classified in advance; a transformation policy setting step of setting an average face image obtained by averaging face images of a predetermined number of people based on a predetermined condition; a face transformation step of weighting the face image and the average face image at a predetermined ratio and synthesizing the weighted face image and the average face image to generate a transformed face image; a whole-body image synthesis step of synthesizing the deformed face image with a whole-body captured image of the first user to generate an avatar image that can be synthesized with a clothing image obtained by capturing an image of clothing accessories; A program that executes the following.
11. A computer that operates as a clothing synthesis device is Determining whether the face image of the first user is one of a plurality of face types classified in advance; An average face image is set by averaging face images of a predetermined number of people based on a predetermined condition; generating a modified face image by weighting the face image of the first user and the average face image at a predetermined ratio and synthesizing the weighted face image and the average face image; an avatar image acquisition step of acquiring an avatar image generated by combining the deformed facial image with a whole-body image of the first user; a clothing synthesis step of generating a clothing-composite avatar image by synthesizing the avatar image with a clothing image obtained by capturing clothing accessories; A program that executes the following.
Citation Information
Patent Citations
Program, information processor, information processing system and method for generating avatar image
JP2016136324A
Information processing device, information processing method, and program storage medium
WO2012161347A1
Generation of avatar reflecting player appearance
WO2012170354A2
