Image processing method and device, chip system, storage medium and program product

By using image processing methods to identify human body characteristics and gender in electronic devices, we generate wear recommendations that meet the user's physical appearance characteristics, solving the problem of cumbersome and inaccurate dressing recommendation process in the existing technology, and improving user experience and Recommended effect.

CN119091232AActive Publication Date: 2024-12-06HONOR DEVICE CO LTD

Patent Information

Application Number
CN202411570931.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-06
Publication Date
2024-12-06
Estimated Expiration
2044-11-06

Smart Images

  • Figure CN119091232A_ABST
    Figure CN119091232A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an image processing method and device, a chip system, a storage medium and a program product, and relates to the technical field of terminals. The method comprises the following steps: identifying position information of feature points such as a forehead, cheeks and a jaw of a human body based on a neural network model, and further determining figure proportions such as facial form, height and weight of a person; and skin color, gender, hairstyle and other information of the human body can be identified based on a neural network model. In this way, proper clothes can be recommended to the user according to the recognized character information, the clothes recommendation accuracy is improved, and therefore the user experience is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to image processing methods, devices, chip systems, storage media and program products. Background Art

[0002] In order to improve user experience, some electronic devices can provide outfit recommendation functions.

[0003] Currently, users can manually input desired clothing information, physical features, or personalized needs, etc. into electronic devices. After receiving the information input by the user, the electronic device can obtain corresponding outfit recommendations based on the information.

[0004] However, the process of users manually inputting clothing information, physical characteristics and personalized needs may be cumbersome and time-consuming, affecting the user experience. Summary of the invention

[0005] The embodiments of the present application provide an image processing method, device, chip system, storage medium and program product, which are applied to the field of terminal technology and can enable electronic devices to recommend suitable outfits to users based on recognized character information, thereby improving the accuracy of outfit recommendations and thus improving user experience.

[0006] In a first aspect, an embodiment of the present application provides an image processing method. The method comprises: acquiring a target image;

[0007] Based on the first model, the first information of the target image is obtained; the first model is used to obtain the position information of the human body parts in the image input to the first model, and the first information includes the position information of the human body parts in the target image; based on the second model, the second information of the target image is obtained; the second model is used to obtain the gender information, skin color information and / or hairstyle information of the person in the image input to the second model, and the second information includes the gender information, skin color information and / or hairstyle information of the person in the target image; based on the target image, the first information, the second information and the third model, the dressing image of the person in the target image is obtained; the third model is used to obtain the physical features of the person in the image input to the third model based on the output information of the first model and the output information of the second model, and determine the dressing image of the person based on the physical features of the person. The electronic device can obtain the physical features of the person based on the target image input by the user, as well as the first model, the second model and the third model, and can also generate a dressing picture that meets the physical features of the person in the target image based on the third model. In this way, users can more intuitively show the actual effect of the dressing recommendation on themselves, improve the user's visual experience, and improve user satisfaction.

[0008] In combination with the first aspect, in some implementations of the first aspect, obtaining first information of a target image based on a first model includes: inputting the target image into the first model; the first model includes one or more first fusion convolution modules; based on the one or more first fusion convolution modules, extracting features of a person in the target image to obtain a first feature image output by the first target fusion convolution module; the first target fusion convolution module is any one of the one or more first fusion convolution modules; wherein the size of the first feature image output by the first target fusion convolution module is smaller than the size of the feature image input to the first target fusion convolution module, and the number of channels of the first feature image output by the first target fusion convolution module is greater than the number of channels of the feature image input to the first target fusion convolution module; performing convolution operations, pooling operations, and full connection operations on the first feature images output by the one or more first fusion convolution modules to obtain a second feature image; the second feature image includes the first information. After the target image is input into the first model, the first information of the target image can be obtained, that is, the position information of each body part of the person in the target image, which is the basis for the subsequent calculation of the face shape, body proportion, fatness, and other physical features of the person based on the position information of the body parts.

[0009] In combination with the first aspect, in some implementations of the first aspect, obtaining the second information of the target image based on the second model includes: inputting the target image into the second model; the second model includes one or more second fused convolution modules; based on the one or more second fused convolution modules, extracting features of the person in the target image to obtain a third feature image output by the second target fused convolution module; the second target fused convolution module is any one of the one or more second fused convolution modules; performing convolution operations, pooling operations and full connection operations on the third feature image output by the one or more second fused convolution modules to obtain a fourth feature image; the fourth feature image includes the second information; wherein the full connection operation includes a first fully connected layer, a second fully connected layer and a third fully connected layer, the first fully connected layer is used to output information related to the gender of the person in the target image; the second fully connected layer is used to output information related to the skin color of the person in the target image; the third fully connected layer is used to output information related to the hairstyle of the person in the target image. Based on the second model, the second information of the person in the target image, that is, the gender information, skin color information and hairstyle information of the person can be identified. In this way, the accuracy of electronic devices in obtaining a person's physical characteristics such as gender, skin color and hairstyle can be improved, and the clothing images output by the third model can be made more consistent with the person's physical characteristics, thereby improving user satisfaction.

[0010] In combination with the first aspect, in certain implementations of the first aspect, based on the target image, the first information, the second information and the third model, obtaining an outfit image of the person in the target image includes: obtaining facial information, body proportion information and / or fat information of the person in the target image based on the first information; inputting the target image, the facial information, body proportion information and / or fat information of the person in the target image, and the gender information, skin color information and / or hairstyle information of the person in the target image into the third model; obtaining an outfit image of the person in the target image based on the third model. The outfit in the outfit image of the person in the target image obtained based on the third model can be an outfit that highly conforms to the physical features of the person. In this way, the results of outfit recommendations can be made more accurate and user satisfaction can be improved.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the first information includes, in the target image, coordinate information of the person's forehead in the target image, coordinate information of the person's chin in the target image, coordinate information of both sides of the person's cheeks in the target image, coordinate information of both sides of the person's shoulders in the target image, coordinate information of both sides of the person's waist in the target image, coordinate information of both sides of the person's knees in the target image, and coordinate information of both sides of the person's ankles in the target image. By obtaining more detailed and accurate coordinate information of each body part in the target image, it is helpful to analyze the face shape, body proportions, fatness and thinness and other physical features of the person in the target image. It also helps to improve the accuracy of outfit recommendations and improve user experience.

[0012] In combination with the first aspect, in some implementations of the first aspect, the face shape information, body proportion information and / or fatness information of the person in the target image are obtained based on the first information, including: determining the face length information of the person based on the coordinate information of the forehead of the person in the target image and the coordinate information of the chin of the person in the target image; determining the face width information of the person based on the coordinate information of the two sides of the cheeks of the person in the target image; calculating the ratio of the face length information of the person and the face width information of the person to determine the face shape information of the person in the target image; determining the height proportion information of the person in the target image based on the coordinate information of the two sides of the shoulders of the person in the target image and the coordinate information of the two sides of the ankles of the person in the target image; determining the fatness proportion information of the person in the target image based on the coordinate information of the two sides of the waist of the person in the target image and the coordinate information of the two sides of the knees of the person in the target image; calculating the ratio of the height proportion information of the person and the fatness proportion information of the person to determine the body proportion information of the person in the target image. Based on the coordinate information of each body part of the person, the physical features such as the face shape information and the body proportion information of the person can be determined. In this way, the dressing images generated by the electronic device can be more in line with the facial shape and body proportions of the person, thereby improving the accuracy of the dressing images recommended to users and enhancing the user experience.

[0013] In a second aspect, an embodiment of the present application provides an image processing device, which may be an electronic device, or a chip or chip system in an electronic device. The image processing device may include a processing unit. The processing unit may be a processor. The device may also include a storage unit, which may be a memory. The storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. When the device is a chip or chip system in an electronic device, the processing unit may be a processor. The processing unit executes the instructions stored in the storage unit so that the electronic device implements an image processing method described in the first aspect or any possible implementation of the first aspect. The storage unit may be a storage unit in the chip (for example, a register, a cache, etc.), or a storage unit in the electronic device located outside the chip (for example, a read-only memory, a random access memory, etc.).

[0014] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, the memory being used to store code instructions, and the processor being used to run the code instructions to execute the method described in the first aspect or any possible implementation of the first aspect.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instructions are stored. When the computer program or instructions are run on a computer, the computer executes the method described in the first aspect or any possible implementation of the first aspect.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product including a computer program. When the computer program runs on a computer, the computer executes the method described in the first aspect or any possible implementation manner of the first aspect.

[0017] In a sixth aspect, the present application provides a chip or a chip system, the chip or chip system comprising at least one processor and a communication interface, the communication interface and the at least one processor are interconnected by a line, and the at least one processor is used to run a computer program or instruction to execute the method described in the first aspect or any possible implementation of the first aspect. The communication interface in the chip can be an input / output interface, a pin or a circuit, etc.

[0018] In a possible implementation, the chip or chip system described above in the present application further includes at least one memory, in which instructions are stored. The memory may be a storage unit inside the chip, such as a register, a cache, etc., or a storage unit of the chip (such as a read-only memory, a random access memory, etc.).

[0019] It should be understood that the second to sixth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0022] Figure 3 A schematic diagram of the overall structure of an outfit recommendation network model provided in an embodiment of the application;

[0023] Figure 4 A schematic diagram of a human body feature model provided in an embodiment of the present application;

[0024] Figure 5 A schematic diagram of the structure of a feature point recognition network model provided in an embodiment of the present application;

[0025] Figure 6 A schematic diagram of the structure of a fused convolution module provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of a multi-classification network model provided in an embodiment of the present application;

[0027] Figure 8 A flowchart of an image processing method provided in an embodiment of the present application;

[0028] Fig. 9 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0030] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first chip and the second chip are only used to distinguish different chips, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0031] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0032] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, and c can be single or multiple.

[0033] With the continuous development of terminal technology and artificial intelligence, users' expectations for electronic devices are also increasing. Users hope that these electronic devices can not only meet basic communication and entertainment needs, but also provide more intelligent and personalized services. For example, in order to improve user experience, some electronic devices can provide outfit recommendation functions.

[0034] In one implementation, the user may manually input desired clothing information, physical features, or personalized needs, etc., into the electronic device. For example, the clothing information may be the user's specific requirements for clothing, such as the color and style of clothing. Physical features may include, for example, the user's height and weight. Personalized needs may include, for example, the current season, photo-taking scene, etc. After receiving the information input by the user, the electronic device may match the information with the clothing information in the clothing database, thereby selecting clothing that meets the user's requirements.

[0035] However, the process of users manually inputting clothing information, physical features, and personalized needs can be cumbersome and time-consuming. For users who are not familiar with the operation, this method may increase the difficulty of use and reduce the user experience. Secondly, since the outfit recommendation results are highly dependent on the information entered by the user, if the information entered by the user is inaccurate or incomplete, such as the user enters the wrong height or weight, or enters unclear clothing requirements, the system's matching results will also be affected, resulting in inaccurate recommendation results and reduced user experience.

[0036] In order to solve the above technical problems, the embodiment of the present application provides an image processing method, which can identify the position information of the forehead, cheeks, chin and other feature points of the human body based on the neural network model, and then determine the face shape, height, fatness and other body proportions of the person; it can also identify the skin color, gender, hairstyle and other information of the human body based on the neural network model. In this way, suitable outfits can be recommended to users based on the identified character information, and the accuracy of outfit recommendations can be improved, thereby improving the user experience.

[0037] It can be understood that the electronic device of the embodiment of the present application can be a terminal device in any form. For example, the electronic device may include: a mobile phone, a tablet computer, a PDA, a laptop computer, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, an electronic device in a 5G network or a public land mobile communication network (public land mobile communication network) to be evolved in the future. The electronic devices in the mobile network (PLMN) are not limited to this in the embodiments of the present application.

[0038] In addition, in the embodiments of the present application, the electronic device may also be an electronic device in an Internet of Things (IoT) system. IoT is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network that interconnects people and machines and things.

[0039] The electronic device in the embodiments of the present application may also be referred to as: user equipment (UE), mobile station (MS), mobile terminal (MT), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent or user device, etc.

[0040] In an embodiment of the present application, the electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system layer. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory (also called main memory). The operating system can be any one or more computer operating systems that implement business processing through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system. The application layer includes applications such as a browser, an address book, a word processing software, and an instant messaging software.

[0041] For example, Figure 1 A schematic structural diagram of an electronic device is shown.

[0042] The electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0043] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may include hardware, software, or a combination of software and hardware.

[0044] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0045] For example, in an embodiment of the present application, the processor 110 can be used to extract location information of feature points in a user's portrait photo, or the processor 110 can also be used to classify features such as skin color or hairstyle in the portrait photo to obtain classification information, etc.

[0046] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or is cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the above-mentioned memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thereby improving the efficiency of the system. For example, in an embodiment of the present application, the processor 110 may be used to store the location information of feature points in a portrait photo or the classification information of skin color and hairstyle, etc., which are not specifically limited here.

[0047] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, at least one application required for a function, etc. The data storage area may store data created during the use of the electronic device, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the internal memory 121 and / or instructions stored in a memory provided in the processor. For example, in an embodiment of the present application, the internal memory 121 may be used to store a trained outfit recommendation network model. For example, it may include parameters and structures of a feature point recognition network model, a multi-classification network model, and a conditional generative adversarial network model.

[0048] In some embodiments, the electronic device may include 1 or N cameras 193, where N is a positive integer greater than 1. The camera 193 may be used to capture static images or videos. For example, in an embodiment of the present application, the electronic device may collect a user's portrait photo through the camera 193.

[0049] Figure 2 It is a software structure block diagram of the electronic device of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, the hardware abstraction layer (hardware adaptation layer, HAL), and the kernel layer.

[0050] The application layer can also be called the application layer, which can include a series of application packages. Figure 2 As shown, the application package may include applications such as phone, music, calendar, camera, game, memo, video, etc. Applications may include system applications and third-party applications.

[0051] Optionally, the application layer may also include an outfit recommendation application, wherein the outfit recommendation application may provide outfit recommendations to the user based on the user's portrait photos.

[0052] The application framework layer can provide an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer can include some predefined functions.

[0053] like Figure 2 As shown, the application framework layer may include an activity manager, a window manager, a resource manager, a notification manager, a content provider, a view system, and the like.

[0054] Android runtime includes core libraries and virtual machines. Android runtime is responsible for the control and management of the Android system.

[0055] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0056] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection. For example, in the embodiment of the present application, the virtual machine can be used to extract the location information of feature points in the user's portrait photo, or the virtual machine can also be used to classify features such as skin color or hairstyle in the portrait photo to obtain classification information.

[0057] The system library can also be called the Native layer, which can include multiple functional modules, such as media libraries, function libraries, and graphics processing libraries (such as OpenGL ES).

[0058] HAL is an abstract structure between the kernel layer and Android runtime. The hardware abstraction layer can be a package of hardware drivers, providing a unified interface for upper-layer applications to call.

[0059] The kernel layer is the layer between hardware and software. The kernel layer may include display driver, camera driver, audio driver, battery driver, Bluetooth driver, CPU driver, USB driver, etc.

[0060] It should be noted that the embodiments of the present application are only illustrated using the Android system as an example. In other operating systems (such as Windows system, IOS system, etc.), as long as the functions implemented by each functional module are similar to those in the embodiments of the present application, the solutions of the present application can also be implemented.

[0061] The following specific embodiments are used to describe the method of the present application in detail. The following embodiments may be combined with each other or implemented independently, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0062] It is understandable that in the embodiment of the present application, the training process of the outfit recommendation network model can be implemented on a cloud server or on an electronic device used by the user. Among them, the electronic devices used by the user may include, for example, mobile phones, watches, tablets, computers, etc., and the cloud server may also be referred to as a cloud server or server. When the model training process of the outfit recommendation network is implemented on a cloud server, the amount of calculation of the electronic devices used by the user can be reduced, the operating performance can be improved, and thus the jamming can be reduced. For the sake of ease of description, the following explanation will be given by taking the cloud server to train the outfit recommendation network model as an example.

[0063] Let's combine Figure 3 The overall structure of the outfit recommendation network model provided by the application embodiment is introduced. Figure 3 As shown, the outfit recommendation network model provided in the embodiment of the present application may include a feature point recognition network model, a multi-classification network model, and a conditional generative adversarial network model.

[0064] It is understandable that the input of the outfit recommendation network model can be a user's portrait photo. Optionally, the portrait photo can be a real-time portrait photo of the user, or a portrait photo stored in an electronic device, which is not limited here.

[0065] When the user inputs the portrait photo into the outfit recommendation network model, the feature point recognition network model can extract multiple feature point information of the human body in the portrait photo. Optionally, the feature point information of the human body can include the position information of key points of the human body, and the key points can include the forehead, chin, both sides of the cheeks, both sides of the shoulders, both sides of the waist, both knees, both ankles, etc.

[0066] It can be understood that the facial shape, body proportions, fatness and other physical features of the person can be calculated through the position information of these key points of the human body.

[0067] For the sake of simplicity, the position information of key points of the human body included in the feature point information is referred to as the position information of the feature points in the following text.

[0068] The multi-classification network model can obtain classification information such as skin color, hairstyle or gender in the user's portrait photo. After the feature point recognition network model obtains the feature point information, and the multi-classification network model obtains the classification information of skin color, hairstyle or gender, the feature point information, the classification information of skin color, hairstyle or gender and the user's portrait photo can be input into the conditional generative adversarial network model.

[0069] Furthermore, the conditional generative adversarial network model can generate one or more corresponding sets of outfit recommendations based on the user's portrait photo and the information output by the feature point recognition network model and the multi-classification network model. Optionally, the outfit recommendations can be described in text form.

[0070] For example, the outfit recommendation described in text may include more detailed clothing types and the corresponding outfit recommendations for each clothing type. The clothing types may include tops, bottoms, coats, and shoes, etc. The corresponding outfit recommendations for each clothing type may be, for example: "Top: white shirt"; "Bottoms: black long skirt"; "Coat: light-colored windbreaker"; "Shoes: white sneakers".

[0071] Alternatively, the outfit recommendation may also include clothing images of various clothing types suitable for the user. The clothing images of various clothing types may include, for example, clothing images of white shirts, clothing images of black long skirts, clothing images of light-colored windbreakers, and clothing images of white sneakers. The clothing images of various clothing types may be a set of outfit recommendations in the form of pictures.

[0072] Furthermore, the outfit recommendation network model may obtain an outfit list based on the one or more sets of outfit recommendations. Optionally, each set of outfit recommendations in the outfit list may include outfit recommendations with text descriptions. Alternatively, each set of outfit recommendations in the outfit list may also include clothing images of multiple clothing types. Alternatively, the outfit list may also include outfit recommendations that combine text descriptions and clothing images, wherein the text descriptions may provide specific clothing types or color suggestions, and the clothing images may provide visual references.

[0073] It should be noted that the clothing types described above, including tops, bottoms, coats, and shoes, are only exemplary descriptions, and the number and specific types of clothing types are not limited. In the actual application process of the embodiments of the present application, more clothing types may be included, for example, clothing types may also include accessories, bags, or glasses, etc. For the sake of brevity, examples are not given one by one here.

[0074] Optionally, the outfit recommendation network model provided in the embodiment of the present application can also be used to generate clothing images of various clothing types based on the conditional generative adversarial network model, and perform image processing on the portrait photo input by the user to obtain the outfit recommendation picture. The characters in the outfit recommendation picture can "wear" the clothing images of various clothing types generated by the conditional generative adversarial network model.

[0075] Furthermore, the outfit list may also include outfit recommendation pictures, so that the outfit list can more intuitively show the user the actual effect of the outfit recommendation on themselves, thereby improving the user's visual experience and increasing user satisfaction.

[0076] In a possible implementation, the outfit recommendation network model can perform image segmentation on the portrait photo input by the user, identify and separate different parts of the person, such as the head, torso and limbs, etc. And it can locate the key points such as shoulders, waist, knees and ankles of the person in the portrait photo based on the feature point information. In addition, the outfit recommendation network model can also perform geometric transformation on the clothing image in the outfit recommendation based on the multiple key points to obtain a deformed clothing image that fits the key points of the person.

[0077] Furthermore, the outfit recommendation network model can obtain corresponding outfit recommendation pictures by splicing different parts of the character such as the head, torso and limbs and clothing images after geometric transformation.

[0078] For example, suppose the original outfit of the person in the portrait photo input by the user is a white short-sleeved shirt and dark blue jeans. The conditional generative adversarial network model can recommend a white shirt and a black long skirt as the outfit suitable for the user based on the person's face shape, body proportion, fatness and thinness, as well as the classification information of skin color, hairstyle or gender.

[0079] Furthermore, the outfit recommendation network model can identify and separate different parts of a person, such as the head, torso, and limbs. It can also perform geometric transformation on the clothing image of the white shirt and black long skirt based on the feature point information. Then, the corresponding outfit recommendation can be obtained based on the head, torso, and limbs of the person and the clothing image of the white shirt and black long skirt after the geometric transformation.

[0080] The following is a detailed introduction to the implementation process of the image processing method of the embodiment of the present application from the aspects of (i) defining the human feature model, (ii) identifying the human feature points, and (iii) identifying the human skin color, gender, and hairstyle.

[0081] 1. Define the human body feature model.

[0082] In order to make the recommended outfit fit the person's physical characteristics, the embodiment of the present application defines a human body feature model, which can be used to identify the person's facial shape, body proportions, fatness, and other physical characteristics.

[0083] Exemplarily, the human body feature model may include multiple feature points of the human body, and the distribution diagram of the multiple feature points may be as follows: Figure 4 The feature points may include the forehead, chin, cheeks, shoulders, waist, knees, ankles, etc. in the portrait.

[0084] Furthermore, multiple feature points of the human feature model can be used to calculate the face shape, body proportions, fatness and other physical features of the person. Exemplarily, the embodiment of the present application can calculate the face shape of the person based on the position information of the four feature points of the forehead, chin and both sides of the cheeks. For example, the electronic device can calculate the length of the face of the person based on the position information of the forehead and chin. And the electronic device can calculate the width of the face of the person based on the position information of both sides of the cheeks. And the electronic device can also calculate the length-to-width ratio of the face of the person based on the length and width of the face.

[0085] After obtaining the face length-to-width ratio of the person, the face shape of the person can be determined based on the face length-to-width ratio of the person. The face length-to-width ratio can be calculated by dividing the face length by the face width.

[0086] In a possible implementation, different face shape settings and their corresponding threshold ranges may be preset in the electronic device. For example, when the face shapes of a person include round face, oval face and long face, the aspect ratio of a round face may correspond to threshold range a, the aspect ratio of an oval face may correspond to threshold range b, and the aspect ratio of a long face may correspond to threshold range c. After determining the threshold range to which the aspect ratio of the face of the person belongs, the electronic device can determine the face shape of the person.

[0087] It should be noted that the above classification of the face shapes of characters into round face, oval face and long face is only an exemplary description, and does not limit the number of face shapes and the names of face shapes. In the actual application process of the embodiments of the present application, the number and names of face shapes can be adjusted and expanded according to specific needs. For example, the electronic device can also add more face shapes such as square face, heart-shaped face, diamond face, etc. to adapt to a wider user group.

[0088] It should be noted that the above method for calculating the face shape of a person based on the position information of the four feature points of the forehead, chin and cheeks is only an exemplary description. In the actual application process of the embodiment of the present application, the calculation of the face shape of a person can also be calculated based on additional feature points to obtain more accurate face shape features of the person.

[0089] Exemplarily, additional feature points may also include cheekbones on both sides or mandibular angles on both sides, that is, the electronic device can confirm more face shape categories based on the forehead, chin and cheeks, as well as the position information of additional feature points, which is not limited here. Therefore, in the actual application process of the embodiments of the present application, the feature points for calculating the face shape are not limited to a few fixed points, but can be flexibly adjusted according to needs to adapt to the diverse characteristics of different users. For the sake of brevity, the following only describes in detail the process of confirming the face shape based on the four feature points of the forehead, chin and cheeks.

[0090] In addition, the embodiment of the present application can also calculate the body proportion or physical features such as fatness and thinness of the person based on the position information of all or part of the feature points in each feature point. Exemplarily, the body proportion in the embodiment of the present application can include waist-to-shoulder ratio, leg-to-body ratio, and upper-to-lower-body ratio. The electronic device can calculate the shoulder width of the person based on the position information of the feature points on both sides of the shoulder, calculate the waist circumference of the person based on the position information of the feature points on both sides of the waist, and calculate the waist-to-shoulder ratio of the person based on the shoulder width and waist circumference.

[0091] Similarly, the electronic device can calculate the leg length of the person based on the position of the feature points of the waist and ankles, calculate the height of the person based on the position information of the feature points of the forehead and ankles, and calculate the leg-to-body ratio of the person based on the leg length and the height. In addition, the electronic device can also calculate the upper body length of the person based on the position information of the feature points of the shoulders and waist, and calculate the upper-to-lower body ratio of the person based on the upper body length and the leg length.

[0092] It should be noted that the above process of calculating the waist-to-shoulder ratio, leg-to-body ratio, and upper-to-lower-body ratio in the body proportions based on the position information of all or part of the feature points is only an exemplary description, and does not limit the body proportion indicators and the feature points used to calculate the body proportions. In actual application, more body proportion indicators can also be calculated based on the position information of additional feature points.

[0093] Exemplarily, the additional feature points may also include feature points on both sides of the buttocks. More body proportion indicators may include, for example, the waist-to-hip ratio. The electronic device may calculate the hip circumference of the person based on the position information of the feature points on both sides of the buttocks, and calculate the waist-to-hip ratio of the person based on the hip circumference and waist circumference. Therefore, in the actual application process of the embodiments of the present application, the feature points for calculating the body proportions are not limited to a few fixed points, but can be flexibly adjusted according to demand to obtain a more accurate body proportion.

[0094] When calculating the fatness or thinness of a person, one possible implementation is to calculate based on the person's height and waist circumference. For example, after obtaining the person's height and waist circumference, the fatness or thinness of the person can be determined based on the ratio of the height and waist circumference. The ratio of the height and waist circumference can be obtained by dividing the height by the waist circumference. When the result obtained by dividing the height by the waist circumference is smaller, it can be understood that the person's waist is fat, and when the result obtained by dividing the height by the waist circumference is larger, it can be understood that the person's waist is thin.

[0095] It is understood that the feature points for calculating the fatness or thinness of a person can also be adjusted according to needs to obtain a more accurate fatness or thinness. For example, the feature points in the human feature model can also include feature points on both sides of the chest, and the chest circumference of the person can be calculated based on the feature points on both sides of the chest. When the chest circumference of the person is large, it can be understood that the chest of the person is fat.

[0096] Optionally, in order to obtain more comprehensive physical features about the fatness and thinness of a person, in addition to obtaining the fatness and thinness of a person based on feature points, the user may also input data such as height and weight. That is, the human feature model may also include the height and weight of a person, and the body mass index (BMI) of a person may be calculated based on the height and weight. BMI may be used to indicate the degree of obesity of a person, and the higher the BMI, the more obese the person is.

[0097] The above process introduces that when a human body feature model includes multiple feature points of a human body, the face shape, body proportions, fatness and thinness and other physical features of the person can be determined based on the multiple feature points. Among them, different physical features can correspond to different outfit recommendations. In addition, the skin color, hairstyle and gender characteristics of the person can also be identified in the human body feature model. Exemplarily, in the human body feature model provided in this embodiment, the skin color can be divided into black, white and yellow, the hairstyle can be divided into long hair and short hair, and the gender can be divided into male and female.

[0098] It should be noted that the above classification of skin color and hairstyle categories is only an exemplary description, and does not limit the names and numbers of skin color and hairstyle categories. In actual application, skin color can also include warm tones and cool tones, and hairstyles can also include straight hair or curly hair. That is, the names and numbers of skin color and hairstyle categories can be adjusted and expanded according to specific needs, so that the human feature model can better adapt to the personalized needs of different users and provide more accurate and diverse outfit recommendations.

[0099] It should be noted that the human feature model defined above does not directly participate in the training of the neural network model. It can also be understood that the human feature model can be used to describe the characteristic data such as the characteristic points, skin color, hairstyle and gender of the person. The human feature model defined in this way helps to improve the consistency and integrity of the data in the sample collection and preprocessing stage, but the human feature model itself is not trained in the subsequent actual training process.

[0100] 2. Identification of human body feature points.

[0101] The embodiment of the present application can identify and obtain the location information of the feature points of a person based on the feature point recognition network model.

[0102] In a possible implementation, before identifying the location information of human feature points, the feature point recognition network model can be trained based on training samples, and the training samples include multiple different images. For example, the training samples can be used as input data and input into the feature point recognition network model, and the location information of human feature points can be learned through deep learning of multiple levels of the model. In the process of model training, the number of images included in the specific training samples is not limited in the embodiments of the present application.

[0103] Optionally, the coordinates of the feature points can be used to represent the position information of the feature points. That is, for one of the feature points, two coordinate values, namely, the horizontal coordinate and the vertical coordinate, can be included. In the case where the human feature model includes 12 feature points, namely, the forehead, the chin, both sides of the cheeks, both sides of the shoulders, both sides of the waist, both sides of the knees, and both sides of the ankles, the position information of the 12 feature points can be represented by 24 coordinate values.

[0104] Optionally, the data format of the 24 coordinate values ​​can be represented by data or vector, which is not limited here. Therefore, the output result of the feature point recognition network model can include 24 dimensional values, which can be used to indicate the position information of the 12 feature points.

[0105] Exemplarily, the feature point recognition network model can be composed of a fusion convolution module, wherein the fusion convolution module can be used for feature extraction. In an embodiment of the present application, two fusion convolution modules can be set as one fusion convolution module group, and 5 groups of fusion convolution module groups can be stacked in the feature point recognition network model, that is, there are a total of 10 fusion convolution modules in the feature point recognition network model. Among them, each group of stacked fusion convolution module groups can make the size of the feature map half of the original size, and the number of channels of the feature map is doubled.

[0106] It should be noted that the size of the feature map may include the height and width of the feature map, that is, the height and width of the feature map are each reduced to half of the original. For ease of description, hereinafter, the height and width of the feature map are each reduced to half of the original and are referred to as the size of the feature map being reduced to half of the original.

[0107] Furthermore, the feature point recognition network model may also include an output network. That is, after the feature map is obtained by feature extraction of the input image, the feature map can be sent to the output network. The output network may be sequentially provided with a convolution layer, a global pooling layer, and a fully connected layer, so that the size of the feature map finally output by the output network is 1×1×24, that is, the number of channels of the feature map finally output is 24. The values ​​output by the 24 channels can be used to indicate the position information of the 12 feature points.

[0108] Figure 5This is a schematic diagram of the structure of the feature point recognition network model provided in the embodiment of the present application. Figure 5 As shown, for the convenience of description, the feature point recognition network model is divided into the first module 501 to the fifth module 505. That is, the first module 501 to the fifth module 505 are used to describe the structure of each part of the feature point recognition network model. In addition, the structure of the image data or feature map data is described in the format of H×W×C, where H can represent height, W can represent width, and C can represent the number of channels.

[0109] (5.1) The first module 501: may include a convolution layer (conv), batch normalization (BN), and an H-Swish (Hard-Swish) activation function. The convolution layer may be used to extract local features from an input feature map. Batch normalization may be used to normalize the output of the convolution layer, thereby stabilizing the training process. The H-Swish activation function may be used to improve training efficiency and model performance. That is, in the first module 501, local features of the input image may be extracted through the convolution layer a to obtain a feature map after convolution, and the feature map after convolution may be sequentially applied with batch normalization and the H-Swish activation function to obtain a feature map a.

[0110] For example, Figure 5 As shown, when the convolution kernel size of the convolution layer a is set to 3×3, the convolution step size can be set to 2, so as to adjust the size of the feature map a output by the convolution layer a to half of the input image. In addition, the number of convolution kernels of the convolution layer a can be set to 16 to adjust the number of channels of the feature map a to 16. It can be understood that when the H×W×C of the input image is 448×448×3, the feature map a of 224×224×16 can be obtained after the input image is extracted by the first module 501.

[0111] It can be understood that the first module 501 can be used to perform preliminary feature extraction and preprocessing on the input image. Through the combination of 3×3 convolution, batch normalization and H-Swish activation function, the first module 501 can extract preliminary features from the input image, and adjust the spatial size and number of channels of the feature map to prepare for feature extraction of subsequent modules.

[0112] (5.2) The second module 502: can be composed of multiple fused convolution modules. It can be understood that the input of the second module 502 can be the feature map a output by the first module 501, and the output of the second module 502 can be the feature map b. In the embodiment of the present application, two fused convolution modules can be set as a group, and the fused convolution module group can be stacked 5 times in the second module 502, that is, the second module 502 can include 10 fused convolution modules.

[0113] For ease of understanding, the following Figure 6 The fused convolution module provided in the embodiment of the present application is introduced. Figure 6 The schematic diagram of the structure of the fused convolution module provided in the embodiment of the present application, wherein H1×W1×C1 can represent the size and number of channels of the input feature map of the fused convolution module. In the fused convolution module, the input feature map can be first processed as follows: Figure 6 The four parts shown in the figure can be processed, and the results of the processing of the four parts can be spliced ​​to obtain a spliced ​​feature map.

[0114] Among them, the processing steps of the four parts are as follows:

[0115] (1) 1×1 convolution and squeeze-and-excitation (SE) module: 1×1 convolution can be used for channel fusion to reduce the dimension of the input feature map. The SE module can enhance significant features by calculating the importance weight of each channel. In other words, the SE module can adjust the weight of each channel through global information, highlight important feature areas, and reduce unnecessary calculations. It can be understood that this part can fuse the features between feature map channels while reducing the amount of calculation, and weight the fused features to highlight significant features.

[0116] Optionally, the number of channels of the 1×1 convolution can be set to C1, that is, the input feature map is convolved using a convolution layer with a convolution kernel size of 1×1 and a channel number of C1, wherein the number of convolution kernels can be set as required and is not limited here. Optionally, the channel vector of the SE module can also be set to 1×1×C1.

[0117] (2) Depthwise separable convolution: It can include depthwise convolution and 1×1 pointwise convolution. Optionally, the depth of depthwise convolution and pointwise convolution can be C1, that is, firstly perform depthwise convolution on the input feature map using a convolution layer with C1 channels, and then perform pointwise convolution on the input feature map using a convolution layer with a convolution kernel size of 1×1 and C1 channels. This part can fuse the features within the neighborhood of the feature map channel while reducing the amount of calculation.

[0118] (3) 3×3 convolution: includes a convolution layer with a convolution kernel size of 3×3, where the depth of the convolution layer can be C1. This part can simultaneously perform spatial and channel feature fusion.

[0119] (4) Skip connection: The input feature map is not processed, that is, the input feature map is directly passed to the output, which can reduce the gradient disappearance phenomenon that occurs during training.

[0120] Furthermore, the fusion convolution module can concatenate the features processed by the above four parts and fuse multiple feature representations to obtain a richer and more comprehensive feature map c. The obtained features will be richer and more comprehensive.

[0121] Furthermore, for the spliced ​​feature map c, the feature map c can be input into path 1 or path 2 according to specific needs. Among them, path 1 can include a convolution layer with a convolution kernel size of 3×3 and a channel number of C2. Path 2 can include a pooling layer. That is, path 1 can perform a 3×3 convolution on feature map c, and path 2 can perform pooling on feature map c.

[0122] It should be noted that in the embodiment of the present application, when the size and number of channels of the feature map need to be changed, the spliced ​​feature map can be sent to path 1, that is, feature extraction is performed on the spliced ​​feature map through path 1. When the size and number of channels of the feature map do not need to be changed, the spliced ​​feature map can be sent to path 2, that is, feature extraction is performed on the spliced ​​feature map through path 2 through pooling operation.

[0123] In the embodiment of the present application, the fused convolution module group can be stacked 5 times in the second module 502, and each stacking can make the size of the feature map half of the original size and the number of channels twice the original size. Taking the size and number of channels of the input feature map a of the second module 502 as 224×224×16, and the size and number of channels of the feature map b to be output as 7×7×512 as an example, the size and number of channels of the feature map a can be changed from 224×224×16 to 112×112×32, 56×56×64, 28×28×128, 14×14×256, 7×7×512 in sequence, thereby obtaining the feature map b.

[0124] Therefore, for the two fused convolution modules in each fused convolution module group, the first fused convolution module can be set to need to change the feature map size and the number of channels, and the second fused convolution module can be set to not need to change the feature map size and the number of channels. That is, the feature map c in the first fused convolution module can be input to path 1, and the feature map c in the second fused convolution module can be input to path 2.

[0125] That is, in each fused convolution module group, the first fused convolution module can make the input feature map size half of the original size and the number of channels double the original size. For example, when the size and number of channels of the input feature map are 224×224×16, the size and number of channels of the feature map can be changed to 112×112×32 after the first fused convolution module. The second fused convolution module does not change the size and number of channels of the input feature map, that is, when the size and number of channels of the input feature map are 112×112×32, the size and number of channels of its output feature map are still 112×112×32.

[0126] It can be understood that since the size of the feature map can be reduced to half of its original size each time it passes through the fused convolution module corresponding to path 1, the convolution step size can be set to 2 during the convolution of path 1 to reduce the size of the feature map to half of its original size.

[0127] It should be noted that, as the number of channels of each set of input and output feature maps is different, the number of convolution kernels in the convolution layer in path 1 of the first fused convolution module in each group may be different, that is, the number of convolution kernels in the convolution layer in path 1 is related to the number of channels of the output feature map. The following takes the first group of fused convolution modules as an example, that is, the size and number of channels of the input feature map are 224×224×16, and the size and number of channels of the output feature map are 112×112×32, as an example to introduce the setting of the number of convolution kernels in the fused convolution module.

[0128] It should also be noted that in the above four processing steps, the number of channels in each part is not limited in the embodiment of the present application, and can be set according to application needs in actual application. That is, the number of channels of the feature map c obtained by splicing can be the same as C1 or different from C1.

[0129] For the first fused convolution module in the first group, the concatenated feature map c is sent to path 1 for a 3×3×C2 convolution operation, where the size of C2 can be consistent with the number of channels of feature map c. Since the size and number of channels of the output feature map of the first fused convolution module are 112×112×32, the convolution layer in path 1 includes 32 3×3×C2 convolution kernels.

[0130] Similarly, for the first fused convolution module in the second group, the convolution layer in path 1 can be set to include 64 3×3×C2 convolution kernels; for the first fused convolution module in the third group, the convolution layer in path 1 can be set to include 128 3×3×C2 convolution kernels; for the first fused convolution module in the fourth group, the convolution layer in path 1 can be set to include 256 3×3×C2 convolution kernels; for the first fused convolution module in the fifth group, the convolution layer in path 1 can be set to include 512 3×3×C2 convolution kernels.

[0131] Optionally, for each fused convolution module in the second module 502, batch normalization and H-Swish activation function may also be applied to the result of the fused convolution module.

[0132] It can be understood that the fused convolution model proposed in the embodiment of the present application can enhance the ability of feature extraction by fusing adjacent feature maps. That is, in the fused convolution module, the inter-channel features, channel neighborhood features, comprehensive features between channel neighborhoods and channels, and original features obtained after the above-mentioned 4-part processing are spliced ​​to obtain a spliced ​​feature map. And the spliced ​​feature map can be further subjected to feature extraction, which can make the extracted features richer, thereby improving the recognition accuracy. In this way, the feature point recognition network model can capture more detailed feature information, which helps to improve the overall performance of the model.

[0133] (5.3) Output network: including a third module 503, a fourth module 504 and a fifth module 505. The third module 503 may be a 1×1 convolution layer, the fourth module 504 may be a global pooling layer, and the fifth module 505 may be a fully connected layer. Optionally, for the third module 503, batch normalization and H-Swish activation function may also be applied to the result of its convolution; and, for the fourth module 504, batch normalization and H-Swish activation function may also be applied to the result of its pooling.

[0134] It can be understood that in the feature point recognition network model, the input image can output a 1×1×24 feature map after passing through the first module 501 to the fifth module 505. The 24 values ​​can correspond to the horizontal and vertical coordinates of the 12 feature points to indicate the position information of each feature point in the training sample. It should be noted that when the accuracy of the feature point recognition network model in recognizing the position information of the character feature points reaches the preset standard, the feature point recognition network model can be used to recognize the human feature points of the image, and can output the position information of the character feature points in the portrait photo.

[0135] In the embodiment of the present application, the location information of the feature points of the person in the portrait photo can be identified through the feature point recognition network model. Furthermore, the person's face shape, body proportions, fatness and thinness and other physical features can be calculated based on the location information of the feature points of the person, so that the outfit recommendation network can recommend outfit recommendations that are more consistent with the user's physical features to the user, making the recommendation results more accurate, thereby improving the user experience.

[0136] (3) Identification of a person’s skin color, gender, and hairstyle.

[0137] The embodiment of the present application can identify and obtain a person's skin color, gender, and hairstyle based on a multi-classification network model.

[0138] In a possible implementation, before recognizing a person's skin color, gender, and hairstyle, the multi-classification network model can be trained based on training samples, and the training samples include multiple different images. For example, the training samples can be used as input data and input into the multi-classification network model, and the skin color, gender, and hairstyle of the person can be learned through deep learning of multiple levels of the model. In the process of model training, the number of images included in the specific training samples is not limited in the embodiments of the present application.

[0139] Optionally, the multi-classification network model can also be composed of fused convolution modules. In the embodiment of the present application, similar to the feature point recognition network model, a fused convolution module group can also be set in the multi-classification network model, and 5 groups of fused convolution module groups can be stacked in the multi-classification network model.

[0140] For example, Figure 7 This is a schematic diagram of the structure of the multi-classification network model provided in the embodiment of the present application. Figure 7 As shown, for the convenience of description, the multi-classification network model is divided into the sixth module 701 to the tenth module 705. It can be understood that the multi-classification network model has the same backbone network as the feature point recognition network model, that is, the sixth module 701 can be the same as the first module 501. For details, please refer to the relevant description of the first module 501, which will not be repeated here.

[0141] Similarly, the seventh module 702 can be the same as the second module 502, and the relevant description of the second module 502 can be specifically referred to. The eighth module 703 can be the same as the third module 503, and the relevant description of the third module 503 can be specifically referred to. The ninth module 704 can be the same as the fourth module 504, and the relevant description of the fourth module 504 can be specifically referred to, which will not be repeated here. The following focuses on the tenth module 705.

[0142] In the embodiment of the present application, the multi-classification network model sets a tenth module 705 in the output network, and the tenth module 705 may include three fully connected layers, which may be learning layers for skin color, gender, and hairstyle, respectively. Among them, the output of the skin color learning layer may be the probability of identifying the skin color of the person. In the case where the outfit recommendation network model divides the skin color of the person into black, white, and yellow, the skin color learning layer may output the probability values ​​of identifying the person as these three skin colors.

[0143] In a possible implementation, after obtaining the probability values ​​of the three skin colors, and when the highest probability value among the three probability values ​​exceeds the preset skin color threshold, the skin color with the highest probability value output in the skin color learning layer can be set as the recognition result of the multi-classification network model. For example, assuming that the probability value of black is 0.8, the probability value of white is 0.2, and the probability value of yellow is 0.1, and 0.8 exceeds the preset skin color threshold, then the recognition result of the person's skin color in the multi-classification network model can be considered to be black.

[0144] It can be understood that the multi-classification network model's recognition process for a person's hairstyle and gender is similar to that for skin color, and will not be elaborated here.

[0145] It can be understood that in the multi-classification network model, the input image can output the probability value of the recognition result of the skin color, hairstyle and gender of the person after passing through the sixth module 701 to the tenth module 705. It should be noted that when the accuracy of the multi-classification network model in recognizing the skin color, hairstyle and gender of the person reaches the preset standard, the multi-classification network model can be used to recognize the portrait photo, and can output the recognition result of the skin color, hairstyle and gender of the person in the portrait photo.

[0146] In an embodiment of the present application, the physical features such as skin color, hairstyle and gender of the person in the portrait photo can be identified through a multi-classification network model, so that the outfit recommendation network can recommend outfit recommendations to the user that are more consistent with the user's physical features, making the recommendation results more accurate, thereby improving the user experience.

[0147] It should be noted that the above-mentioned identification of a person's skin color, hairstyle, and gender through a multi-classification network model is only an exemplary description. In the actual application of the outfit recommendation network model, the multi-classification network model can also include classifications of other physical features, such as age group, glasses wearing conditions, etc. Specific classification features can be adjusted and expanded according to actual needs to better meet the personalized needs of different user groups. I will not go into details here.

[0148] 4. Generate outfit recommendations.

[0149] The embodiment of the present application can generate outfit recommendations for characters based on a conditional generative adversarial network model.

[0150] In a possible implementation, before generating outfit recommendations for a person, the conditional generative adversarial network model can be trained based on training samples, where the training samples include multiple different images. Optionally, the input data of the conditional generative adversarial network model can include training samples, location information of each feature point obtained by the feature point recognition network model in the training samples, and physical features such as gender, skin color, and hairstyle obtained by the multi-classification network model in the training samples.

[0151] It should be noted that the conditional generative adversarial network model includes a generator and a discriminator. The generator can be used to generate one or more outfit recommendations based on the position information of each feature point in the input data and the classification information of skin color, hairstyle and gender. Optionally, the outfit recommendation can be one or more recommended images, and the one or more recommended images can be used to indicate the outfit recommendation generated by the outfit recommendation network model for the character.

[0152] Correspondingly, the discriminator can output a recommendation probability value based on the real images in the training samples in the input data and the recommended images generated by the generator, and the recommendation probability value can indicate the possibility that the recommended image is a real image.

[0153] Furthermore, the conditional generative adversarial network model can be trained adversarially based on input data. After multiple rounds of iterative training, when the generator and discriminator in the conditional generative adversarial network model reach the preset standard, the conditional generative adversarial network model can be used to generate outfit recommendations for users based on the image, the location information of each feature point obtained by the feature point recognition network model, and the physical features such as gender, skin color and hairstyle obtained by the multi-classification network model.

[0154] Optionally, the preset standard may be that the generator is able to generate highly realistic recommendation images, while the discriminator has difficulty in distinguishing between the recommendation images and real images.

[0155] In an embodiment of the present application, a conditional generative adversarial network model can be used to generate outfit recommendations that are more consistent with the user's physical characteristics, making the recommendation results more accurate, thereby improving the user experience.

[0156] It can be understood that the outfit recommendation network model provided in the embodiment of the present application adopts the same end-to-end neural network model, that is, according to different recognition targets, such as different physical features, feature extraction and target recognition are performed separately through multiple learning links. In this way, each learning link can focus on feature learning of a specific target, thereby improving the specificity and accuracy of recognition, so that the outfit recommendations generated by the outfit recommendation network model are more suitable for the physical features of the person, the recommendation effect is better, and the user satisfaction is improved.

[0157] Figure 8 The image processing method of the embodiment of the present application is shown. The method includes:

[0158] S801, acquiring a target image.

[0159] The target image may be a photo containing a portrait, which may be understood as the portrait photo mentioned above. The target image may be a portrait photo taken by an electronic device in real time, or a portrait photo stored in the electronic device, which is not limited here.

[0160] S802, obtaining first information of the target image based on the first model; the first model is used to obtain position information of human body parts in the image input into the first model, and the first information includes the position information of the human body parts in the target image.

[0161] Among them, the human body parts can be understood as the feature points mentioned above, that is, the human body parts can include but are not limited to the forehead, chin, both sides of the cheeks, both sides of the shoulders, both sides of the waist, both knees, both ankles and other parts. The first information can be understood as the feature point information mentioned above, that is, the first information can include the position information of the human body parts in the target image. Optionally, the position information can be in array form, or can also be in coordinate form, which is not limited here.

[0162] The first model can be understood as the feature point recognition network model mentioned above. The model structure and implementation process of the first model can refer to the relevant description of the feature point recognition network model mentioned above, which will not be repeated here. It can be understood that for the image input into the first model, the first model can obtain the position information of the human body parts of the image. In an embodiment of the present application, after the electronic device obtains the target image, the target image can be input into the first model, and the position information of the human body parts in the target image can be identified based on the first model.

[0163] S803. Obtain second information of the target image based on the second model; the second model is used to obtain gender information, skin color information and / or hairstyle information of the person in the image input into the second model, and the second information includes the gender information, skin color information and / or hairstyle information of the person in the target image.

[0164] In the embodiment of the present application, gender can be divided into male and female, skin color can be divided into black, white and yellow, and hairstyle can be divided into long hair and short hair. Gender information can be understood as classification information of the gender of the person in the target image by the second model, wherein the classification information can be probability information, that is, the second model can identify the probability that the person in the target image is male and the probability that the person is female.

[0165] Similarly, skin color information can be understood as classification information of the skin color of the person in the target image by the second model, and hairstyle information can be understood as classification information of the hairstyle of the person in the target image by the second model, which will not be repeated here.

[0166] The second model can be understood as the multi-classification network model mentioned above, and the structure or implementation process of the second model can refer to the relevant description of the multi-classification network model mentioned above, which will not be repeated here. It can be understood that for the image input into the second model, the second model can obtain the gender information, skin color information and / or hairstyle information of the image. In an embodiment of the present application, after the electronic device obtains the target image, the target image can be input into the second model, and the gender information, skin color information and / or hairstyle information of the person can be obtained based on the second model.

[0167] It should be noted that the second information in the embodiment of the present application includes the gender information, skin color information and / or hairstyle information of the person, which is only an exemplary description. In the actual application process of the image processing method provided in the embodiment of the present application, the second model can also identify other characteristics of the person. For example, the second model can also identify the age group and eye wearing condition of the person. Correspondingly, the second information can also include the age group information, eye wearing condition information, etc. of the person. The specific classification features can be adjusted and expanded according to actual needs to better meet the personalized needs of different user groups.

[0168] S804. Obtain an outfit image of a person in the target image based on the target image, the first information, the second information and the third model. The third model is used to obtain the physical features of the person in the image input into the third model based on the output information of the first model and the output information of the second model, and determine the outfit image of the person based on the physical features of the person.

[0169] Among them, the output information of the first model can be understood as the position information of the human body part of the image input to the first model. Similarly, the output information of the second model can be understood as the gender information, skin color information and / or hairstyle information of the image input to the second model. The third model can be understood as the conditional generative adversarial network model mentioned above, and the structure or implementation process of the third model can refer to the relevant description of the conditional generative adversarial network model mentioned above, which will not be repeated here.

[0170] The physical features of the character can refer to the above description, that is, the physical features can include but are not limited to the physical features such as the face shape, body proportions, fatness and thinness of the character, as well as the gender, skin color and hairstyle of the character. It can be understood that the third model can obtain the physical features such as the face shape, body proportions, fatness and thinness of the character based on the position information of the human body parts. The specific implementation process can refer to the above description, which will not be repeated here.

[0171] Correspondingly, the third model can also obtain the physical features such as the gender, skin color and hairstyle of the person based on the gender information, skin color information and / or hairstyle information of the person. Exemplarily, when each information is expressed in probability, the third model can select the feature value with the largest probability value as the physical feature corresponding to the person. Assuming that in the skin color information of the person, the probability value of black is 0.8, the probability value of white is 0.2, and the probability value of yellow is 0.1, the third model can obtain that the skin color of the person is black.

[0172] Furthermore, the third model can also determine the outfit image of the person based on the person's physical features, where the outfit image can be understood as the outfit recommendation image mentioned above. The specific implementation process can refer to the relevant description of generating the outfit recommendation image mentioned above, which will not be repeated here. That is, the person in the outfit image is generated based on the person in the input image.

[0173] Specifically, when the image input to the first model is the target image, the output information of the first model can be understood as the first information. Similarly, when the image input to the second model is the target image, the output information of the second model can be understood as the second information. It can be understood that after the target image, the first information, and the second information are input to the third model, an outfit image of the person in the target image can be obtained. The outfit of the person in the outfit image can be understood as an outfit that is suitable for the person's physical features.

[0174] In the embodiment of the present application, the electronic device can obtain the physical features of the person based on the target image input by the user, as well as the first model, the second model and the third model, and can also generate a clothing picture that conforms to the physical features of the person in the target image based on the third model. In this way, the user can more intuitively display the actual effect of the clothing recommendation on himself, improve the user's visual experience, and improve user satisfaction.

[0175] Optional, in Figure 8On the basis of the corresponding embodiments, the first information of the target image is obtained based on the first model, including: inputting the target image into the first model; the first model includes one or more first fusion convolution modules; based on the one or more first fusion convolution modules, the characters in the target image are feature extracted to obtain the first feature image output by the first target fusion convolution module; the first target fusion convolution module is any one of the one or more first fusion convolution modules; wherein the size of the first feature image output by the first target fusion convolution module is smaller than the size of the feature image input to the first target fusion convolution module, and the number of channels of the first feature image output by the first target fusion convolution module is greater than the number of channels of the feature image input to the first target fusion convolution module; performing convolution operations, pooling operations and full connection operations on the first feature images output by one or more first fusion convolution modules to obtain a second feature image; the second feature image includes the first information.

[0176] The first fused convolution module can be understood as the fused convolution module in the feature point recognition network model mentioned above. Figure 6 The description of the fused convolution module in the embodiment will not be repeated here. The first target fused convolution module can be understood as the fused convolution module corresponding to path 1 in a fused convolution module group of the feature point recognition network model mentioned above.

[0177] That is, the size of the first feature map output by the first target fusion convolution module is half the size of the feature image input to the first target fusion convolution module, and the number of channels of the first feature map output by the first target fusion convolution module is twice the number of channels of the feature image input to the first target fusion convolution module.

[0178] The first feature image can be understood as the feature image b mentioned above, the convolution operation can be performed by the third module 503 mentioned above, the pooling operation can be performed by the fourth module 504 mentioned above, and the full connection operation can be performed by the fifth module 505 mentioned above. For details, please refer to the relevant description above and will not be repeated here.

[0179] That is, after the feature map b is subjected to convolution operation, pooling operation and full connection operation, a second feature image can be obtained, and the second feature image can be understood as the feature map output by the fifth module 505 above. The feature map can include the first information, that is, the feature map can be used to indicate the position information of the human body part.

[0180] Exemplarily, assuming that the human body parts in the embodiment of the present application are the 12 parts of the forehead, chin, cheeks, shoulders, waist, knees, ankles, the second feature map can be a 1×1×24 feature map. The 24 values ​​can correspond to the horizontal and vertical coordinates of the 12 human body parts to indicate the position information of each human body part in the target image. Similarly, in the case where the human body parts also include other parts of the person, the number of channels of the second feature image can be twice the number of human body parts to indicate the horizontal and vertical coordinates of each human body part respectively.

[0181] In an embodiment of the present application, after the target image is input into the first model, the first information of the target image, that is, the position information of each body part of the person in the target image, can be obtained, which serves as a basis for the subsequent calculation of the person's facial shape, body proportions, fatness and other physical features based on the position information of the body parts.

[0182] Optional, in Figure 8 On the basis of the corresponding embodiments, the second information of the target image is obtained based on the second model, including: inputting the target image into the second model; the second model includes one or more second fused convolution modules; based on the one or more second fused convolution modules, the characters in the target image are feature extracted to obtain a third feature image output by the second target fused convolution module; the second target fused convolution module is any one of the one or more second fused convolution modules; the third feature image output by the one or more second fused convolution modules is subjected to convolution operation, pooling operation and full connection operation to obtain a fourth feature image; the fourth feature image includes the second information; wherein the full connection operation includes a first fully connected layer, a second fully connected layer and a third fully connected layer, the first fully connected layer is used to output information related to the gender of the character in the target image; the second fully connected layer is used to output information related to the skin color of the character in the target image; the third fully connected layer is used to output information related to the hairstyle of the character in the target image.

[0183] Among them, the second fused convolution module can be understood as the fused convolution module in the multi-classification network model mentioned above. For details, please refer to the above Figure 6 The relevant description of the fused convolution module in the embodiment will not be repeated here. The second target fused convolution module can be understood as the fused convolution module corresponding to path 1 in a fused convolution module group of the multi-classification network model mentioned above.

[0184] Similar to the first target fusion convolution module, the size of the third feature map output by the second target fusion convolution module is half the size of the feature image input to the second target fusion convolution module, and the number of channels of the third feature map output by the second target fusion convolution module is twice the number of channels of the feature image input to the second target fusion convolution module.

[0185] Further, for the third feature image output by the second target fusion convolution module, a convolution operation, a pooling operation and a full connection operation can be performed to obtain a fourth feature image. Among them, the convolution operation and the pooling operation can be performed by the eighth module 703 and the ninth module 704 respectively, which will not be repeated here.

[0186] The full connection operation can be performed by the tenth module 705 mentioned above, that is, the first fully connected layer can be understood as the gender learning layer mentioned above, the second fully connected layer can be understood as the skin color learning layer mentioned above, and the first fully connected layer can be understood as the hairstyle learning layer mentioned above. For details, please refer to the relevant description above, which will not be repeated here.

[0187] In the embodiment of the present application, the second information of the person in the target image, that is, the person's gender information, skin color information and hairstyle information, can be identified based on the second model. In this way, the accuracy of the electronic device in obtaining the person's gender, skin color, hairstyle and other physical features can be improved, and the dressing image output by the third model can be made more consistent with the person's physical features, thereby improving user satisfaction.

[0188] Optional, in Figure 8 On the basis of the corresponding embodiments, based on the target image, the first information, the second information and the third model, an outfit image of the person in the target image is obtained, including: obtaining the facial shape information, body proportion information and / or fatness information of the person in the target image based on the first information; inputting the target image, the facial shape information, body proportion information and / or fatness information of the person in the target image, and the gender information, skin color information and / or hairstyle information of the person in the target image into the third model; obtaining the outfit image of the person in the target image based on the third model.

[0189] In the embodiment of the present application, the face shape of the character can be divided into, for example, a round face, an oval face, and a long face. The body proportion information can include, for example, the waist-to-shoulder ratio, the leg-to-body ratio, and the upper-to-lower-body ratio. The fatness information can include, for example, the fatness of various parts of the human body, for example, the fatness of the waist and chest. For details, please refer to the relevant description of the fatness of the character above, which will not be repeated here.

[0190] In the embodiment of the present application, the outfit in the outfit image of the person in the target image obtained based on the third model can be an outfit that highly conforms to the person's physical features. In this way, the result of the outfit recommendation can be more accurate and user satisfaction can be improved.

[0191] Optional, in Figure 8Based on the corresponding embodiments, the first information includes, in the target image, the coordinate information of the forehead of the character in the target image, the coordinate information of the chin of the character in the target image, the coordinate information of both sides of the cheeks of the character in the target image, the coordinate information of both sides of the shoulders of the character in the target image, the coordinate information of both sides of the waist of the character in the target image, the coordinate information of both sides of the knees of the character in the target image, and the coordinate information of both sides of the ankles of the character in the target image.

[0192] The coordinate information may include the horizontal coordinate and the vertical coordinate of each human body part in the image. Through the horizontal coordinate and the vertical coordinate of each human body part, the electronic device can obtain the position of each human body part in the target image.

[0193] For example, the coordinate information of the forehead, chin and cheeks of a person can be used to calculate the face shape of the person. Similarly, the position information of all or part of the body parts of the person can be used to calculate the body proportion or fatness and other physical features of the person. The specific calculation process can be referred to the above description and will not be repeated here.

[0194] In the embodiment of the present application, by obtaining relatively detailed and accurate coordinate information of each body part in the target image, it is helpful to analyze the face shape, body proportion, fatness and other physical features of the person in the target image. It also helps to improve the accuracy of outfit recommendations and improve user experience.

[0195] Optional, in Figure 8 On the basis of the corresponding embodiments, the face shape information, body proportion information and / or fatness information of the person in the target image are obtained based on the first information, including: determining the face length information of the person based on the coordinate information of the forehead of the person in the target image and the coordinate information of the chin of the person in the target image; determining the face width information of the person based on the coordinate information of both sides of the cheeks of the person in the target image; calculating the ratio of the face length information of the person to the face width information of the person to determine the face shape information of the person in the target image; determining the height proportion information of the person in the target image based on the coordinate information of both sides of the shoulders of the person in the target image and the coordinate information of both sides of the ankles of the person in the target image; determining the fatness proportion information of the person in the target image based on the coordinate information of both sides of the waist of the person in the target image and the coordinate information of both sides of the knees of the person in the target image; calculating the ratio of the height proportion information of the person to the fatness proportion information of the person to determine the body proportion information of the person in the target image.

[0196] Among them, the face shape information may include the aspect ratio of the face, that is, the face shape information can be understood as the aspect ratio of the face mentioned above. For details, please refer to the relevant description of calculating the aspect ratio of the face mentioned above, which will not be repeated here. Specifically, the electronic device can determine the face length information of the person based on the difference between the ordinate in the coordinate information of the forehead of the person and the ordinate of the coordinate information of the chin of the person. Similarly, the electronic device can also determine the face width information of the person based on the difference between the abscissa in the coordinate information of the two sides of the cheeks of the person.

[0197] It should be noted that the above process of determining facial shape information based on the forehead, chin and cheeks of a person is only an exemplary description. In actual application, the human body parts for calculating facial shape information may not be limited to these fixed parts, but may be flexibly adjusted according to needs to adapt to the diverse characteristics of different users.

[0198] In addition, the embodiment of the present application can also calculate the body proportion information of the person. Specifically, the electronic device can calculate the center position of the shoulder based on the coordinate information of both sides of the shoulder in the target image, and can calculate the center position of the ankle based on the coordinate information of both sides of the ankle in the target image. Further, the height proportion information of the person is determined based on the difference between the vertical coordinates of the two center positions.

[0199] Similarly, the electronic device can use the difference between the horizontal coordinates of the coordinate information of the two sides of the waist in the target image as the waist circumference of the person, and use the difference between the horizontal coordinates of the coordinate information of the two sides of the knees in the target image as the knee circumference of the person. Further, the electronic device can determine the fat-to-thin ratio information of the person based on the waist circumference and the knee circumference.

[0200] Furthermore, the electronic device may determine the body proportion information of the person based on the ratio of the height proportion information and the fat-thin proportion information.

[0201] It should be noted that the above method of determining the body proportion information of the person in the target image through the height ratio information and the fat-thin ratio information is only an exemplary description. In the actual application process of the embodiment of the present application, other body proportion information may also be included, such as waist-to-shoulder ratio, leg-to-body ratio and upper-to-lower-body ratio. The calculation method of these body proportions can refer to the relevant description of the calculation of waist-to-shoulder ratio, leg-to-body ratio and upper-to-lower-body ratio in the above text, which will not be repeated here.

[0202] In the embodiment of the present application, the facial shape information and body proportion information of the person can be determined based on the coordinate information of each body part of the person. In this way, the outfit image generated by the electronic device can be more consistent with the facial shape and body proportion of the person, thereby improving the accuracy of the outfit image recommended to the user and enhancing the user experience.

[0203] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0204] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to achieve the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily appreciate that, in combination with the method steps of each example described in the embodiment disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0205] The embodiment of the present application can divide the functional modules of the device implementing the method according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0206] like Fig. 9 FIG. 9 is a schematic diagram of a chip structure provided by an embodiment of the present application. The chip 900 includes one or more (including two) processors 901 , a communication line 902 , a communication interface 903 and a memory 904 .

[0207] In some implementations, the memory 904 stores the following elements: executable modules or data structures, or a subset thereof, or an extended set thereof.

[0208] The method described in the above embodiment of the present application can be applied to the processor 901, or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor 901 or an instruction in the form of software. The above processor 901 can be a general-purpose processor (for example, a microprocessor or a conventional processor), a digital signal processor (digital signal processing, DSP), an application specific integrated circuit (application specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates, transistor logic devices or discrete hardware components. The processor 901 can implement or execute the methods, steps and logic block diagrams related to each processing disclosed in the embodiment of the present application.

[0209] The steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. Among them, the software module can be located in a mature storage medium in the field such as a random access memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable read only memory (EEPROM). The storage medium is located in the memory 904, and the processor 901 reads the information in the memory 904 and completes the steps of the above method in combination with its hardware.

[0210] The processor 901 , the memory 904 , and the communication interface 903 may communicate with each other via the communication line 902 .

[0211] In the above embodiments, the instructions stored in the memory for execution by the processor may be implemented in the form of a computer program product, wherein the computer program product may be pre-written in the memory, or may be downloaded and installed in the memory in the form of software.

[0212] The embodiment of the present application also provides a computer program product including one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that includes one or more available media integrated. For example, the available medium may include a magnetic medium (e.g., a floppy disk, a hard disk or a tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0213] The present application also provides a computer-readable storage medium. The methods described in the above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. Computer-readable media may include computer storage media and communication media, and may also include any medium that can transfer a computer program from one place to another. The storage medium may be any target medium that can be accessed by a computer.

[0214] As a possible design, the computer readable medium may include a compact disc read-only memory (CD-ROM), RAM, ROM, EEPROM or other optical disc storage; the computer readable medium may include a magnetic disk storage or other magnetic disk storage device. Moreover, any connection line may also be appropriately referred to as a computer readable medium. For example, if the software is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically using lasers.

[0215] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

Claims

1. An image processing method, applied to an electronic device, characterized in that: The method comprises: Get the target image; Obtaining first information of the target image based on a first model; the first model is used to obtain position information of a human body part in an image input into the first model, the first information including the position information of the human body part in the target image; obtaining second information of the target image based on the second model; the second model is used to obtain gender information, skin color information and / or hairstyle information of a person in the image input into the second model, and the second information includes the gender information, skin color information and / or hairstyle information of the person in the target image; Based on the target image, the first information, the second information and the third model, an outfit image of the person in the target image is obtained; the third model is used to obtain the physical features of the person in the image input into the third model based on the output information of the first model and the output information of the second model, and determine the outfit image of the person based on the physical features of the person.

2. The method according to claim 1, characterized in that: The obtaining first information of the target image based on the first model includes: Inputting the target image into the first model; the first model includes one or more first fusion convolution modules; Based on one or more first fusion convolution modules, feature extraction is performed on the person in the target image to obtain a first feature image output by the first target fusion convolution module; the first target fusion convolution module is any one of the one or more first fusion convolution modules; wherein the size of the first feature image output by the first target fusion convolution module is smaller than the size of the feature image input to the first target fusion convolution module, and the number of channels of the first feature image output by the first target fusion convolution module is greater than the number of channels of the feature image input to the first target fusion convolution module; A convolution operation, a pooling operation, and a full connection operation are performed on the first feature image output by the one or more first fusion convolution modules to obtain a second feature image; the second feature image includes the first information.

3. The method according to claim 2, characterized in that The obtaining the second information of the target image based on the second model includes: Inputting the target image into the second model; the second model includes one or more second fusion convolution modules; Based on one or more second fused convolution modules, feature extraction is performed on the person in the target image to obtain a third feature image output by the second target fused convolution module; the second target fused convolution module is any one of the one or more second fused convolution modules; Performing a convolution operation, a pooling operation, and a full connection operation on the third feature image output by the one or more second fusion convolution modules to obtain a fourth feature image; the fourth feature image includes the second information; Among them, the fully connected operation includes a first fully connected layer, a second fully connected layer and a third fully connected layer, the first fully connected layer is used to output information related to the gender of the person in the target image; the second fully connected layer is used to output information related to the skin color of the person in the target image; the third fully connected layer is used to output information related to the hairstyle of the person in the target image.

4. The method according to claim 3, characterized in that The obtaining, based on the target image, the first information, the second information and the third model, a dressing image of a person in the target image includes: Obtaining face shape information, body proportion information and / or fatness information of the person in the target image based on the first information; Inputting the target image, facial shape information, body proportion information and / or fatness information of the person in the target image, and gender information, skin color information and / or hairstyle information of the person in the target image into the third model; Based on the third model, an outfit image of the person in the target image is obtained.

5. The method according to claim 4, characterized in that The first information includes, in the target image, coordinate information of the forehead of the person in the target image, coordinate information of the chin of the person in the target image, coordinate information of both sides of the cheeks of the person in the target image, coordinate information of both sides of the shoulders of the person in the target image, coordinate information of both sides of the waist of the person in the target image, coordinate information of both sides of the knees of the person in the target image, and coordinate information of both sides of the ankles of the person in the target image.

6. The method according to claim 5, characterized in that Obtaining face shape information, body proportion information, and / or fatness information of the person in the target image based on the first information includes: Determine the face length information of the person based on the coordinate information of the forehead of the person in the target image and the coordinate information of the chin of the person in the target image; Determining face width information of the person based on coordinate information of both sides of the person's cheeks in the target image; Calculating the ratio of the face length information of the person to the face width information of the person to determine the face shape information of the person in the target image; Determine the height ratio information of the person in the target image based on the coordinate information of the shoulders of the person in the target image and the coordinate information of the ankles of the person in the target image; Determine fat-thin ratio information of the person in the target image based on coordinate information of both sides of the person's waist in the target image and coordinate information of both sides of the person's knees in the target image; The ratio of the height ratio information of the character to the fat-to-thin ratio information of the character is calculated to determine the body ratio information of the character in the target image.

7. An electronic device, characterized in that: The electronic device comprises: one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the electronic device to execute the method according to any one of claims 1 to 6.

8. A chip system, characterized in that: The chip system is applied to an electronic device, and the chip system includes one or more processors, and the one or more processors are used to call computer instructions so that the electronic device executes the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises a computer program code, and when the computer program code is run on an electronic device, the electronic device executes the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Individualized outfit recommendation method and system

    CN106951448A

  • Dressing and collocation recommendation method and device, as well as server and storage medium

    CN107993131A

  • Beauty makeup fitting recommendation method and device

    CN112560540A

  • Garment style matching algorithm based on face features

    CN115482577A

  • Clothing customization algorithm based on human body three-dimensional characteristics

    CN116310069A

Cited By

  • Garment matching recommendation method based on physical characteristics

    CN122064833A