Image processing method, device, apparatus and computer readable medium

Abstract

The embodiments of the present disclosure disclose a method, apparatus, device and computer-readable medium for image processing. A specific implementation of the method includes: determining at least one nail image in a target image; for each nail image in the at least one nail image, determining the category of the nail image; for each category of nail images, determining relevant information of the nail image, wherein the relevant information includes nail key points and nail areas. Since the nail image is accurate, the nail-related information determined based on the nail image is also accurate.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, apparatus, device, and computer-readable medium for image processing. Background Art

[0002] At present, adding special effects to images is a very popular field according to user needs. For example, some special effects can be added to the user's nails. For example, the user's nails can be dyed, and the user's nails can be made longer. Among them, in order to make the image special effects work well, it is key to accurately find the nail-related information, which can be nail key points and nail areas. Summary of the invention

[0003] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0004] Some embodiments of the present disclosure provide an image processing method, apparatus, device, and computer-readable medium.

[0005] In a first aspect, some embodiments of the present disclosure provide a method for image processing, the method comprising: determining at least one nail image in a target image; for each nail image in the at least one nail image, determining a category of the nail image; for each category of nail image, determining relevant information of the nail image, wherein the relevant information includes nail key points and nail areas.

[0006] In a second aspect, some embodiments of the present disclosure provide an image processing device, comprising: a first determination unit, configured to determine at least one nail image in a target image; a second determination unit, configured to determine, for each nail image in the at least one nail image, a category of the nail image; and a third determination unit, configured to determine, for each category of nail images, relevant information of the nail image, wherein the relevant information includes nail key points and nail areas.

[0007] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any method as described in the first aspect.

[0008] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, any method in the first aspect is implemented.

[0009] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: at least one nail image is determined in the target image; the above-mentioned determination of at least one nail image in the target image lays the foundation for the next step of determining the category of the nail image and the related information of the nail image. For each nail image in the above-mentioned at least one nail image, the category of the above-mentioned nail image is determined; by determining the category of the nail image, the related information can be obtained on the premise of refining the above-mentioned nail image. For each category of nail images, the related information of the above-mentioned nail image is determined, wherein the above-mentioned related information includes nail key points and nail areas. Here, since the nail image is accurate, correspondingly, the nail-related information determined based on the nail image is also accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0011] Figure 1 is a schematic diagram of an application scenario of an image processing method according to some embodiments of the present disclosure;

[0012] Figure 2 is a schematic diagram of an application scenario of an image processing method according to some embodiments of the present disclosure;

[0013] Figure 3 are flowcharts of some embodiments of the image processing method according to the present disclosure;

[0014] Figure 4 are flowcharts of other embodiments of the image processing method according to the present disclosure;

[0015] Figure 5 is a schematic structural diagram of some embodiments of an image processing apparatus according to the present disclosure;

[0016] Figure 6 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 It is a schematic diagram of an application scenario of the image processing method according to some embodiments of the present disclosure.

[0024] like Figure 1 As shown in the figure 100, the electronic device 101 can determine at least one nail image 103 in a given target image 102, wherein the target image 102 includes a camera, a cup, a pen, and a hand. Here, the number of the at least one nail can be any number. For example, there are 5 nails in the at least one nail image 103. Figure 2 ,like Figure 2 As shown in the mark 200, based on the five nails in at least one nail image 201, the electronic device 101 can determine the nail image. Here, the nail image can be as follows Figure 2The little finger nail image 202, the ring finger nail image 203, the middle finger nail image 204, the index finger nail image 205, and the thumb nail image 206 are included in the image. Finally, the nail related information is determined according to the nail image. The above related information includes nail key points and nail areas.

[0025] It is understandable that the image processing method can be performed by the above-mentioned electronic device 101. Among them, the electronic device 101 can be hardware or software. When the electronic device 101 is hardware, it can be various electronic devices with information processing capabilities, including but not limited to smart phones, tablet computers, e-book readers, laptop computers, desktop computers, servers, etc. When the electronic device 101 is software, it can be installed in the electronic devices listed above. It can be implemented as multiple software or software modules for providing distributed services, for example, or it can be implemented as a single software or software module. No specific limitation is made here.

[0026] It should be understood that Figure 1 The number of electronic devices in the embodiment is only for illustration. Any number of electronic devices may be provided according to implementation requirements.

[0027] Continue to refer Figure 3 , shows a process 300 of some embodiments of the image processing method according to the present disclosure. The image processing method comprises the following steps:

[0028] Step 301: determine at least one nail image in a target image.

[0029] In some embodiments, the above execution entities (e.g. Figure 1 The electronic device shown in the figure can determine at least one nail image in various ways. The target image can be any image. In practice, it can be an image including nails.

[0030] In some optional implementations of some embodiments, the target image is input into a pre-trained nail detection model to obtain at least one position information of the at least one nail image; based on the at least one position information, the target image is cropped to obtain the at least one nail image.

[0031] Among them, the above-mentioned nail detection model can be any target detection model derived from a convolutional neural network, and the executor of the above-mentioned method can crop the above-mentioned target image in various ways to obtain the above-mentioned at least one nail image.

[0032] As an example, an image containing a ring, a mobile phone, and a hand is input into the residual neural network, and the fingernails of the hand in the output image will be annotated with a labeling box to obtain at least one annotated fingernail image.

[0033] In practice, the sample data used in the above-mentioned nail detection model can be obtained in various ways. Among them, the above-mentioned sample data can be an image including nails. In practice, the sample data can be obtained by taking pictures with a mobile phone. The sample data can also be obtained from a database. Here, the sample data includes pictures showing nails in various scenes (for example, indoors and outdoors). At the same time, the sample data also shows various shapes, angles and appearances that may appear on the hands. The Adam optimizer can be used to train the nail detection model. At the same time, the hyperparameters in the training can be set. For example, the learning rate can be set to 0.01, and the penalty for the weight can be set to 10 -5 The learning rate decrease trend can use a multi-step curve, and the number of samples per round of training can be 96.

[0034] Step 302: for each nail image in at least one nail image, determine the category of the nail image.

[0035] In some embodiments, for each nail image in at least one nail image, the execution subject may determine the category of the nail image in a variety of ways, wherein the categories may include: ring finger nail, middle finger nail, index finger nail, pinky finger nail, and thumb nail.

[0036] In some optional implementations of some embodiments, the output of the nail detection model includes multiple probability maps, which are used to characterize the category information of the nail image; and the above-mentioned determination of the category of the nail image includes: determining the category corresponding to the nail image based on the multiple probability maps. Among them, the nail detection model can be any deep neural network derived from the convolutional neural network. The above-mentioned probability map can be a matrix. The value at each position in the matrix is ​​between 0-1, indicating the probability that the position has a nail. Here, a threshold value can be set, such as 0.6. In response to the presence of a position with a probability value greater than 0.6 in the matrix, it can be considered that this place is a nail. In this way, the approximate position P of the nail can be preliminarily determined. At the same time, the model will also output an offset relative to the position P and the length and width of the nail annotation box. Among them, the above-mentioned offset is used to fine-tune the approximate position P of the nail, so that the nail position changes from an approximate position to a more accurate position. Finally, the frame including the nail image is determined. That is, the position information of the nail image is determined. At the same time, the nail model detection will also output the corresponding confidence map of the nail category (i.e., the probability of thumb, index finger, middle finger, ring finger, pinky finger, and background), which is also a value between 0 and 1. When the confidence of the nail category is greater than a predetermined threshold, it indicates that there is a nail at that location. At the same time, based on the corresponding confidence map, a category corresponding to the maximum confidence is determined, thereby obtaining the category of the detected nail image.

[0037] Step 303: for each category of nail images, determine relevant information of the nail images, wherein the relevant information includes nail key points and nail regions.

[0038] In some embodiments, for each category of nail images, the execution subject may determine the relevant information of the nail image in various ways. The relevant information may include nail key points and nail areas. Here, the nail key points may be used to characterize the position information of the nail. In practice, there may be multiple nail key points, for example, 8.

[0039] As an example, the execution subject may find nail key points and nail regions from the nail image by manual labeling.

[0040] In some optional implementations of some embodiments, the above-mentioned execution entity can input the above-mentioned nail image into a pre-trained nail key point detection and segmentation model to generate the above-mentioned fingernail area and fingernail key points. Here, the nail key point detection and segmentation model can be roughly divided into two modules logically. Among them, the function of one module is nail key point detection. The other module is nail segmentation. Among them, the above-mentioned nail key point detection and segmentation model is trained on the basis of the initial deep neural network. The above-mentioned initial deep neural network may include: a lightweight network MobileNet, a lightweight network ShuffleNet.

[0041] The sample data used in the nail key point detection module can be obtained in various ways. Among them, the above sample data can be an image including nails. In practice, the sample data can be obtained by taking a photo with a mobile phone. The sample data can also be obtained from a database. Here, the sample data includes pictures showing nails in various scenes (for example, indoors and outdoors). At the same time, the sample data also shows various forms, angles and appearances that nails may appear in.

[0042] After collecting the nail pictures, you can mark the nail pictures. In practice, it is necessary to mark the key points of the nails.

[0043] In practice, the Adam optimizer can be used to train the nail key point detection module. At the same time, the hyperparameters in the training can be set. For example, the learning rate can be set to 0.01, and the penalty coefficient of the weight can be set to 10. -5 The learning rate decrease trend can use a multi-step curve, and the number of samples per round of training can be 96.

[0044] The process of the method for detecting nail key points using the above-mentioned nail key point detection module can be roughly summarized as follows: the nail image is input into the nail key point detection module, and a probability map of each position in the above-mentioned nail image being a nail key point is obtained. The probability map can be a matrix. The value at each position in the matrix is ​​between 0 and 1, indicating the probability that the position has a nail key point. In practice, each position of the nail image can be traversed to find the position whose probability value meets the preset threshold. These positions are determined as nail key points.

[0045] The sample data used in the nail segmentation module can be obtained in various ways. Among them, the above sample data can be an image including nails. In practice, the sample data can be obtained by taking a photo with a mobile phone. The sample data can also be obtained from a database. Here, the sample data includes pictures showing nails in various scenes (for example, indoors and outdoors). At the same time, the sample data also shows various shapes, angles and appearances that nails may appear in.

[0046] After collecting the nail image, the nail image can be annotated. In practice, it is necessary to annotate the area where the nail is located.

[0047] In practice, the Adam optimizer can be used to train the nail segmentation module. At the same time, the hyperparameters in the training can be set. For example, the learning rate can be set to 0.01, and the penalty for the weight can be set to 10. -7 The learning rate decrease trend can use a multi-step curve, and the number of samples per round of training can be 96.

[0048] The process of using the above-mentioned nail segmentation module can be roughly summarized as follows: Input the nail image into the nail segmentation module, and obtain the probability map of each position in the above-mentioned nail image being a nail area. The probability map can be a matrix. The value at each position in the matrix is ​​between 0 and 1, indicating the probability that the position has a nail area. In practice, each position of the nail image can be traversed to find the position whose probability value meets the preset threshold. These positions are determined as nail areas.

[0049] In some embodiments, the above-mentioned nail detection model, nail key point detection model, nail key point detection and segmentation model include at least one of the following: lightweight network MobileNet, lightweight network ShuffleNet, inverted residuals, and channel exchange shuffle channel.

[0050] In some optional implementations of some embodiments, based on the nail key points and the nail area, a preset decoration image is added to the nail area, wherein the preset decoration image may be any image.

[0051] As an example, the nail area may be painted with embellishments such as stars or moons.

[0052] One of the above-mentioned embodiments of the present disclosure has the following beneficial effects: at least one nail image is determined in the target image; the above-mentioned determination of at least one nail image in the target image lays the foundation for the next step of determining the category of the nail image and the related information of the nail image. For each nail image in the above-mentioned at least one nail image, the category of the above-mentioned nail image is determined; by determining the category of the nail image, the related information can be obtained on the premise of refining the above-mentioned nail image. For each category of nail images, the related information of the above-mentioned nail image is determined, wherein the above-mentioned related information includes nail key points and nail areas. Here, since the nail image is accurate, correspondingly, the nail-related information determined based on the nail image is also accurate.

[0053] Further references Figure 4 , which shows a process 400 of another embodiment of the image processing method. The process 400 of the image processing method includes the following steps:

[0054] Step 401: determine at least one nail image in a target image.

[0055] Step 402: for each nail image in the at least one nail image, determine the category of the nail image.

[0056] Step 403: for each category of nail images, determine relevant information of the nail images.

[0057] In some embodiments, the specific implementation of steps 401 to 403 and the technical effects brought about can be referred to Figure 3 Steps 301 to 303 in the corresponding embodiments are not described in detail here.

[0058] Step 404: In response to the nail verification information not meeting the preset condition, the nail key points and the nail area are discarded.

[0059] In some embodiments, in response to the nail verification information not meeting the preset condition, the execution entity may discard the nail key point and the nail area. Here, the nail verification information is used to characterize whether the nail image shows a nail. In practice, the nail verification information may be a probability. When the probability is large, it may be considered that the nail image shows a nail. Here, the preset condition may be greater than a preset threshold. For example, greater than 70%.

[0060] As an example, in response to the nail verification information being no greater than 70%, the nail key point and the nail region are discarded.

[0061] As can be seen from the figure, Figure 3 Compared with the description of some corresponding embodiments, Figure 4 The process 400 of the image processing method in some corresponding embodiments reflects that while generating the nail key points and the above-mentioned nail area, nail verification information is also generated. Here, the obtained nail image can be verified through the nail verification information. Only when the verification information meets the preset conditions, the nail key points and the above-mentioned nail area corresponding to the nail image are retained. Vice versa. Through the verification information, the influence of non-nail areas on the final result can be effectively avoided. By adding the verification information, the credibility of the final nail key points and the above-mentioned nail area is higher.

[0062] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an image processing device. These device embodiments are similar to Figure 3 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0063] like Figure 5 As shown, the image processing apparatus 500 of some embodiments includes: a first determination unit 501, a second determination unit 502 and a third determination unit 503. The first determination unit 501 is configured to determine at least one nail image in the target image; the second determination unit 502 is configured to determine the category of each nail image in the at least one nail image; the third determination unit 503 is configured to determine the relevant information of the nail image for each category of nail images, wherein the relevant information includes nail key points and nail regions.

[0064] In some optional implementations of some embodiments, the nail-related information further includes nail verification information. And the method further includes a discarding unit. The discarding unit is configured to discard the nail key point and the nail area in response to the nail verification information not meeting a preset condition.

[0065] In some optional implementations of some embodiments, the first determination unit 501 of the image processing device 500 is further configured to input the above-mentioned target image into a pre-trained nail detection model to obtain at least one position information of the above-mentioned at least one nail image; based on the above-mentioned at least one position information, the above-mentioned target image is cropped to obtain the above-mentioned at least one nail image.

[0066] In some optional implementations of some embodiments, the output of the nail detection model further includes multiple probability maps, which are used to characterize category information of the nail image; and the second determination unit 502 is further configured to determine the category corresponding to the nail image based on the multiple probability maps.

[0067] In some optional implementations of some embodiments, the third determination unit 503 may be further configured to input the nail image into a pre-trained nail key point detection and segmentation model to generate the fingernail region and fingernail key points.

[0068] In some optional implementations of some embodiments, the apparatus 500 further includes an adding unit, wherein the adding unit is configured to add a preset decoration image on the nail region based on the nail key points and the nail region.

[0069] It is understood that the units described in the device 500 are similar to those described in the reference Figure 3 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 500 and the units included therein, and will not be described in detail here.

[0070] Reference below Figure 6 , which shows an electronic device (eg, Figure 1 A schematic diagram of the structure of an electronic device (electronic device in FIG. 600). Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0071] like Figure 6 As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 605 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0072] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0073] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0074] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0075] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0076] The computer-readable medium may be included in the device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs, and determines at least one nail image in the target image; for each nail image in the at least one nail image, determines the category of the nail image; for each category of nail image, determines the relevant information of the nail image, wherein the relevant information includes nail key points and nail areas.

[0077] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0078] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0079] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a first determination unit, a second determination unit, and a third determination unit. The names of these units do not constitute limitations on the units themselves in some cases, for example, the first determination unit may also be described as "a unit for determining at least one nail image in a target image".

[0080] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0081] According to one or more embodiments of the present disclosure, there is provided a method for image processing, comprising: determining at least one nail image in a target image; for each nail image in the at least one nail image, determining a category of the nail image; for each category of nail images, determining relevant information of the nail image, wherein the relevant information includes nail key points and nail areas.

[0082] According to one or more embodiments of the present disclosure, the nail related information further includes: nail verification information. In response to the nail verification information not meeting a preset condition, the nail key point and the nail region are discarded.

[0083] According to one or more embodiments of the present disclosure, the target image is input into a pre-trained nail detection model to obtain at least one position information of the at least one nail image; based on the at least one position information, the target image is cropped to obtain the at least one nail image.

[0084] According to one or more embodiments of the present disclosure, the output of the nail detection model further includes a plurality of probability maps, which are used to characterize category information of the nail image; and based on the plurality of probability maps, the category corresponding to the nail image is determined.

[0085] According to one or more embodiments of the present disclosure, the above-mentioned nail image is input into a pre-trained nail key point detection and segmentation model to generate the above-mentioned fingernail region and fingernail key points.

[0086] According to one or more embodiments of the present disclosure, based on the nail key points and the nail area, a preset decoration image is added to the nail area.

[0087] According to one or more embodiments of the present disclosure, there is provided an image processing apparatus, comprising: a first determination unit, configured to determine at least one nail image in a target image; a second determination unit, configured to determine, for each nail image in the at least one nail image, a category of the nail image; and a third determination unit, configured to determine, for each category of nail images, relevant information of the nail image, wherein the relevant information includes nail key points and nail areas.

[0088] According to one or more embodiments of the present disclosure, the nail-related information further includes nail verification information. The method further includes a discarding unit. The discarding unit is configured to discard the nail key points and the nail area in response to the nail verification information not meeting a preset condition.

[0089] According to one or more embodiments of the present disclosure, the first determination unit of the image processing device is further configured to input the above-mentioned target image into a pre-trained nail detection model to obtain at least one position information of the above-mentioned at least one nail image; based on the above-mentioned at least one position information, the above-mentioned target image is cropped to obtain the above-mentioned at least one nail image.

[0090] According to one or more embodiments of the present disclosure, the output of the nail detection model further includes multiple probability maps, which are used to characterize category information of the nail image; and the second determination unit is further configured to determine the category corresponding to the nail image based on the multiple probability maps.

[0091] According to one or more embodiments of the present disclosure, the third determination unit may be further configured to input the nail image into a pre-trained nail key point detection and segmentation model to generate the fingernail region and fingernail key points.

[0092] According to one or more embodiments of the present disclosure, the apparatus further includes an adding unit, wherein the adding unit is configured to add a preset decoration image on the nail area based on the nail key points and the nail area.

[0093] According to one or more embodiments of the present disclosure, an electronic device is provided, including: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0094] According to one or more embodiments of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, wherein when the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0095] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A method for image processing, comprising: determining at least one nail image in the target image; For each of the at least one nail image, determining a category of the nail image, wherein the categories of the nail image include a ring finger nail, a middle finger nail, an index finger nail, a pinky finger nail, and a thumb nail, and the category of the nail image is determined by a nail detection model, wherein an output of the nail detection model includes a plurality of probability maps, and the probability maps are used to represent category information of the nail image; For each category of nail images, determine the nail key points and nail regions of the nail image; wherein the nail key points are used to characterize the position information of the nails; Based on the nail key points and the nail area, adding a preset decoration image on the nail area; The output of the nail detection model further includes a corresponding confidence map of the nail type, the probability map is a matrix, and determining the category of the nail image includes: Based on the multiple probability maps, determining a position of a nail in the nail image; In response to the presence of a first position having a probability value greater than a first threshold in the matrix, determining that the first position is a position of a nail; In response to the confidence level of the nail type being greater than a second threshold, determining that there is a nail at the first position; Based on the confidence map, a category corresponding to the nail at the first position in the nail image is determined.

2. The method according to claim 1, wherein: Nail-related information also includes: nail verification information; and The method further comprises: In response to the nail verification information not meeting a preset condition, the nail key point and the nail area are discarded.

3. The method according to claim 1, wherein: The step of determining at least one nail image in the target image comprises: Inputting the target image into a pre-trained nail detection model to obtain at least one position information of the at least one nail image; Based on the at least one piece of position information, the target image is cropped to obtain the at least one nail image.

4. The method according to claim 1, wherein: For each category of nail images, determining relevant information of the nail images includes: The nail image is input into a pre-trained nail key point detection and segmentation model to generate the nail region and the nail key points.

5. An image processing device, comprising: A first determining unit, configured to determine at least one nail image in the target image; a second determining unit, configured to determine, for each of the at least one nail image, a category of the nail image, wherein the categories of the nail image include a ring finger nail, a middle finger nail, an index finger nail, a pinky finger nail, and a thumb nail, wherein the category of the nail image is determined by a nail detection model, wherein an output of the nail detection model includes a plurality of probability maps, and wherein the probability maps are used to represent category information of the nail image; A third determining unit is configured to determine, for each category of nail images, nail key points and nail regions of the nail image; wherein the nail key points are used to characterize position information of the nail; and based on the nail key points and the nail region, add a preset decoration image to the nail region; The output of the nail detection model further includes a map of corresponding confidences of the nail types, the probability map is a matrix, and the second determination unit is further configured to: Based on the multiple probability maps, determining a position of a nail in the nail image; In response to the presence of a first position having a probability value greater than a first threshold in the matrix, determining that the first position is a position of a nail; In response to the confidence level of the nail type being greater than a second threshold, determining that there is a nail at the first position; Based on the confidence map, a category corresponding to the nail at the first position in the nail image is determined.

6. An electronic device comprising: one or more processors; A storage device having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 4.

7. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images