Method, device and server for identifying makeup color number of facial object

Abstract

The present application provides a method, device and server for identifying the makeup color number of a facial object. When a video is played on a video playback interface, if an instruction for identifying the makeup color number of a facial object is detected, the target video image currently displayed on the video playback interface is determined; the target face of the facial object of the makeup color number to be identified is determined from the faces contained in the target video image; and the video clip corresponding to the target face is obtained. Since the video clip corresponding to the target face includes multiple continuous frames of video images, the video clip is very likely to be continuously shot by the actor to whom the target face belongs without changing the makeup model of the facial object. Therefore, the target color value of the facial object of the target face in each frame of the video image contained in the video clip is the same, and the target makeup color number determined based on the video clip is more accurate. The user can obtain the target makeup color number by inputting the above instruction while watching the video, which makes the operation simple and ensures real-time performance.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a makeup color number recognition method, device and server for facial objects. Background Art

[0002] When watching movies and TV shows, users not only pay attention to the plot, but also often have great interest in the makeup of the actors, such as the color of the actor's lipstick, the color of the eyebrow pencil, the color of the eye contact lens, the color of the blush, and other facial makeup colors.

[0003] At present, the method for obtaining the makeup color number of an actor's face object is that after watching a film or TV work, the user enters the name of the corresponding film or TV work and the name of the actor in the browser to search for the makeup color number of the actor. As a result, the process of obtaining the makeup color number of the actor's face object is cumbersome and cannot be achieved in real time. Summary of the invention

[0004] In view of this, the present application provides a method, device and server for identifying the makeup color number of a facial object to solve the problem that the makeup color number of the actor's facial object cannot be fed back to the user in a timely manner, and the process of obtaining the makeup color number of the actor's facial object is cumbersome.

[0005] To achieve the above objectives, this application provides the following technical solutions:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for identifying a makeup color number of a facial object is provided, comprising:

[0007] When a video is played on a video playback interface, if an instruction to identify a makeup color number of a facial object is detected, a target video image currently displayed on the video playback interface is determined, and the facial object is a part of a human face decorated with makeup;

[0008] Determine a target human face to which the facial object of the makeup color number to be identified belongs from the human faces included in the target video image;

[0009] Acquire a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image;

[0010] For each video image in each frame of the video image, a facial object image composed of a local part of a facial object included in the target face is obtained from the video image to obtain facial object images corresponding to each frame of the video image;

[0011] Extracting target color values ​​of the facial object images corresponding to the respective frames of video images;

[0012] Based on the target color values ​​corresponding to the facial object images corresponding to the respective frames of video images, a target makeup color number corresponding to the facial object in the target face is determined.

[0013] According to a second aspect of an embodiment of the present disclosure, a makeup color number recognition device for a facial object is provided, comprising:

[0014] A first determination module is used to determine the target video image currently displayed on the video playback interface when a video is played on the video playback interface, if an instruction to identify the makeup color number of a facial object is detected, and the facial object is a part of the human face decorated with makeup;

[0015] A second determination module is used to determine a target face to which the facial object of the makeup color number to be identified belongs from the faces included in the target video image;

[0016] A first acquisition module is used to acquire a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image;

[0017] A second acquisition module is used to acquire, for each video image in each frame of the video image, a facial object image composed of a local part of a facial object included in the target face, so as to obtain facial object images corresponding to each frame of the video image;

[0018] An extraction module, used for extracting target color values ​​of the facial object images corresponding to the respective frames of video images;

[0019] The third determination module is used to determine the target makeup color number corresponding to the facial object in the target face based on the target color value corresponding to the facial object image corresponding to each frame of the video image.

[0020] According to a third aspect of an embodiment of the present disclosure, a server is provided, comprising:

[0021] processor;

[0022] a memory for storing instructions executable by the processor;

[0023] Wherein, the processor is configured to execute the instructions to implement the makeup color number recognition method for a facial object as described in the first aspect.

[0024] Through the above technical scheme, it can be known that in the makeup color number recognition method for facial objects provided by the present application, when a video is played on a video playback interface, if an instruction to recognize the makeup color number of a facial object is detected, it means that there is a face decorated with the makeup color number that the user is interested in in the target video image currently displayed on the video playback interface; the target face to which the facial object with the makeup color number to be recognized belongs is determined from the faces contained in the target video image; and the video clip corresponding to the target face is obtained. Since the video clip corresponding to the target face includes multiple consecutive video frames, and the multiple consecutive video frames all include the target face, the video clip is very likely to be continuously shot by the actor to whom the target face belongs without changing the makeup color number of the facial object. Therefore, the target color value of the facial object of the target face in each frame of the video image contained in the video clip is the same. Based on the target attribute value of the facial object in the target face in each frame of the video image in the video clip, the target makeup color number of the facial object determined is more accurate. There is no need for users to search in the browser themselves. They can input the instructions of the makeup color number of the facial object while watching the video, and then execute the makeup color number recognition method of the facial object provided by the present application to provide the user with the target makeup color number, making the process of obtaining the makeup color number of the actor's facial object simple to operate, and the target makeup color number can be obtained while watching the video, ensuring real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0026] Figure 1 A schematic diagram of the hardware architecture involved in the embodiments of the present application;

[0027] Figure 2 A flowchart of a makeup color number recognition method for a facial object provided in an embodiment of the present application;

[0028] Figure 3 A schematic diagram of multiple key points detected by a facial key point detection method;

[0029] Figure 4 A schematic diagram of video clips corresponding to faces provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of a target area when the facial object is lips as an example provided in an embodiment of the present application;

[0031] Figure 6A structural diagram of a makeup color number recognition device for a facial object provided in an embodiment of the present application;

[0032] Figure 7 The invention is a block diagram showing a device for a server according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The embodiments of the present application provide a method, device and server for identifying makeup color numbers of facial objects. Before introducing the embodiments of the present application, the hardware architecture involved in the present application is first described.

[0035] like Figure 1 , which is a schematic diagram of a hardware architecture involved in an embodiment of the present application, the hardware architecture includes an electronic device 11 and a server 12.

[0036] Exemplarily, the electronic device 11 can be any electronic product that can interact with a user through one or more methods such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as a mobile phone, a laptop computer, a tablet computer, a PDA, a personal computer, a wearable device, a smart TV, a PAD, etc.

[0037] It should be noted that Figure 1 This is just an example. There are many types of electronic devices, not limited to Figure 1 Smartphone in the.

[0038] Exemplarily, the server 12 may be a single server, or a server cluster consisting of multiple servers, or a cloud computing server center. The server 12 may include a processor, a memory, a network interface, and the like.

[0039] Exemplarily, the electronic device 11 may establish a connection and communicate with the server 12 via a wireless communication network.

[0040] The electronic device 11 runs a client, which may be an application client or a web client.

[0041] Exemplarily, the type of the client may be: video playback type, live broadcast type.

[0042] Figure 1The description is made by taking the live broadcast type of the client running in the electronic device 11 as an example.

[0043] The user can watch videos through the client running on the electronic device 11. For example, the video can be a movie or TV series video, a live video, an advertising video, a short video, etc.

[0044] When watching a video, if the user is interested in the makeup color number of a certain actor's facial object in the video, the user can input an instruction to identify the makeup color number of the facial object. The electronic device 11 sends the instruction to identify the makeup color number of the facial object to the server 12. The server 12 executes the makeup color number recognition method for the facial object provided in the embodiment of the present application to obtain the target makeup color number of the facial object, and feeds it back to the electronic device 11. The electronic device 11 can display it to the user, so that the user can check the target makeup color number of the facial object in time without searching in the browser, making the operation simple. The target makeup color number of the facial object can be known during the video viewing process, ensuring real-time performance.

[0045] Exemplarily, the electronic device 11 may display the target makeup color number to the user in the form of a barrage; or, the electronic device 11 may receive a text message sent by the server 12, which includes the target makeup color number, and display it to the user in the form of a text message, or, the electronic device 11 may receive a comment content sent by the server 12, which includes the target makeup color number, and display it to the user in the form of a comment content.

[0046] Exemplarily, the face object is located on the face of a person, and the face object is a part of the face that is decorated with makeup. The face object can be at least one of: lips, eyebrows, eyes, cheeks, and eyelashes, and is not specifically limited to the implementation of the solution. If the face object is lips, the makeup color number of the face object is the lipstick color number, if the face object is eyebrows, the makeup color number of the face object is the eyebrow pencil color number, if the face object is eyes, the makeup color number of the face object is the contact lens color number, if the face object is eyelashes, the makeup color number of the face object is the mascara color number, and if the face object is cheeks, the makeup color number of the face object is the blush color number.

[0047] Exemplarily, there are many ways for a user to input instructions for identifying the makeup color number of a facial object, such as voice input, text input, or a touch operation that meets preset conditions, and there is no specific limitation on what can be implemented.

[0048] like Figure 1 As shown, the user inputs the instruction of the makeup color number of the face object in the form of text input, such as "I want the lipstick color number of a girl". In this case, the face object is lips, the makeup color number of the face object is the lipstick color number, and the face object belongs to a person. Figure 1The girl in the video image shown, that is, the target face is the girl in the video image.

[0049] Exemplarily, the text input may be a barrage input, or a comment input.

[0050] Exemplarily, the input of the touch operation that meets the preset conditions may be a preset duration for touching one or more set keys, or the touch trajectory may be the same as the set trajectory.

[0051] Those skilled in the art should understand that the above-mentioned electronic devices and servers are only examples, and other existing or future electronic devices or servers that are applicable to the present disclosure should also be included in the protection scope of the present disclosure and are included here by reference.

[0052] The makeup color number recognition method for facial objects provided by the embodiment of the present application is described below in combination with the above-mentioned hardware architecture.

[0053] like Figure 2 As shown, it is a flowchart of the makeup color number recognition method for facial objects provided in an embodiment of the present application. The method can be applied to the server in the above-mentioned hardware architecture. The method involves the following steps S21 to S26 during implementation.

[0054] Step S21: When playing a video on a video playback interface, if an instruction to identify the makeup color number of a facial object is detected, the target video image currently displayed on the video playback interface is determined, and the facial object is a part of the human face decorated with makeup.

[0055] Exemplarily, the instruction for identifying the makeup color number of the facial object may be obtained from the bullet comments, or from the comment content, and is not specifically limited to what can be implemented in the solution.

[0056] In an optional implementation, if an instruction to identify the makeup color number of a facial object is detected, it is considered that the user is interested in the makeup color number of the facial objects of a certain actor or multiple actors in the target video image displayed on the video playback interface at the moment the instruction is detected, so the video image currently displayed on the video playback interface is determined as the target video image.

[0057] Exemplarily, the video playback interface is an interface in a client running on the electronic device 11. The electronic device 11 may send the target video image to the server 12.

[0058] Step S22: determining a target human face to which the facial object of the makeup color number to be identified belongs from the human faces included in the target video image.

[0059] It is understandable that the target video image may include one or more faces. Figure 1The video image shown is a target video image, and the target video image includes two faces.

[0060] In an optional implementation, all the faces included in the target video image may be determined as target faces, that is, the makeup color numbers of the facial objects in all the faces included in the target video image need to be identified.

[0061] In an optional implementation, the instruction for identifying the makeup color number of the face object may include: the name of the actor to whom the target face belongs. For example, if the input comment is: Wow, the lipstick color of actor A is so beautiful. Then the target face is the face of actor A contained in the target video image.

[0062] Step S23: obtaining a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image.

[0063] Each frame of the video image includes the target face.

[0064] There are many ways to implement step S23, and the embodiments of the present application provide but are not limited to the following two.

[0065] The first implementation of step S23 includes the following steps A11 to A15.

[0066] Step A11: Determine the position number of the target video image in the video.

[0067] The position number is the order in which the target video image is located in the video. For example, if the target video image is the 5th frame of the video, the position number of the target video image is 5.

[0068] Step A12: obtaining a target face identifier corresponding to the target face contained in the target video image having the position number from the video information corresponding to the video.

[0069] Step A13: Obtain the target position number of the video image having the face corresponding to the target face identifier included in the video information.

[0070] Step A14: obtaining a video segment corresponding to the target face having the target face identifier, which is composed of a video image having the target position number and the target video image.

[0071] Among them, the face identifiers of faces belonging to the same actor in two frames of video images with adjacent position numbers in the video information are the same, the face identifiers of faces belonging to the same actor in two frames of video images with non-adjacent position numbers are different, and the face identifiers of faces belonging to different actors are different.

[0072] Since the face identifiers of the faces of the same actor in two frames of video images with adjacent position numbers in the video information are the same, the face identifiers of the faces of the same actor in two frames of video images with non-adjacent position numbers are different, and the face identifiers of the faces of different actors are different, the target position numbers of the video images with the faces corresponding to the target face identifiers obtained in step A13 are continuous position numbers. That is, the video image frames contained in the video clip corresponding to the target face are continuous multiple frames of video images.

[0073] The following describes a method for obtaining video information of a video.

[0074] Exemplarily, for each video, the video information of the video may be obtained in advance, and the specific method includes the following steps A21 to A25.

[0075] Step A21: Initialize i=1.

[0076] Step A22: Obtain the i-th frame of video image from the video.

[0077] Step A23: Detect whether there is a face in the i-th frame of video image. If not, set i=i+1 and return to step A22. If so, execute step A24.

[0078] Exemplarily, whether a face exists in the i-th frame of video image can be determined by a method of detecting facial key points.

[0079] Exemplary methods for detecting key points of a face include but are not limited to the following methods: ASM (Active Shape Model) algorithm, or AAM (Active Appearance Models) algorithm, or a cascaded shape regression CPR (Cascaded pose regression) model, or a method based on deep learning.

[0080] Facial key point detection refers to locating the key areas of the face, including eyebrows, eyes, nose, mouth, facial contours, etc.

[0081] like Figure 3 , which is a schematic diagram of multiple key points detected by the face key point detection method.

[0082] like Figure 3 As shown in Figure 2, multiple key points in the face can be detected by the face key point detection method. Figure 3The black solid dots represent key points. The position of the corresponding facial object can be determined by the position of the key points. For example, the position of the eyebrows can be determined by the key points of the eyebrows, the position of the eyes can be determined by the key points of the eyes, the position of the nose can be determined by the key points of the nose, and the position of the mouth can be determined by the key points of the mouth. The facial area where the face is located can be determined by the key points of the facial contour.

[0083] Exemplarily, the face region of the face can be obtained by sequentially connecting the key points of the face contour. Exemplarily, the face region can be a square region including the face contour, such as Figure 3 shown.

[0084] For example, the number of key points of a face can be Figure 3 The number of key points shown is large, and it can also be compared Figure 3 The number of key points shown is small, which is determined based on actual conditions and is not limited here. For example, the number of key points of a face can be 106.

[0085] Step A24: for each face included in the i-th frame of video image, determine whether the face already exists in the i-1-th frame of video image; if not, execute step A25; if yes, execute step A26.

[0086] Step A25: assigning a new first face identifier to the face included in the i-th frame of video image; and correspondingly storing the position number of the i-th frame of video image, the first face identifier, and the face having the first face identifier located in the face area of ​​the i-th frame of video image.

[0087] Assume that the face area is a rectangular area.

[0088] Then the corresponding format of “the position number of the i-th frame of video image, the first face identifier, and the face with the first face identifier located in the face area of ​​the i-th frame of video image” can be as follows:

[0089] "frame id" (the position number of the video image): 1,

[0090] {“face id”: 11,

[0091] "facebox" (face area): [upper left vertex coordinates, lower left vertex coordinates, upper right vertex coordinates, lower right vertex coordinates],

[0092] "face keypoint": [(x11, y11) (x12, y12), ..., (x1 106 , y1 106 )],

[0093] "face angle": [α1 x , α1 y , α1 z ]};

[0094] }

[0095] It is understandable that, since the facial object needs to be extracted from the face, the coordinates of the key points in the face need to be identified, so that the position of the facial object in the face can be determined. Based on this, step A25 can also include: storing the coordinates of the key points of the face corresponding to the i-th frame of the video image, such as the above-mentioned face keypoint. The above is explained by taking the number of key points of the face as 106 as an example.

[0096] In an optional implementation, since the facial object is located in the face of a person, if the facial angle of the face is large, that is, there is almost no frontal face in the video image, the facial object in the face of the person cannot be obtained. Since the facial angle of the face is related to whether the facial object in the face can be obtained, the facial angle of the face corresponding to the i-th frame of the video image can also be stored, such as the above-mentioned face angle.

[0097] The facial angle of a human face refers to the rotation angle of the human face relative to the front face.

[0098] Step A26: Determine the second face identification ID corresponding to the face included in the (i-1)th frame of video image.

[0099] Step A27: Correspondingly store the position number of the i-th frame of video image, the second face identifier, and the face with the second face identifier located in the face area of ​​the i-th frame of video image; set i=i+1, and return to step A22.

[0100] Assume that the video includes 3 frames of video images. If i=1, n faces are identified from the first frame of video image, and the face identifiers of the n faces are face identifier 11, face identifier 12, ..., face identifier 1n. Then the information obtained about the first frame of video image is as follows:

[0101] ″frame id″: 1,

[0102] {"face id":11,

[0103] ″facebox″: [upper left vertex coordinates, lower left vertex coordinates, upper right vertex coordinates, lower right vertex coordinates],

[0104] "face keypoint": [(x11, y11) (x12, y12), ..., (x1 106 , y1 106 )],

[0105] "face angle": [α1 x , α1 y , α1 z ]};

[0106] {"face id":12,

[0107] ″facebox″: [upper left vertex coordinates, lower left vertex coordinates, upper right vertex coordinates, lower right vertex coordinates],

[0108] "face keypoint": [(x21, y21) (x22, y22), ..., (x2 106 , y2 106 )],

[0109] "face angle": [α2 x , α2 y , α2 z ]};

[0110] };

[0111] , ...,

[0112] {"face id":1n,

[0113] ″facebox″: [upper left vertex coordinates, lower left vertex coordinates, upper right vertex coordinates, lower right vertex coordinates],

[0114] "face keypoint": [(xn1,yn1)(xn2,yn2),...,(xn 106 , yn 106 )],

[0115] ″face angle”: [αn x ,αn y ,αn z ]};

[0116] }

[0117] Assume i=2, if the second frame of video image includes two faces, one of the two faces and the face with face identifier 11 belong to the same actor, and the other face is different from the n faces in the first frame of video image, then the information obtained about the second frame of video image is as follows:

[0118] ″frame id″: 2,

[0119] {"face id":11,

[0120] ″facebox″: [upper left vertex coordinates, lower left vertex coordinates, upper right vertex coordinates, lower right vertex coordinates],

[0121] "face keypoint": [(x211, y211) (x212, y212), ..., (x21 106 ,y21 106 )],

[0122] "face angle": [α21 x , α21 y , α21 z ]};

[0123] {"face id": 22,

[0124] "facebox": [x220, y220, x221, y221],

[0125] "face keypoint": [(x221, y221) (x222, y222), ..., (x22 106 ,y22 106 )],

[0126] "face angle": [α22 x , α22 y , α22 z ]};

[0127] }

[0128] Assuming i=3, if the third frame of video image includes a human face, and the human face and the face with face identifier 11 in the second frame of video image belong to the same actor, then the information obtained about the third frame of video image is as follows:

[0129] ″frame id″: 3,

[0130] {"face id":11,

[0131] ″facebox″: [upper left vertex coordinates, lower left vertex coordinates, upper right vertex coordinates, lower right vertex coordinates],

[0132] "face keypoint": [(x311, y311) (x312, y312), ..., (x31 106 ,y31 106 )],

[0133] "face angle": [α31 x , α31 y , α31z ]};

[0134] }

[0135] The video information of the video includes: information of the first frame of video image, information of the second frame of video image and information of the third frame of video image.

[0136] The following is an explanation of why the embodiment of the present application uses a video clip to obtain the target makeup color number of the facial object.

[0137] It is understandable that in the process of shooting a video, there may be differences in light intensity, or there may be differences in the plot (for example, an actor is splashed with water, or the actor enters a dimly lit room), which may result in differences in the color values ​​of the facial objects of the same face in multiple consecutive frames of video images. If the target makeup color number of the facial object is determined by the color value of the facial object obtained from one frame of video image, it may not be accurate. Based on this, in an embodiment of the present application, the color value of the facial object is determined by a video clip to eliminate the influence of light intensity, so that the target makeup color number of the facial object obtained is more accurate.

[0138] It is understandable that during the shooting process, the shooting time may be discontinuous. Although an episode of a TV series is presented to the user, this episode of the TV series may be shot over 2 days, 3 days or 1 week. Therefore, there is a possibility that the makeup color number of the same actor's facial object may change. For example, when shooting on the first day, the actor's lipstick color number is color 1, and when shooting on the second day, the actor's lipstick color number is color 2. However, within a small time period, such as 1 hour or 1 day, the probability of the makeup color number of the actor's facial object changing is low, and the color value of the actor's facial object should be the same.

[0139] Based on this, it can be considered that if multiple consecutive video frames all include actor A, then there is a high probability that the multiple consecutive video frames are continuously shot by actor A within a small time period, that is, the color value of actor A's facial object should be the same in these multiple consecutive video frames. For example, if the 1st to 10th video frames in the video include actor A's face, and the 18th to 25th video frames include actor A's face, there is a high probability that actor A's makeup may not change from the 1st to 10th video frames, and actor A's makeup may be different from that of the 18th to 25th video frames. Therefore, the video clip corresponding to the target face has the following requirements: the video clip corresponding to the target face contains multiple frames of continuous video images, and each frame of video images includes the target face.

[0140] Since the target color value of the facial object of actor A is obtained based on multiple frames of continuous video images, the influence of light or plot during the shooting process is reduced, making the target makeup color number of the facial object of actor A obtained based on the target color value of the facial object of actor A based on multiple frames of video images more accurate.

[0141] Step S24: for each video image in each frame of the video image, a facial object image composed of local parts of the facial object included in the target face is obtained from the video image to obtain facial object images corresponding to each frame of the video image.

[0142] In an optional implementation, since the key points of the facial object are known, the key points of the facial object can be connected to obtain the contour of the facial object. The position area where the facial object image is located in the video image is the area where the contour of the facial object is located. Figure 3 As shown, by connecting the key points of the eyebrows, you can get the outline of the eyebrows, and by connecting the key points of the lips, you can get the outline of the lips.

[0143] Step S25: extracting the target color value of the facial object image corresponding to each frame of video image.

[0144] Exemplarily, an average value of colors in the facial object image is determined as the target color value.

[0145] Exemplarily, for each facial object image corresponding to a video image, the following steps B11 to B12 may be performed.

[0146] Step B11: Convert the facial object image from the RGB color space to the CIELab color space.

[0147] Unlike the RGB color space, the CIELab color space is committed to perceptual uniformity. The L component closely matches human brightness perception, and the input color scale of the a and b components is used to make precise color balance. This is closer to human vision and the target color value is more accurate.

[0148] The facial object image can be converted from the RGB color space to the CIELab color space (also known as L*a*b*) using standard conversion parameters given by the International Commission on Illumination. The CIELab color space represents color using three values: "L*" represents perceived brightness, "a*" and "b*" represent the four unique colors of human vision, such as red, green, blue and yellow.

[0149] Step B12: Extracting target color values ​​from the face object image belonging to the CIELab color space.

[0150] Assume that the video clip corresponding to the target face includes 4 frames of video images, and the target color values ​​of the facial object images corresponding to the 4 frames of video images are: [188, 83, 77], [168, 79, 70], [169, 80, 70], and [188, 79, 70] respectively.

[0151] Step S26: determining a target makeup color number corresponding to the facial object in the target face based on the target color values ​​corresponding to the facial object images corresponding to the respective frames of video images.

[0152] There are many ways to implement step S26, and the embodiments of the present application provide but are not limited to the following two.

[0153] The first implementation of step S26 includes the following steps C11 to C12.

[0154] Step C11: Calculate the average value of the target color values ​​corresponding to the plurality of facial object images.

[0155] Step C12: searching for the target makeup color number corresponding to the average value from the preset correspondence between the color value and the makeup color number.

[0156] The following is an example of a face object being lips. In this case, the makeup color number is a lipstick color number. Assume that the correspondence between the preset color value and the makeup color number is as shown in Table 1.

[0157] Table 1

[0158] Lipstick color Color Value 109 [166，77，68] 347 [166，77，71] 918 [166，77，75] ... ...

[0159] If Table 1 contains lipstick colors corresponding to the average values, assuming that the average value calculated in step C11 is [166, 77, 68], the lipstick color 109 is used as the target makeup color of the facial object.

[0160] If Table 1 does not include the lipstick color number corresponding to the average value, then for each lipstick color number in Table 1, the color difference between the color value corresponding to the lipstick color number and the average value can be calculated using the CIE2000 color difference formula. Assuming the average value is [188, 83, 77], the color difference between the color value of lipstick color number 109 and the average value is 23 / 150, the color difference between the color value of lipstick color number 347 and the average value is 22 / 150, and the color difference between the color value of lipstick color number 918 and the average value is 12 / 150. The lipstick color number with the smallest color difference can be determined as the target makeup color number of the facial object.

[0161] The second implementation method of step S26 includes the following steps C21 to C25.

[0162] Step C21: for each facial object image, calculating the color difference between each color value and a target color value corresponding to the facial object image.

[0163] Exemplarily, the color difference is calculated using the CIE2000 color difference formula.

[0164] Step C22: for each facial object image, based on the preset correspondence between color values ​​and makeup color numbers, obtaining candidate makeup color numbers corresponding to the first preset number of color values ​​with smaller color differences, so as to obtain candidate makeup color numbers corresponding to the plurality of facial object images respectively.

[0165] Exemplarily, the preset number can be determined based on actual conditions and is not limited here.

[0166] Step C23: grouping the same candidate makeup color numbers among the candidate makeup color numbers corresponding to the plurality of facial object images into the same color number set.

[0167] Step C24: Determine a target makeup color number set containing the largest number of candidate makeup color numbers from the plurality of color number sets.

[0168] Step C25: Determine that the candidate makeup color number included in the target makeup color number set is the target makeup color number.

[0169] That is, vote on the candidate makeup color numbers, and the color number with the highest number of votes is the target makeup color number.

[0170] In the makeup color number recognition method for facial objects provided by the present application, when a video is played on a video playback interface, if an instruction to recognize the makeup color number of a facial object is detected, it means that there is a face decorated with the makeup color number that the user is interested in in the target video image currently displayed on the video playback interface; the target face to which the facial object with the makeup color number to be recognized belongs is determined from the faces contained in the target video image; and a video clip corresponding to the target face is obtained. Since the video clip corresponding to the target face includes multiple consecutive video frames, and the multiple consecutive video frames all include the target face, the video clip is very likely to be continuously shot by the actor to whom the target face belongs without changing the makeup color number of the facial object. Therefore, the target color value of the facial object of the target face in each frame of the video image contained in the video clip is the same. Based on the target attribute value of the facial object in the target face in each frame of the video image in the video clip, the target makeup color number of the facial object determined is more accurate. There is no need for users to search in the browser themselves. They can input the instructions of the makeup color number of the facial object while watching the video, and then execute the makeup color number recognition method of the facial object provided by the present application to provide the user with the target makeup color number, making the process of obtaining the makeup color number of the actor's facial object simple to operate, and the target makeup color number can be obtained while watching the video, ensuring real-time performance.

[0171] It is understandable that the target video image may include one or more faces. For example, all faces included in the target video image may be determined as target faces. For example, faces corresponding to video clips that meet set conditions may be determined as target faces. Assuming that the target video image includes multiple faces, the process of determining the target face is described below, which includes the following steps D11 to D12.

[0172] Step D11: Obtain video clips corresponding to the multiple faces respectively.

[0173] The video clip corresponding to the face is composed of a plurality of continuous frames of video images in the video that contain the face, and the video clip contains the target video image.

[0174] It is understandable that, since the user is interested in the makeup of the face included in the target video image, all video clips should include the target video image. Exemplarily, the video clip includes the target video image, that is, one video image in each frame of video image is the target video image, and the makeup of the facial object of the same target face in each frame of video image included in the video clip is the same.

[0175] It can be understood that, since the video information of the video has been obtained, the video clips corresponding to the above-mentioned multiple faces can be obtained from the video information. The method includes the following steps E11 to E13.

[0176] Step E11: Determine the position number of the target video image in the video.

[0177] Step E12: Obtaining face identifiers corresponding to the plurality of faces contained in the target video image having the position number from the video information corresponding to the video.

[0178] Step E13: For each of the face identifiers, obtain the target position number of the video image having the face corresponding to the face identifier included in the video information; obtain a video segment corresponding to the face having the face identifier, which is composed of the video image having the target position number and the target video image.

[0179] Among them, the face identifiers of faces belonging to the same actor in two frames of video images with adjacent position numbers in the video information are the same, the face identifiers of faces belonging to the same actor in two frames of video images with non-adjacent position numbers are different, and the face identifiers of faces belonging to different actors are different.

[0180] Since the face identifiers of faces belonging to the same actor in two frames of video images with adjacent position numbers in the video information are the same, the face identifiers of faces belonging to the same actor in two frames of video images with non-adjacent position numbers are different, and the face identifiers of faces belonging to different actors are different, the target position number of the video image with the face corresponding to the face identifier obtained in step E13 is a continuous position number, that is, the obtained video image with the target position number and the target video image are continuous multi-frame video images.

[0181] For the description of the video information, please refer to the description of step A21 to step A25, which will not be repeated here.

[0182] It is understandable that video clips corresponding to each face in the video can be obtained based on the video information.

[0183] like Figure 4 , which is a schematic diagram of video clips corresponding to each face provided in an embodiment of the present application.

[0184] Assume that a video contains Q video images, and the Q video images include a total of M different faces (the faces belonging to the same actor in two adjacent video images belong to the same face and have the same face identifier, the faces belonging to the same actor in two non-adjacent video images belong to different faces and have different face identifiers, and the faces belonging to different actors have different face identifiers), and assume that the face identifiers of the M faces are face identifier 1, face identifier 2,..., face identifier M.

[0185] Figure 4 Each rectangle in corresponds to a frame of video image, a video image filled with white does not contain the face corresponding to the corresponding face identifier, and a video image filled with gray contains the face corresponding to the corresponding face identifier. For example, in the image sequence corresponding to face identifier 1, the first to sixth frames of video image contain the face corresponding to face identifier 1, and the seventh to the Qth frames of video image do not include the face corresponding to face identifier 1.

[0186] Step D12: for each face, if the number of frames of video images that meet a preset condition in each frame of video images contained in the video clip corresponding to the face is greater than or equal to a set threshold, the face is determined to be the target face.

[0187] The video image that meets the set condition means that the accuracy of the target color value extracted from the facial object contained in the face in the video image is greater than or equal to the preset probability.

[0188] Exemplarily, the threshold value may be set based on actual conditions; Exemplarily, the threshold value=the number of video images included in the video clip*the set ratio.

[0189] It is understandable that if the number of frames of video images satisfying the preset condition in each frame of video images included in the video clip is less than a set threshold, the target makeup color number of the facial object obtained based on the video clip is inaccurate.

[0190] The following is a detailed description of the process of detecting whether a video image meets the set conditions. The process involves the following two situations.

[0191] The first case: if the ratio of the area of ​​the face located in the face region of the video image to the area of ​​the video image is greater than a first threshold, it is determined that the video image meets the set condition.

[0192] It can be understood that if the ratio of the area of ​​the face region to the area of ​​the video image is less than the first threshold, it means that the face region in the video image is very small. If the face region is very small, the area where the face object is located in the face region is even smaller, and the process of extracting the color value of the facial object from the video image may be inaccurate.

[0193] Assuming that each frame of video image is a rectangular image, assuming that the length of the video image is w and the width is h, assuming that the face area is a rectangular area, and the coordinates of the four vertices of the rectangular area are [upper left vertex (x1, y1), lower left vertex (x1, y2), upper right vertex (x2, y1), lower right vertex (x2, y2)], then the area ratio of the face area and the video image = (x2-x1)×(y2-y1) / w*h.

[0194] The second case: if the facial angle of the face in the video image is less than or equal to a preset angle, it is determined that the video image meets the condition, and the facial angle refers to the rotation angle of the face contained in the video image relative to the front face.

[0195] It is understandable that, since the facial object is located on the human face, if the facial angle of the human face in the video image is larger, for example, higher than the preset angle, the video image may only include a part of the facial object, such as the edge of the facial object, or even exclude the facial object. It is understandable that the color value of the local part of the facial object cannot represent the color value of the facial object. For example, the lipstick color of the lips is greatly affected by the lip lines because the lips have lip lines. That is, the color of the lip edge is a combination of the lipstick color and the lip line color, so the color value obtained based on the lip edge is inaccurate. Therefore, it is considered that if the facial angle of the human face in the video image is greater than the preset angle, it is determined that the video image does not meet the set condition.

[0196] If the video image corresponding to the face contains a large number of video images that do not meet the set conditions, it is considered that the target makeup color number of the facial object corresponding to the face obtained through the video image is inaccurate. Therefore, if the number of frames of video images that meet the preset conditions in each frame of video images contained in the video clip corresponding to the face is greater than or equal to a set threshold, the face is determined to be the target face.

[0197] It is understandable that the number of target faces included in the target video image may be one or more. Although it is believed that the color values ​​of the facial objects in the target faces in each frame of video images contained in the video clip corresponding to the target face should be the same, if the color value of the facial object in the target face changes rapidly due to drastic changes in shooting light during the shooting process, the target makeup color number of the facial object in the target face obtained based on the video clip is inaccurate. Based on this, an embodiment of the present application provides the following method: if the variance of the target color value of the facial object image corresponding to each frame of video image is less than or equal to a second threshold, a step of determining the target makeup color number corresponding to the facial object in the target face is performed based on the target color value corresponding to the facial object image corresponding to each frame of video image.

[0198] The variance of the color of the facial object image corresponding to each frame of the video image in the video clip corresponding to the target face is calculated. If the variance is greater than the second threshold, it means that the video clip has a rapid change in the color of the facial object due to drastic changes in shooting light, etc., and is not suitable for makeup color number recommendation.

[0199] It is understandable that the color values ​​of different positions of the facial object may be different. Exemplarily, the average of the color values ​​of different regions of the facial object is used as the target color value corresponding to the facial object.

[0200] It is understandable that the closer to the edge of the facial object or the narrower the area at both ends, the more likely it is that inaccurate color values ​​will be extracted; the more easily the edge of the facial object is affected by the original color of the facial object, the more likely it is that inaccurate color values ​​will be extracted. For example, if the facial object is lips, since there are many lip lines at the corners of the mouth, the color of the corners of the mouth is a mixture of the lipstick color and the lip line color.

[0201] In order to make the makeup color number of the face object recommended for the user visually closer to the real color number of the actor's face object, the area where the true color of the face object is least obvious or the true color of the face object is most uniform is extracted. The specific method includes the following steps F11 to F14.

[0202] Step F11: Acquire key points of the target face in the video image.

[0203] Step F12: searching for target key points corresponding to the local part of the facial object of the target face from the key points of the target face, based on the correspondence between the preset local part of the facial object and the key points forming the area where the local part is located.

[0204] If the facial object type is lips, the target key points include: the upper key point of the middle of the lower lip and the lower key point of the middle of the lower lip.

[0205] It is understandable that the principle of drawing eyebrows is that the eyebrows are light at the beginning and dark at the end, so the middle area of ​​the eyebrows can be used as the area where the facial object image is located. If the facial object is an eyebrow, the target key point includes the eyebrow peak key point.

[0206] It is understandable that the cosmetic contact lens is located on the iris of the eye, so the area where the iris is located can be the area where the facial object image is located. If the facial object is an eye, the target key points include key points for defining the iris.

[0207] Step F13: constructing a target area where the local part of the facial object of the target face is located through the target key points.

[0208] If the facial object type is lips, step F13 is specifically as follows: taking the distance between the upper key point in the middle of the lower lip and the lower key point in the middle of the lower lip as the length of the side of the target area, and taking the upper key point in the middle of the lower lip and the lower key point in the middle of the lower lip as the midpoints of the sides of the target area respectively to construct the target area.

[0209] The methods of constructing the target area corresponding to different facial objects may be different.

[0210] like Figure 5 , which is a schematic diagram of the target area provided in an embodiment of the present application when the facial object is lips as an example.

[0211] like Figure 5 As shown, the upper key point 51 of the middle of the lower lip and the lower key point 52 of the middle of the lower lip, the target area is as follows Figure 5 The square area shown.

[0212] like Figure 5 As shown, the side length of the target area is the distance between the upper key point 51 in the middle of the lower lip and the lower key point 52 in the middle of the lower lip, and the upper key point 51 in the middle of the lower lip and the lower key point 52 in the middle of the lower lip are respectively the midpoints of the side length of the target area.

[0213] Step F14: Determine that the image of the target area in the video image is the facial object image.

[0214] The method is described in detail in the embodiments disclosed in the above-mentioned application. The method of the application can be implemented by various forms of devices. Therefore, the application also discloses a device, and a specific embodiment is given below for detailed description.

[0215] like Figure 6 As shown, it is a structural diagram of a makeup color number recognition device for a facial object provided by an embodiment of the present application, the device includes: a first determination module 61, a second determination module 62, a first acquisition module 63, a second acquisition module 64, an extraction module 65 and a third determination module 66, wherein:

[0216] A first determination module 61 is used to determine the target video image currently displayed on the video playback interface when a video is played on the video playback interface, if an instruction for identifying the makeup color number of a facial object is detected, wherein the facial object is a part of a human face decorated with makeup;

[0217] A second determination module 62 is used to determine a target face to which the facial object of the makeup color number to be identified belongs from the faces included in the target video image;

[0218] A first acquisition module 63 is used to acquire a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image;

[0219] A second acquisition module 64 is used to acquire, for each video image in each frame of the video image, a facial object image composed of a local part of the facial object included in the target face, so as to obtain facial object images corresponding to each frame of the video image;

[0220] An extraction module 65, used for extracting target color values ​​of the facial object images corresponding to the respective frames of video images;

[0221] The third determination module 66 is used to determine the target makeup color number corresponding to the facial object in the target face based on the target color value corresponding to the facial object image corresponding to each frame of the video image.

[0222] In an optional implementation, the second acquisition module includes:

[0223] A first acquisition unit, used to acquire key points of the target face in the video image;

[0224] A first searching unit is used to search for a target key point corresponding to the local part of the facial object of the target face from the key points of the target face, based on the correspondence between the local part of the facial object and the key points forming the area where the local part is located.

[0225] A construction unit, used for constructing a target area where a local part of a facial object of the target face is located by using the target key points;

[0226] The first determining unit is used to determine that the image of the target area in the video image is the facial object image.

[0227] In an optional implementation, the facial object is lips, and the target key points include: an upper key point in the middle of the lower lip and a lower key point in the middle of the lower lip; the first search unit is specifically used to: use the distance between the upper key point in the middle of the lower lip and the lower key point in the middle of the lower lip as the length of the side of the target area, and use the upper key point in the middle of the lower lip and the lower key point in the middle of the lower lip as the midpoints of the sides of the target area, respectively.

[0228] In an optional implementation, the target video image includes multiple faces, and the second determination module includes:

[0229] A second acquisition unit is used to acquire video clips corresponding to the multiple faces respectively;

[0230] The second determination unit is used to determine, for each human face, that the face is the target face if the number of video images that meet the preset conditions in the video clips corresponding to the face is greater than or equal to a set threshold; the video image that meets the set conditions means that the accuracy of the target color value extracted from the facial object contained in the face in the video image is greater than or equal to a preset probability.

[0231] In an optional implementation, the method further includes:

[0232] a fourth determination module, configured to determine that the video image satisfies the set condition if the ratio of the area of ​​the face located in the face region of the video image to the area of ​​the video image is greater than a first threshold; and / or,

[0233] The fifth determination module is used to determine that the video image meets the set condition if the facial angle of the face in the video image is less than or equal to a preset angle, and the facial angle refers to the rotation angle of the face contained in the video image relative to the front face.

[0234] In an optional implementation, the second obtaining unit includes:

[0235] A third determining unit, used to determine a position number of the target video image in the video;

[0236] A third acquisition unit is used to acquire face identifiers corresponding to multiple faces contained in the target video image with the position number from the video information corresponding to the video;

[0237] A fourth acquisition unit is configured to acquire, for each face identifier, a target position number of a video image having a face corresponding to the face identifier included in the video information; and acquire a video segment corresponding to the face having the face identifier, which is composed of a video image having the target position number and the target video image;

[0238] Among them, the face identifiers of faces belonging to the same actor in two frames of video images with adjacent position numbers in the video information are the same, the face identifiers of faces belonging to the same actor in two frames of video images with non-adjacent position numbers are different, and the face identifiers of faces belonging to different actors are different.

[0239] In an optional implementation, the method further includes:

[0240] The triggering module is configured to trigger the third determining module if the variance of the target color values ​​of the facial object images respectively corresponding to the frames of video images is less than or equal to a second threshold.

[0241] In an optional implementation, the third determining module includes:

[0242] a calculation unit, for calculating, for each facial object image, a color difference between each color value and a target color value corresponding to the facial object image;

[0243] a fifth acquisition unit, configured to acquire, for each facial object image, candidate makeup color numbers corresponding to a first preset number of color values ​​with smaller color differences based on a preset correspondence relationship between color values ​​and makeup color numbers, so as to obtain candidate makeup color numbers corresponding to the plurality of facial object images respectively;

[0244] A division unit, configured to divide the same candidate makeup color numbers among the candidate makeup color numbers respectively corresponding to the plurality of facial object images into the same color number set;

[0245] a fourth determining unit, configured to determine, from the plurality of color number sets, a target makeup color number set containing a maximum number of candidate makeup color numbers;

[0246] The fifth determining unit is used to determine that the candidate makeup color number included in the target makeup color number set is the target makeup color number.

[0247] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0248] Figure 7 The invention is a block diagram showing a device for a server according to an exemplary embodiment.

[0249] The server includes, but is not limited to, a processor 71 , a memory 72 , a network interface 73 , an I / O controller 74 , and a communication bus 75 .

[0250] It should be noted that those skilled in the art can understand that Figure 7 The server structure shown in the figure does not constitute a limitation on the server, and the server may include Figure 7 More or fewer components may be shown, or certain components may be combined, or the components may be arranged differently.

[0251] Combine the following Figure 7 A detailed introduction to the various components of the server:

[0252] The processor 71 is the control center of the server. It uses various interfaces and lines to connect various parts of the entire server. It executes various functions of the server and processes data by running or executing software programs and / or modules stored in the memory 72, and calling data stored in the memory 72, so as to monitor the server as a whole. The processor 71 may include one or more processing units; illustratively, the processor 71 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 71.

[0253] The processor 71 may be a central processing unit (CPU), or an application specific integrated circuit ASIC (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention;

[0254] The memory 72 may include a memory, such as a high-speed random access memory (RAM) 721 and a read-only memory (ROM) 722, and may also include a large-capacity storage device 723, such as at least one disk storage, etc. Of course, the server may also include hardware required for other services.

[0255] The memory 72 is used to store instructions executable by the processor 71. The processor 71 has the following functions:

[0256] When a video is played on a video playback interface, if an instruction to identify a makeup color number of a facial object is detected, a target video image currently displayed on the video playback interface is determined, and the facial object is a part of a human face decorated with makeup;

[0257] Determine a target human face to which the facial object of the makeup color number to be identified belongs from the human faces included in the target video image;

[0258] Acquire a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image;

[0259] For each video image in each frame of the video image, a facial object image composed of a local part of a facial object included in the target face is obtained from the video image to obtain facial object images corresponding to each frame of the video image;

[0260] Extracting target color values ​​of the facial object images corresponding to the respective frames of video images;

[0261] Based on the target color values ​​corresponding to the facial object images corresponding to the respective frames of video images, a target makeup color number corresponding to the facial object in the target face is determined.

[0262] A wired or wireless network interface 73 is configured to connect the server to a network.

[0263] The processor 71, the memory 72, the network interface 73 and the I / O controller 74 may be interconnected via a communication bus 75, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0264] In an exemplary embodiment, the server may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the above-mentioned method for identifying the makeup color number of a facial object.

[0265] In an exemplary embodiment, the present disclosure provides a storage medium including instructions, such as a memory 72 including instructions, and the instructions can be executed by a processor 71 of a server to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0266] In an exemplary embodiment, a computer-readable storage medium is also provided, which can be directly loaded into the internal memory of a computer, such as the above-mentioned memory 72, and contains software code. After the computer program is loaded and executed, it can implement the above-mentioned method for identifying the makeup color number of a facial object.

[0267] In an exemplary embodiment, a computer program product is also provided, which can be directly loaded into the internal memory of a computer, such as the memory included in the server, and contains software code. After being loaded and executed by the computer, the computer program can implement the above-mentioned method for identifying the makeup color number of a facial object.

[0268] It should be noted that the features described in the various embodiments in this specification can be replaced or combined with each other. For the device or system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0269] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of further restrictions, the elements limited by the statement "comprise a..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0270] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0271] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying makeup color number of a facial object, characterized in that: include: When a video is played on a video playback interface, if an instruction to identify a makeup color number of a facial object is detected, a target video image currently displayed on the video playback interface is determined, and the facial object is a part of a human face decorated with makeup; Determine a target human face to which the facial object of the makeup color number to be identified belongs from the human faces included in the target video image; Acquire a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image; For each video image in each frame of the video image, a facial object image composed of a local part of a facial object included in the target face is obtained from the video image to obtain facial object images corresponding to each frame of the video image; Extracting target color values ​​of the facial object images corresponding to the respective frames of video images; Based on the target color values ​​corresponding to the facial object images corresponding to the respective frames of video images, a target makeup color number corresponding to the facial object in the target face is determined.

2. The method for identifying makeup color number of a facial object according to claim 1, characterized in that: Acquiring a facial object image composed of local parts of the facial object contained in the target face from the video image, comprising: Obtaining key points of the target face in the video image; From the correspondence between the preset local parts of the facial object and the key points forming the area where the local parts are located, searching for the target key points corresponding to the local parts of the facial object of the target face from the key points of the target face; Constructing a target area where a local part of the facial object of the target face is located by using the target key points; An image of the target area in the video image is determined to be the facial object image.

3. The method for identifying makeup color number of a facial object according to claim 2, characterized in that: The facial object is lips, and the target key points include: an upper key point in the middle of the lower lip and a lower key point in the middle of the lower lip; Constructing a target area where a local part of the facial object of the target face is located by using the target key points includes: The target area is constructed by taking the distance between the upper key point of the middle of the lower lip and the lower key point of the middle of the lower lip as the length of the side of the target area, and taking the upper key point of the middle of the lower lip and the lower key point of the middle of the lower lip as the midpoints of the sides of the target area respectively.

4. The method for identifying makeup color number of a facial object according to any one of claims 1 to 3, characterized in that: The target video image includes a plurality of human faces, and determining the target human face to which the facial object of the makeup color number to be identified belongs from the human faces included in the target video image includes: Obtaining video clips corresponding to the multiple faces respectively; For each human face, if the number of video images that meet preset conditions in the video clip corresponding to the face is greater than or equal to a set threshold, the face is determined to be the target face; the video image that meets the set condition means that the accuracy of the target color value extracted from the facial object contained in the face in the video image is greater than or equal to a preset probability.

5. The method for identifying makeup color number of a facial object according to claim 4, characterized in that: Determining the video image that meets the set condition includes: If the ratio of the area of ​​the face located in the face region of the video image to the area of ​​the video image is greater than a first threshold, it is determined that the video image meets the set condition; and / or, If the facial angle of the face in the video image is less than or equal to a preset angle, it is determined that the video image meets the set condition, and the facial angle refers to the rotation angle of the face contained in the video image relative to the front face.

6. The method for identifying makeup color number of a facial object according to claim 4 or 5, characterized in that: Obtaining video clips corresponding to the multiple faces, respectively, including: Determine the position number of the target video image in the video; Obtaining face identifiers corresponding to a plurality of faces contained in the target video image having the position number from the video information corresponding to the video; For each of the face identifiers, obtaining a target position number of a video image having a face corresponding to the face identifier included in the video information; obtaining a video segment corresponding to the face having the face identifier, which is composed of a video image having the target position number and the target video image; Among them, the face identifiers of faces belonging to the same actor in two frames of video images with adjacent position numbers in the video information are the same, the face identifiers of faces belonging to the same actor in two frames of video images with non-adjacent position numbers are different, and the face identifiers of faces belonging to different actors are different.

7. The method for identifying makeup color number of a facial object according to any one of claims 1 to 3, characterized in that: Also includes: If the variance of the target color values ​​of the facial object images corresponding to each frame of video image is less than or equal to a second threshold, a step of determining a target makeup color number corresponding to the facial object in the target face based on the target color values ​​corresponding to the facial object images corresponding to each frame of video image is performed.

8. The method for identifying makeup color number of a facial object according to any one of claims 1 to 3, characterized in that: Determining a target makeup color number corresponding to the facial object in the target face based on the target color values ​​corresponding to the facial object images corresponding to the frames of video images, including: For each facial object image, calculating the color difference between each color value and a target color value corresponding to the facial object image; For each facial object image, based on a preset correspondence between color values ​​and makeup color numbers, obtaining candidate makeup color numbers corresponding to a first preset number of color values ​​with smaller color differences, so as to obtain candidate makeup color numbers corresponding to the plurality of facial object images respectively; Classifying the same candidate makeup color numbers among the candidate makeup color numbers corresponding to the multiple facial object images respectively into the same color number set; Determining a target makeup color number set containing a maximum number of candidate makeup color numbers from the plurality of color number sets; Determine that the candidate makeup color number included in the target makeup color number set is the target makeup color number.

9. A makeup color recognition device for a facial object, characterized in that: include: A first determination module is used to determine the target video image currently displayed on the video playback interface when a video is played on the video playback interface, if an instruction to identify the makeup color number of a facial object is detected, and the facial object is a part of the human face decorated with makeup; A second determination module is used to determine a target face to which the facial object of the makeup color number to be identified belongs from the faces included in the target video image; A first acquisition module is used to acquire a video clip corresponding to the target face, wherein each frame of video image contained in the video clip is a plurality of continuous frames of video image in the video, and each frame of video image contains the target video image; A second acquisition module is used to acquire, for each video image in each frame of the video image, a facial object image composed of a local part of a facial object included in the target face, so as to obtain facial object images corresponding to each frame of the video image; An extraction module, used for extracting target color values ​​of the facial object images corresponding to the respective frames of video images; The third determination module is used to determine the target makeup color number corresponding to the facial object in the target face based on the target color value corresponding to the facial object image corresponding to each frame of the video image.

10. A server, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the makeup color number recognition method for a facial object as described in any one of claims 1 to 8.

Images