A method and device for generating a character portrait based on portrait analysis

Abstract

The present invention discloses a method and device for generating a character personality portrait based on portrait analysis. The method comprises: extracting character image feature information of image data of known persons by using a pre-trained classification model, generating a character feature data pool according to the character image feature information, wherein the character image feature information comprises: hairstyle, hair color, skin color, eyes, and expression; calibrating the character of each character image feature information in the character feature data pool according to the character features of the known persons; extracting the character image feature information of each user image to be identified in the image data pool to be analyzed; matching the character image feature information of each user image to be identified with the character image feature information in the portrait feature data pool, and determining the character category of the character image feature information of each user image to be identified according to the matching result; generating a character personality portrait of each user image in the image data pool to be analyzed according to the character category of each user image.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a method and device for generating a character profile based on portrait analysis. Background Art

[0002] Nowadays, customer centers contain a large amount of customer information and customer data. How to accurately find the corresponding customers from massive data has become a thorny problem. Therefore, massive user information can be processed to generate vivid user portraits. However, how to transform a rigid customer data into a more vivid customer character portrait so that the corresponding business personnel can easily grasp the customer's personality and preferences when communicating with the customer has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides a method and device for generating a character profile based on portrait analysis.

[0004] According to one aspect of the present invention, a method for generating a character profile based on portrait analysis is provided, comprising:

[0005] Use the pre-trained classification model to extract the character image feature information of the image data of known persons, and generate a portrait feature data pool based on the character image feature information. The character image feature information includes: hairstyle, hair color, skin color, eyes, and expression;

[0006] Character calibration is performed on each character image feature information in the portrait feature data pool according to the known character features of the person;

[0007] Extracting the image feature information of the person to be identified of each user image in the image data pool to be analyzed;

[0008] Matching the image feature information of the person to be identified of each user image with the image feature information of the person in the portrait feature data pool, and determining the personality category of the image feature information of the person to be identified of each user image according to the matching result;

[0009] According to the personality category of each user image, a personality portrait of each user image in the image data pool to be analyzed is generated; wherein,

[0010] The loss functions of the classification model include: cross entropy loss function and binary cross entropy loss function;

[0011] The cross entropy loss function is:

[0012] ;

[0013] In the formula, Yi and Pi represent the true label and predicted value of the i-th image data respectively;

[0014] The binary cross entropy loss function is:

[0015] ;

[0016] Where N is the total number of pixels, y is the binary true label of each pixel, and p is the predicted value of each pixel.

[0017] Optionally, extracting person image feature information of image data of known persons using a pre-trained classification model includes:

[0018] Convert image data into a grayscale image;

[0019] Load the pre-trained classification model to classify the grayscale image and extract the character image feature information, where the classification model includes: face detector and face key point detector.

[0020] Optionally, load a pre-trained classification model to classify the grayscale image and extract the character image feature information, including:

[0021] Performing face detection on the grayscale image by using a face detector to determine a face grayscale image;

[0022] Perform feature extraction on the grayscale face image to determine the character image feature information.

[0023] Optionally, feature extraction is performed on the grayscale image of the face to determine feature information of the person image, including:

[0024] Extract hairstyle from grayscale face image to determine hairstyle feature information;

[0025] Extract hair color from the grayscale image of the face to determine the hair color feature information;

[0026] Extract skin color from grayscale face images to determine skin color feature information;

[0027] Extract eyes from the grayscale image of the face and determine the eye feature information;

[0028] Perform expression extraction on the grayscale face image to determine the expression feature information.

[0029] Optionally, performing hairstyle extraction on the grayscale face image to determine hairstyle feature information includes:

[0030] Detect the key points of the face using a face key point detector;

[0031] Determine the initial hair position of the hair area according to the face shape and position of the face grayscale image;

[0032] The initial hair position is adjusted according to the distance and angle relationship between key points to determine the hair area;

[0033] The threshold segmentation method is used to segment the hair area and obtain the hair binary mask;

[0034] The texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information.

[0035] Optionally, the initial hair position is adjusted according to the distance and angle relationship between the key points to determine the hair area, including:

[0036] Select predetermined key points as references to determine the head shape, wherein the predetermined key points include eyebrows and cheekbones;

[0037] According to the head shape, the hair boundary and the head skin area are divided by a preset distance threshold and a preset angle threshold to determine the hair boundary, wherein the preset distance threshold and the preset angle threshold are set according to average distance and standard deviation statistical information;

[0038] Based on the hair boundaries, determine the hair area.

[0039] Optionally, a threshold segmentation method is used to segment the hair region to obtain a hair binary mask, including:

[0040] grayscale or color space conversion is performed on the hair region to determine a converted image of the hair region;

[0041] Segmenting the hair and background in the hair region conversion image according to a preset threshold value to determine the hair foreground and background regions, wherein the preset threshold value is set according to an adaptive threshold selection method;

[0042] Morphological operations are used to post-process the hair foreground and background regions to obtain the hair binary mask.

[0043] Optionally, a texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information, including:

[0044] extracting an image patch of a hair region from the image data according to a hair binary mask;

[0045] grayscale or color space conversion is performed on the image block to determine the converted image block;

[0046] The local binary pattern algorithm is used to extract the texture features of the hair area from the converted image blocks;

[0047] Encode the texture features and determine the encoding vector;

[0048] Repeatedly extracting and encoding the texture according to the hair binary mask to obtain multiple encoding vectors;

[0049] Multiple encoding vectors are merged to obtain hairstyle feature information.

[0050] According to another aspect of the present invention, there is provided a device for generating a character profile based on portrait analysis, comprising:

[0051] The first extraction module is used to extract character image feature information of image data of known persons using a pre-trained classification model, and generate a portrait feature data pool according to the character image feature information, wherein the character image feature information includes: hairstyle, hair color, skin color, eyes, and expression;

[0052] The calibration module is used to calibrate the personality of each person's image feature information in the portrait feature data pool according to the known personality characteristics of the person;

[0053] The second extraction module is used to extract the image feature information of the person to be identified of each user image in the image data pool to be analyzed;

[0054] A matching module, used to match the image feature information of the person to be identified of each user image with the image feature information of the person in the portrait feature data pool, and determine the personality category of the image feature information of the person to be identified of each user image according to the matching result;

[0055] A generation module is used to generate a character profile of each user image in the image data pool to be analyzed according to the character category of each user image; wherein,

[0056] The loss functions of the classification model include: cross entropy loss function and binary cross entropy loss function;

[0057] The cross entropy loss function is:

[0058] ;

[0059] In the formula, Yi and Pi represent the true label and predicted value of the i-th image data respectively;

[0060] The binary cross entropy loss function is:

[0061] ;

[0062] Where N is the total number of pixels, y is the binary true label of each pixel, and p is the predicted value of each pixel.

[0063] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method described in any one of the above aspects of the present invention.

[0064] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of the above aspects of the present invention.

[0065] Therefore, this application forms a portrait feature data pool through intelligent portrait analysis, and then determines the personality type of the user image to be analyzed by extracting features from the user image to be analyzed and matching the feature information in the Iceberg portrait feature data pool, thereby easily completing the character portraits of tens of thousands of customers. In conversations with customers, it is easy to grasp the other party's personality and take the initiative in the conversation. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0067] Figure 1 It is a flowchart of a method for generating a character profile based on portrait analysis provided by an exemplary embodiment of the present invention;

[0068] Figure 2 It is a schematic diagram of the structure of a device for generating a character profile based on portrait analysis provided by an exemplary embodiment of the present invention;

[0069] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0070] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0071] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0072] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present invention are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0073] It should also be understood that, in the embodiments of the present invention, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0074] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0075] In addition, the term "and / or" in the present invention is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.

[0076] It should also be understood that the description of the various embodiments of the present invention focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0077] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0078] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0079] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0080] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0081] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0082] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0083] Exemplary Methods

[0084] Figure 1 FIG. 1 is a flow chart of a method for generating a character profile based on portrait analysis provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the method 100 for generating a character portrait based on portrait analysis includes the following steps:

[0085] Step 101, using a pre-trained classification model to extract character image feature information of image data of known persons, and generating a portrait feature data pool according to the character image feature information, wherein the character image feature information includes: hairstyle, hair color, skin color, eyes, and expression;

[0086] Step 102, character calibration is performed on each person image feature information in the portrait feature data pool according to the known person's character features;

[0087] Step 103, extracting the image feature information of the person to be identified of each user image in the image data pool to be analyzed;

[0088] Step 104, matching the image feature information of the person to be identified of each user image with the image feature information of the person in the portrait feature data pool, and determining the personality category of the image feature information of the person to be identified of each user image according to the matching result;

[0089] Step 105, generating a character profile of each user image in the image data pool to be analyzed according to the character category of each user image; wherein,

[0090] The loss functions of the classification model include: cross entropy loss function and binary cross entropy loss function;

[0091] The cross entropy loss function is:

[0092] ;

[0093] In the formula, Yi and Pi represent the true label and predicted value of the i-th image data respectively;

[0094] The binary cross entropy loss function is:

[0095] ;

[0096] Where N is the total number of pixels, y is the binary true label of each pixel, and p is the predicted value of each pixel.

[0097] Specifically, the customer data of the customer center contains a lot of customer image data. By extracting customer portraits, we gradually extract customer appearance features, such as hairstyle, hair color, skin color, eye size, expression, and many other features. By manually marking the personalities of familiar customers and matching them with the portrait features, we form a big data pool and analyze that, for example, short-haired customers are mostly calm and long-haired customers are mostly exaggerated. Then, we perform big data analysis on unfamiliar customer voices to obtain unfamiliar customer personality portraits. The specific steps include:

[0098] 1. Extract the hairstyle, hair color, skin color, eye size, expression, etc. of familiar people's images through code, extract a large amount of familiar people's portrait feature information, and generate a portrait feature data pool;

[0099] 2. For those whose portrait features have been extracted, enter the corresponding personality traits based on personal impressions;

[0100] 3. Use the code to match the portrait features with the personalities entered by familiar people to generate a data pool;

[0101] 4. Extract the facial features of unfamiliar customers, match them with the facial-personality characteristics in the data pool, and generate a personality portrait of the customer.

[0102] In one embodiment, this application takes hairstyle features as an example, and the steps of extracting hairstyle features of a person image are as follows:

[0103] 1. Load the pre-trained face detector and face key point detector;

[0104] 2. Load image data;

[0105] 3. Convert image data into grayscale image;

[0106] 4. Use face detector to detect faces;

[0107] 5. Extract the hairstyle for each detected face;

[0108] (1) First, perform face key point detection. Use the face key point detection algorithm - the face key point detector in the dlib library to detect the key points of the face.

[0109] (2) Extracting the hair region: Based on key point detection, the approximate location of the hair region is determined by manual rule-based methods according to the shape and position of the face. The distance and angle relationship between key points is used to estimate the hair region.

[0110] [1] Head shape estimation: Use facial key points to estimate the head shape. Key points such as eyebrows and cheekbones can be selected as references. Based on the positions of these key points, the shape of the head can be roughly determined;

[0111] [2] Hair boundary estimation: Based on the shape of the head, use manual rules to separate the hair boundary from the skin area of ​​the head. Depending on the shape and position of the head, some thresholds can be set to filter out points outside the head;

[0112] [3] Distance threshold: Use the distance between key points to determine the boundary of the hair area. The threshold can be set based on statistical information such as the average distance and standard deviation. Usually, the distance between key points at the edge of the head is larger, so a larger distance threshold can be used;

[0113] [4] Angle threshold: Use the angle between key points to determine the boundary of the hair region. The threshold can be set based on statistical information such as the average angle and standard deviation. Usually, the angle of the hair region varies greatly, so a larger angle threshold can be used;

[0114] [5] Further refinement: Based on the distance and angle thresholds, the approximate location of the hair area is obtained. Then further image processing methods and edge detection can be used to further refine and extract the hair area.

[0115] (3) Hair segmentation: Segment the extracted hair area to obtain a binary mask of the hair. Threshold segmentation can be used:

[0116] [1] Preprocessing: Preprocess the hair region, such as grayscale or color space conversion. If the hair region is a color image, it can be converted to a grayscale image to simplify processing;

[0117] [2] Threshold selection: Select an appropriate threshold to separate the hair from the background. Some adaptive threshold selection methods can be used, such as Otsu's method, to select an appropriate threshold based on the histogram distribution characteristics of the hair and background;

[0118] [3] Segmentation: The pixels in the image are segmented according to the selected threshold to obtain a binary image, in which the hair area is marked as foreground (white) and the background area is marked as background (black);

[0119] [4] Post-processing: The segmentation results are post-processed to remove noise and fill holes. Morphological operations such as dilation and erosion can be used to remove small noise points and fill holes in the hair area.

[0120] (4) Hair style feature extraction: According to the application requirements and the definition of the hairstyle, the texture feature extraction algorithm - local binary pattern can be used:

[0121] [1] Hair region selection: Use the previously extracted hair binary mask and apply it to the original image to extract the image patch of the hair region;

[0122] [2] Image preprocessing: Preprocess the image blocks of the hair area, such as grayscale or color space conversion;

[0123] [3] Feature extraction: Use the local binary pattern algorithm to extract texture features in the image block of the hair area. The LBP algorithm describes texture features by comparing the grayscale value of each pixel with its surrounding pixels and encoding the result into a binary value;

[0124] [4] Feature encoding: Encode the extracted LBP features, for example, by using a histogram to perform statistics on them. The LBP features of each image block are represented as a feature vector;

[0125] [5] Feature fusion: Repeat the above steps on multiple hair regions in the entire image to obtain multiple LBP feature vectors. These feature vectors can be fused, for example, by concatenating them or calculating the average to obtain the final hairstyle feature representation.

[0126] 6. Draw a rectangle for the hairstyle area.

[0127] 7. Display the result image.

[0128] In the above description, the image is read and processed, and the pre-trained face detector and key point detector are loaded. Then, the detected face area is used to extract the hairstyle area through the location of the facial key points, and a rectangle of the hairstyle area is drawn on the original image to extract the hairstyle feature information.

[0129] In one embodiment, the loss function of the classification model of the present application is selected as follows:

[0130] This application regards feature extraction as a classification task, and the corresponding loss function is generally a cross entropy loss function. In addition, if the hairstyle of a person is extracted at the pixel level, a computer vision technology called "segmentation" may be used, such as U-Net, Mask R-CNN and other models. In this case, the loss function may be at the pixel level, and may be a binary cross entropy loss function (Binary Cross Entropy Loss) or Dice Loss, etc. Therefore, a loss function based on binary cross entropy between pixels is also used.

[0131] Therefore, this application forms a portrait feature data pool through intelligent portrait analysis, and then determines the personality type of the user image to be analyzed by extracting features from the user image to be analyzed and matching the feature information in the Iceberg portrait feature data pool, thereby easily completing the character portraits of tens of thousands of customers. In conversations with customers, it is easy to grasp the other party's personality and take the initiative in the conversation.

[0132] Exemplary Devices

[0133] Figure 2 FIG. 1 is a schematic diagram of a structure of a device for generating a character profile based on portrait analysis provided by an exemplary embodiment of the present invention. Figure 2 As shown, the device 200 includes:

[0134] The first extraction module 210 is used to extract character image feature information of image data of known persons using a pre-trained classification model, and generate a portrait feature data pool according to the character image feature information, wherein the character image feature information includes: hairstyle, hair color, skin color, eyes, and expression;

[0135] The calibration module 220 is used to calibrate the personality of each person image feature information in the portrait feature data pool according to the personality characteristics of the known person;

[0136] The second extraction module 230 is used to extract the image feature information of the person to be identified of each user image in the image data pool to be analyzed;

[0137] The matching module 240 is used to match the image feature information of the person to be identified of each user image with the image feature information of the person in the portrait feature data pool, and determine the personality category of the image feature information of the person to be identified of each user image according to the matching result;

[0138] The generating module 250 is used to generate a character profile of each user image in the image data pool to be analyzed according to the character category of each user image; wherein,

[0139] The loss functions of the classification model include: cross entropy loss function and binary cross entropy loss function;

[0140] The cross entropy loss function is:

[0141] ;

[0142] In the formula, Yi and Pi represent the true label and predicted value of the i-th image data respectively;

[0143] The binary cross entropy loss function is:

[0144] ;

[0145] Where N is the total number of pixels, y is the binary true label of each pixel, and p is the predicted value of each pixel.

[0146] Optionally, the first extraction module 210 uses a pre-trained classification model to extract person image feature information of image data of a known person, including:

[0147] A conversion submodule, used to convert image data into a grayscale image;

[0148] The extraction submodule is used to load a pre-trained classification model to classify the grayscale image and extract the character image feature information, wherein the classification model includes: a face detector and a face key point detector.

[0149] Optionally, extract submodules, including:

[0150] A first determining unit is used to perform face detection on the grayscale image through a face detector to determine a face grayscale image;

[0151] The second determining unit is used to extract features from the grayscale image of the face and determine the feature information of the person image.

[0152] Optionally, the second determining unit includes:

[0153] The first determination subunit is used to extract the hairstyle from the grayscale image of the face and determine the hairstyle feature information;

[0154] The second determination subunit is used to extract the hair color from the grayscale image of the face and determine the hair color feature information;

[0155] The third determination subunit is used to extract skin color from the grayscale image of the face and determine skin color feature information;

[0156] The fourth determination subunit is used to extract eyes from the grayscale image of the face and determine eye feature information;

[0157] The fifth determination subunit is used to extract expression from the grayscale image of the face and determine expression feature information.

[0158] Optionally, the first determining subunit includes:

[0159] Detect the key points of the face using a face key point detector;

[0160] Determine the initial hair position of the hair area according to the face shape and position of the face grayscale image;

[0161] The initial hair position is adjusted according to the distance and angle relationship between key points to determine the hair area;

[0162] The threshold segmentation method is used to segment the hair area and obtain the hair binary mask;

[0163] The texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information.

[0164] Optionally, the initial hair position is adjusted according to the distance and angle relationship between the key points to determine the hair area, including:

[0165] Select predetermined key points as references to determine the head shape, wherein the predetermined key points include eyebrows and cheekbones;

[0166] According to the head shape, the hair boundary and the head skin area are divided by a preset distance threshold and a preset angle threshold to determine the hair boundary, wherein the preset distance threshold and the preset angle threshold are set according to average distance and standard deviation statistical information;

[0167] Based on the hair boundaries, determine the hair area.

[0168] Optionally, a threshold segmentation method is used to segment the hair region to obtain a hair binary mask, including:

[0169] grayscale or color space conversion is performed on the hair region to determine a converted image of the hair region;

[0170] Segmenting the hair and background in the hair region conversion image according to a preset threshold value to determine the hair foreground and background regions, wherein the preset threshold value is set according to an adaptive threshold selection method;

[0171] Morphological operations are used to post-process the hair foreground and background regions to obtain the hair binary mask.

[0172] Optionally, a texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information, including:

[0173] extracting an image patch of a hair region from the image data according to a hair binary mask;

[0174] grayscale or color space conversion is performed on the image block to determine the converted image block;

[0175] The local binary pattern algorithm is used to extract the texture features of the hair area from the converted image blocks;

[0176] Encode the texture features and determine the encoding vector;

[0177] Repeatedly extracting and encoding the texture according to the hair binary mask to obtain multiple encoding vectors;

[0178] Multiple encoding vectors are merged to obtain hairstyle feature information.

[0179] Exemplary Electronic Devices

[0180] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes one or more processors 31 and a memory 32 .

[0181] The processor 31 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0182] The memory 32 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may run the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above and / or other desired functions. In one example, the electronic device may also include: an input device 33 and an output device 34, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0183] In addition, the input device 33 may also include, for example, a keyboard, a mouse, and the like.

[0184] The output device 34 can output various information to the outside. The output device 34 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0185] Of course, to simplify, Figure 3Only some of the components related to the present invention in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device may also include any other appropriate components.

[0186] Exemplary computer program products and computer-readable storage media

[0187] In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present invention described in the above-mentioned "Exemplary Method" section of this specification.

[0188] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0189] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for information mining of historical change records according to various embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0190] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0191] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0192] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0193] The block diagrams of the devices, systems, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, systems, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0194] The method and system of the present invention may be implemented in many ways. For example, the method and system of the present invention may be implemented by software, hardware, firmware or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0195] It should also be noted that in the system, device and method of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. The above description of the disclosed aspects is provided to enable any technician in the field to make or use the present invention. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined here can be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown here, but in accordance with the widest range consistent with the principles and novel features disclosed here.

[0196] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for generating a character profile based on portrait analysis, characterized in that: include: Extracting character image feature information from image data of known persons using a pre-trained classification model, and generating a portrait feature data pool based on the character image feature information, wherein the character image feature information includes: hairstyle, hair color, skin color, eyes, and expression; Performing personality calibration on each person image feature information in the portrait feature data pool according to the personality features of the known person; Extracting the image feature information of the person to be identified of each user image in the image data pool to be analyzed; Matching the image feature information of the person to be identified of each user image with the image feature information of the person in the portrait feature data pool, and determining the personality category of the image feature information of the person to be identified of each user image according to the matching result; According to the personality category of each user image, a personality portrait of each user image in the image data pool to be analyzed is generated; wherein, The loss functions of the classification model include: a cross entropy loss function and a binary cross entropy loss function; The cross entropy loss function is: L=-∑Yi log(Pi) In the formula, Yi and Pi represent the true label and predicted value of the i-th image data respectively; The binary cross entropy loss function is: L=-1 / N*∑[y*log(p)+(1-y)*log(1-p)] Where N is the total number of pixels, y is the binary true label of each pixel, and p is the predicted value of each pixel; Use the pre-trained classification model to extract person image feature information from image data of known persons, including: Converting the image data into a grayscale image; Loading the pre-trained classification model to classify the grayscale image and extract the character image feature information, wherein the classification model includes: a face detector and a face key point detector; Loading the pre-trained classification model to classify the grayscale image and extracting the character image feature information includes: Performing face detection on the grayscale image by using the face detector to determine a face grayscale image; Extracting features from the grayscale image of the human face to determine feature information of the human face image; Extracting features from the grayscale image of the face to determine feature information of the person image includes: Extracting the hairstyle from the grayscale image of the face to determine the hairstyle feature information; Extracting hair color from the grayscale image of the face to determine hair color feature information; Extracting skin color from the grayscale image of the face to determine skin color feature information; Extracting eyes from the grayscale image of the face to determine eye feature information; Extracting facial expressions from the grayscale image to determine facial feature information; Performing hairstyle extraction on the face grayscale image to determine hairstyle feature information includes: Detect the key points of the face using a face key point detector; Determining an initial hair position of a hair region according to a face shape and a position of the face grayscale image; The initial hair position is adjusted according to the distance and angle relationship between the key points to determine the hair area; Segmenting the hair region using a threshold segmentation method to obtain a hair binary mask; A texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information.

2. The method according to claim 1, characterized in that The initial hair position is adjusted according to the distance and angle relationship between the key points to determine the hair area, including: Selecting the predetermined key points as references to determine the head shape, wherein the predetermined key points include eyebrows and cheekbones; According to the head shape, the hair boundary and the head skin area are divided by a preset distance threshold and a preset angle threshold to determine the hair boundary, wherein the preset distance threshold and the preset angle threshold are set according to average distance and standard deviation statistical information; The hair region is determined according to the hair boundary.

3. The method according to claim 1, characterized in that The hair region is segmented using a threshold segmentation method to obtain a hair binary mask, including: grayscale or color space conversion is performed on the hair region to determine a hair region conversion image; Segmenting the hair and background in the hair region conversion image according to a preset threshold to determine the hair foreground and background regions, wherein the preset threshold is set according to an adaptive threshold selection method; A morphological operation is used to perform post-processing operations on the hair foreground and the background area to obtain the hair binary mask.

4. The method according to claim 1, characterized in that: The texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information, including: extracting an image block of a hair region from the image data according to the hair binary mask; Performing grayscale conversion or color space conversion on the image block to determine a converted image block; extracting texture features of the hair area from the converted image block using a local binary pattern algorithm; Encoding the texture feature to determine a coding vector; Repeatedly extracting and encoding texture according to the hair binary mask to obtain multiple encoding vectors; The plurality of encoding vectors are merged to obtain the hairstyle feature information.

5. A device for generating a character profile based on portrait analysis, characterized in that: include: A first extraction module is used to extract character image feature information of image data of known persons using a pre-trained classification model, and generate a portrait feature data pool according to the character image feature information, wherein the character image feature information includes: hairstyle, hair color, skin color, eyes, and expression; A calibration module, used for calibrating the personality of each person image feature information in the portrait feature data pool according to the personality characteristics of the known person; The second extraction module is used to extract the image feature information of the person to be identified of each user image in the image data pool to be analyzed; A matching module, used for matching the image feature information of the person to be identified of each user image with the image feature information of the person in the portrait feature data pool, and determining the personality category of the image feature information of the person to be identified of each user image according to the matching result; A generating module is used to generate a character character portrait of each user image in the image data pool to be analyzed according to the character category of each user image; wherein, The loss functions of the classification model include: a cross entropy loss function and a binary cross entropy loss function; The cross entropy loss function is: L=-∑Yi log(Pi) In the formula, Yi and Pi represent the true label and predicted value of the i-th image data respectively; The binary cross entropy loss function is: L=-1 / N*∑[y*log(p)+(1-y)*log(1-p)] Where N is the total number of pixels, y is the binary true label of each pixel, and p is the predicted value of each pixel; The first extraction module uses a pre-trained classification model to extract person image feature information from image data of known persons, including: A conversion submodule, used for converting the image data into a grayscale image; An extraction submodule, used for loading the pre-trained classification model to classify the grayscale image and extract the character image feature information, wherein the classification model includes: a face detector and a face key point detector; Extract submodules, including: A first determining unit, configured to perform face detection on the grayscale image by using the face detector to determine a face grayscale image; A second determining unit is used to extract features from the face grayscale image to determine feature information of the person image; The second determining unit includes: A first determining subunit is used to extract the hairstyle from the grayscale image of the face and determine the hairstyle feature information; A second determination subunit is used to extract hair color from the face grayscale image to determine hair color feature information; A third determination subunit is used to extract skin color from the grayscale image of the face and determine skin color feature information; A fourth determining subunit is used to extract eyes from the face grayscale image to determine eye feature information; A fifth determination subunit, configured to extract expression from the grayscale face image and determine expression feature information; The first determining subunit comprises: Detect the key points of the face using a face key point detector; Determining an initial hair position of a hair region according to a face shape and a position of the face grayscale image; The initial hair position is adjusted according to the distance and angle relationship between the key points to determine the hair area; Segmenting the hair region using a threshold segmentation method to obtain a hair binary mask; A texture feature extraction algorithm is used to extract the texture of the hair binary mask to determine the hairstyle feature information.

6. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to any one of claims 1 to 4.

Images