Video person identification method and device, electronic equipment and storage medium

Abstract

The application provides a video character recognition method and device, electronic equipment and storage medium, wherein the video character recognition method comprises: acquiring a face feature vector of each image frame in a to-be-recognized video; for each face feature vector, adding the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, the similar face feature vector matrix being a matrix with a similarity greater than a preset threshold or a newly-built matrix in a similar face feature vector matrix set; matching the similar face feature vectors in the target similar face feature vector matrix with preset video characters in a preset video character library to recognize a target video character in the to-be-recognized video. The method can avoid the defect that the to-be-recognized video is difficult to recognize the video character due to blurring and serious degradation, thereby greatly improving the accuracy of recognizing the video character from the blurred and seriously degraded video, and the method is widely applicable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to a video character recognition method and device, an electronic device and a storage medium. BACKGROUND

[0002] At present, the character recognition method refers to a technology capable of recognizing or verifying the identity of the subject in an image, and with the increasing prevalence of movies, TV series and variety shows, how to recognize video characters in video is particularly important.

[0003] In related technologies, the video frames in the video are usually detected for faces, and then the detected face images are extracted for features, and then the video characters in the video are recognized based on the extracted face feature information.

[0004] However, when there are problems such as low face resolution, large scale variation range, severe illumination and pose variation, disguise, occlusion and shooting angle in the video, blurring and degradation will occur, resulting in low-quality face images, and when the face image is degraded too much, important or key face feature information will inevitably be lost when using the existing video character recognition method to extract features, resulting in low accuracy of recognizing video characters from blurred and severely degraded video and limited application scope. SUMMARY

[0005] The present application provides a video character recognition method, device, electronic device and storage medium to solve the problem of low accuracy of recognizing video characters from blurred and severely degraded video and limited application scope in the prior art, greatly improving the accuracy of recognizing video characters from blurred and severely degraded video and having a wide application scope.

[0006] The present application provides a video character recognition method, comprising:

[0007] obtaining face feature vectors of each image frame in a to-be-recognized video;

[0008] For each face feature vector, the face feature vector is added to a similar face feature vector matrix to determine a target similar face feature vector matrix, and the similar face feature vector matrix is a matrix or a newly created matrix in a similar face feature vector matrix set having a similarity greater than a preset threshold with the face feature vector;

[0009] The similar face feature vectors in the target similar face feature vector matrix are matched with preset video characters in a preset video character library to recognize target video characters in the to-be-recognized video.

[0010] According to the video character recognition method provided by the application, the face feature vector is added to the similar face feature vector matrix set, and the target similar face feature vector matrix is determined.

[0011] In the case that the similar face feature vector matrix set is empty, the face feature vector is added to the newly built similar face feature vector matrix, and the target similar face feature vector matrix is determined.

[0012] In the case that the similar face feature vector matrix set is not empty, according to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, the face feature vector is added to the similar face feature vector matrix matched with the face feature vector, or the face feature vector is added to the newly built similar face feature vector matrix, and the target similar face feature vector matrix is determined.

[0013] According to the video character recognition method provided by the application, the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined.

[0014] For each similar face feature vector matrix, the similarity between the face feature vector and each similar face feature vector in the similar face feature vector matrix is determined.

[0015] The average of each similarity is determined as the similarity between the face feature vector and the similar face feature vector matrix.

[0016] The maximum similarity in the similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined as the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set.

[0017] According to the video character recognition method provided by the application, the face feature vector is added to the similar face feature vector matrix matched with the face feature vector, or the face feature vector is added to the newly built similar face feature vector matrix, and the target similar face feature vector matrix is determined according to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set.

[0018] According to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, the size relationship between the target similarity and a first preset threshold is compared.

[0019] In a case where the target similarity is greater than or equal to the first preset threshold, the face feature vector is added to a similar face feature vector matrix corresponding to the target maximum similarity, and the target similar face feature vector matrix is determined.

[0020] In a case where the target similarity is less than the first preset threshold, the face feature vector is added to a newly created similar face feature vector matrix, and the target similar face feature vector matrix is determined.

[0021] According to the video character recognition method provided by the application, the similar face feature vectors in the target similar face feature vector matrix are matched with preset video characters in a preset video character library, and a target video character in the video to be recognized is recognized, which comprises the following steps:

[0022] In a case where the maximum dimension of the target similar face feature vector matrix is greater than or equal to a second preset threshold, the character feature similarity between each similar face feature vector in the target similar face feature vector matrix and each preset video character in the preset video character library is determined.

[0023] Based on a plurality of character feature similarities, the number of similar face feature vectors with the most similar character features to the same preset video character in the preset video character library is determined.

[0024] In a case where the number is greater than a third preset threshold, the same preset video character in the preset video character library is identified as a target video character in the video to be recognized.

[0025] According to the video character recognition method provided by the application, the face feature vector of each image frame in the video to be recognized is obtained, which comprises the following steps:

[0026] The video to be recognized is sequentially decoded to obtain a plurality of ordered image frames of the video to be recognized.

[0027] Face information detection is performed on each ordered image frame.

[0028] After each detected face information is cropped and aligned, a feature vector is extracted, so as to obtain the face feature vector of each image frame in the video to be recognized.

[0029] According to the video character recognition method provided by the application, the face information detection performed on each ordered image frame comprises the following steps:

[0030] Each ordered image frame is input into a preset face information detection model for face information detection; the preset face information detection model is a face information detection model pre-trained to convergence.

[0031] According to the video character recognition method provided by the application, the feature vector is extracted after the cutting and alignment based on each detected face information, so as to obtain the face feature vector of each image frame in the video to be recognized, which comprises the following steps:

[0032] After the cutting and alignment based on each detected face information, the face information after cutting and alignment is obtained.

[0033] The face feature vector of each image frame in the video to be recognized is obtained by inputting the face information after cutting and alignment into a preset face feature vector extraction model for face feature vector extraction; the preset face feature vector extraction model is a face feature vector extraction model pre-trained to convergence.

[0034] The application further provides a video character recognition device, which comprises the following components:

[0035] The acquisition module is configured to acquire the face feature vector of each image frame in the video to be recognized.

[0036] The determination module is configured to, for each face feature vector, add the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, wherein the similar face feature vector matrix is a matrix in a similar face feature vector matrix set having a similarity greater than a preset threshold with the face feature vector or a newly created matrix.

[0037] The recognition module is configured to match the similar face feature vector in the target similar face feature vector matrix with a preset video character in a preset video character library to recognize a target video character in the video to be recognized.

[0038] The application further provides an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the video character recognition method of any of the above when executing the program.

[0039] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the video character recognition method of any of the above.

[0040] The video character recognition method, device, electronic equipment and storage medium provided by the application, wherein the video character recognition method, the terminal device determines the face feature vector matching of each image frame in the to-be-recognized video first, then adds the face feature vector to the similar face feature vector matrix for each face feature vector, determines the target similar face feature vector matrix, the similar face feature vector matrix is the matrix or newly-built matrix in the similar face feature vector matrix set and the similarity between the face feature vector is greater than the preset threshold, and finally the similar face feature vector in the target similar face feature vector matrix is matched with the preset video character in the preset video character library to recognize the target video character in the to-be-recognized video. With the technical means of combining all the face feature vectors of the same video character appearing multiple times in multiple face feature vectors into a target similar face feature vector matrix and matching the target similar face feature vector matrix with each preset video character, the defect that the to-be-recognized video is difficult to recognize the video character due to blurring and serious degradation is avoided, so that the accuracy of recognizing the video character from the blurred and seriously degraded video is greatly improved, and the application range is wide. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 It is a flowchart of the video character recognition method provided by the application;

[0043] Figure 2 It is a flowchart of face information detection provided by the application;

[0044] Figure 3 It is a process diagram of determining the target similar face feature vector matrix provided by the application;

[0045] Figure 4 It is a process diagram of constructing the similar face feature vector matrix set provided by the application;

[0046] Figure 5 It is a process diagram of recognizing the target video character provided by the application;

[0047] Figure 6 It is a structure diagram of the video character recognition device provided by the application;

[0048] Figure 7 It is a structure diagram of the electronic equipment provided by the application. DETAILED DESCRIPTION

[0049] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] At present, the face recognition method refers to a technology capable of recognizing or verifying the identity of a subject in an image, and with the increasing prevalence of movies, TV series and variety shows, how to recognize video characters in video and television videos is particularly important.

[0051] It is considered that the face recognition method is usually composed of four parts of face detection, face alignment, face representation and face matching. The face detection is used for detecting the position of a face in an image and returning face key point coordinate information. The face alignment is used for performing a radiation transformation according to the face key point coordinate information and detecting a face region. The face representation is used for extracting face feature information from the detected face region. The face matching is used for obtaining a similarity score by comparing two face feature information and indicating whether the two face feature information come from the same person.

[0052] However, when there are problems such as low face resolution, large scale variation range, severe illumination and pose variation, disguise, occlusion and shooting angle in video and television videos, blurring and degradation will be caused, and low-quality face images will be generated. Therefore, when the face image is degraded too much, important or key face feature information will inevitably be lost when using the existing target recognition method to extract features, thereby leading to low accuracy of recognizing video characters and limited application scope.

[0053] In order to solve the above technical problems, the present application provides a video character recognition method, device, electronic equipment and storage medium, wherein the execution subject of the video character recognition method can be a terminal device or a server. The terminal device or the server at least has a video decoding function, an information detection function, an information extraction function, an information comparison function and a data processing function. In addition, the terminal device can be a personal computer (PC), a portable device, a notebook computer, a smart phone, a tablet computer and other electronic devices such as a portable wearable device. The server can use an independent server or a server cluster composed of multiple servers, such as a physical server containing an independent host, a virtual server carried by a host cluster, a cloud server and the like. The specific form of the terminal device or the server is not limited in the present application. The following method embodiments are described taking the terminal device as an example.

[0054] Referring to Figure 1 , a flowchart of a video character recognition method provided by an embodiment of the present application is shown in Figure 1 , the video character recognition method comprises the following steps:

[0055] Step 110, obtaining a face feature vector of each image frame in a to-be-recognized video.

[0056] The to-be-recognized video can be part or all of a movie, a TV series or other video, such as a movie or a TV episode, or a complete movie or TV series. Here, no specific limitation is made. The to-be-recognized video is blurred and degraded due to low face resolution, large scale variation range, drastic changes in illumination and posture, disguising, occlusion and shooting angle, etc. In addition, each face feature vector can be used to represent the features of the face information contained in the corresponding image frame in the to-be-recognized video. The face information of the corresponding image frame in the to-be-recognized video can correspond to a face feature vector, and the face information of the corresponding image frame in the to-be-recognized video can be the face information of a movie star, a crowd actor, an audience or other staff. Here, no specific limitation is made.

[0057] Specifically, the terminal device can obtain the to-be-recognized video by manually inputting the to-be-recognized video in a client application connected to the terminal device, or can obtain the to-be-recognized video by selecting at least one video from a pre-stored set of video and uploading the to-be-recognized video to the processing module of the terminal device, such as a processor, through a video information uploading module.

[0058] Further, the terminal device can obtain the face feature vector of each image frame in the to-be-recognized video by using a pre-stored extraction method for extracting the face feature vector of the input video. Alternatively, the terminal device can directly receive the face feature vector of each image frame in the to-be-recognized video uploaded by the user, such as by manually inputting the face feature vector of each image frame in the to-be-recognized video on the terminal device, or by manually inputting the face feature vector of each image frame in the to-be-recognized video in a client application connected to the terminal device, or by uploading the face feature vector of each image frame in the to-be-recognized video. Here, no specific limitation is made to the way the terminal device obtains the to-be-recognized video and the face feature vector of each image frame in the to-be-recognized video.

[0059] Step 120, for each face feature vector, adding the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, the similar face feature vector matrix being a matrix in the similar face feature vector matrix set having a similarity greater than a preset threshold with the face feature vector or a newly built matrix.

[0060] The target similar face feature vector matrix contains at least one similar face feature vector. When the similar face feature vector matrix set is not an empty set, there is at least one similar face feature vector matrix, and the similar face feature vector matrix contains at least one face feature vector. In addition, when there are at least two face feature vectors in the similar face feature vector matrix, the at least two face feature vectors are similar face feature vectors belonging to the same type of person feature.

[0061] Specifically, the terminal device first constructs a similar face feature vector matrix set R, which is used to classify face feature vectors with similar relationships into a similar face feature vector matrix and is initially an empty set; and as the classification process for each face feature vector continues, the number of category index values of the similar face feature vector matrix set R and the dimensions of the similar face feature vector matrix corresponding to each category index value will also increase, and finally the similar face feature vector matrix set R will save n similar face feature vector matrices, n being the number of category index values, i.e., the total number of persons appearing in the video to be identified; and the similar face feature vector matrix corresponding to each category index value can be A i (j x m), i e 1, 2, …, n, i being the current category index value, the similar face feature vector matrix A i The 1 x m size similar face feature vector of the i-th person (i.e., the i-th type of person) appearing in the video to be identified is combined to be the face information detected j times, j e 1, 2, …, k, k and n being positive integers; the similar face feature vector matrix A i The size of each similar face feature vector in the similar face feature vector matrix A

[0062] Based on this, the terminal device can perform similarity analysis on each face feature vector and the similar face feature vector matrix set R, and based on the similarity analysis result, determine whether to add the current face feature vector to the matrix in the similar face feature vector matrix set R with a similarity greater than a preset threshold, or newly build a matrix in the similar face feature vector matrix set R and add it, thereby determining the target similar face feature vector matrix corresponding to the current face feature vector. Then, the next face feature vector is selected as the current face feature vector, and step 120 is repeatedly executed. Until the target similar face feature vector matrix corresponding to each face feature vector is obtained.

[0063] Step 130, matching the similar face feature vectors in the target similar face feature vector matrix with the preset video characters in the preset video character library, identifying the target video character in the video to be identified.

[0064] Wherein, the preset video character library Y is a r x m dimensional matrix constructed in advance and composed of r 1 x m size preset face feature vectors, each 1 x m size preset face feature vector corresponds to a preset video character, and each preset video character can be a standard video character or a reference video character corresponding to a movie star. In addition, r and m are positive integers.

[0065] Specifically, the terminal device matches each similar face feature vector in the target similar face feature vector matrix with each preset video character in the preset video character library, so as to determine which preset video character in the preset video character library has the highest matching degree with all similar face feature vectors in the target similar face feature vector matrix, and the preset video character corresponding to the highest matching degree is determined as the target video character in the video to be identified.

[0066] The video character recognition method provided by the application, the terminal device determines the face feature vector matching of each image frame in the to-be-recognized video first, then adds the face feature vector to a similar face feature vector matrix for each face feature vector, determines a target similar face feature vector matrix, the similar face feature vector matrix is a matrix with a similarity greater than a preset threshold in a similar face feature vector matrix set or a newly-built matrix, and finally matches the similar face feature vector in the target similar face feature vector matrix with a preset video character in a preset video character library to recognize a target video character in the to-be-recognized video. With the technical means of combining all the face feature vectors of the same video character appearing multiple times in multiple face feature vectors into a target similar face feature vector matrix and matching the target similar face feature vector matrix with each preset video character, the defect that the to-be-recognized video is difficult to recognize the video character due to blurring and serious degradation is avoided, so that the accuracy of recognizing the video character from the blurred and seriously degraded video is greatly improved, and the application range is wide.

[0067] Optionally, the specific implementation process of step 110 can include:

[0068] First, the to-be-recognized video is sequentially decoded to obtain a plurality of ordered image frames of the to-be-recognized video; then each ordered image frame is further detected for face information; then the feature vector is extracted after the alignment and cutting based on each detected face information, so as to obtain the face feature vector of each image frame in the to-be-recognized video.

[0069] Specifically, the terminal device can sequentially decode the to-be-recognized video uploaded by the video information uploading module in combination with the video stream characteristics of the to-be-recognized video, so as to decode the to-be-recognized video into a plurality of ordered image frames, that is, to obtain a plurality of ordered image frames of the to-be-recognized video, such as image frame 1, image frame 2, …, image frame K, K is a positive integer greater than 1; if 25 ordered image frames are decoded per second and the to-be-recognized video is a 40-second video segment taken from a movie, K = 25 * 40.

[0070] Further, the terminal device sequentially performs face information detection on each ordered image frame. Since the ordered image frames have the possibility of detecting multiple face information, in the case that the ordered image frame detects face information, label management can be performed on the detected at least one face information, so that the at least one face information detected by the ordered image frame carries a label. For example, when image frame 2 detects 5 face information, the labels of the 5 face information detected can be face information 1, face information 2, face information 3, face information 4 and face information 5 in turn. At this time, the feature vectors are extracted after the at least one face information detected by the ordered image frame is cropped and aligned, and the face feature vectors extracted from the ordered image frame are obtained. The number of face information detected from the ordered image frame is the same as the number of face feature vectors extracted and one-to-one correspondence, for example, when image frame 2 detects 5 face information, image frame 2 can correspond to 5 face feature vectors extracted. In this way, the face feature vectors of each image frame in the video to be recognized are obtained.

[0071] It should be noted that for the extraction of the face feature vector, a network model FaceNet based on a convolutional neural network (CNN) can be used to extract the face feature vector, and a new loss function is designed. FaceNet maps the face to a 128-dimensional vector space and designs a Triplet loss function. The independent variables of the Triplet loss function include three parts: a certain face (called anchor), a face matching the person (called positive sample), and a face not matching the person (called negative sample). The goal is to train the anchor to be close to the positive and far from the negative. In this way, the face feature vector can be extracted. Other network models for extracting face feature vectors can also be used for extraction. Herein, no specific limitation is made.

[0072] It should be noted that for the K ordered image frames, face information detection is sequentially performed. If image frame 1 detects face information, the face feature vector is extracted corresponding to the detected at least one face information. Otherwise, if image frame 1 does not detect face information, face information detection is performed on the next image frame (i.e. image frame 2). Until the face information detection is completed for the K ordered image frames.

[0073] The video character recognition method provided by the embodiment of the application obtains the face feature vector of each image frame in the video to be recognized by sequentially performing face information detection on the multiple ordered image frames of the video to be recognized decoded and extracting a feature vector after each detected face information is cropped and aligned. In this way, the decoding video, face information detection, cropping and alignment, and feature vector extraction technologies are combined to improve the efficiency and reliability of obtaining the face feature vector, thereby laying a foundation for the accurate recognition of the subsequent video characters.

[0074] Optionally, the face information detection is performed on each ordered image frame, and the specific implementation process can include:

[0075] Each ordered image frame is input into a preset face information detection model for face information detection. The preset face information detection model is a face information detection model that is pre-trained to convergence.

[0076] Specifically, in order to improve the rate of video decoding and the accuracy of face information detection, the characteristics of a central processing unit (CPU) and a graphics processing unit (GPU) can be combined for video decoding and face information detection. For details, refer to the flowchart of face information detection shown in FIG. 8. Figure 2 When the terminal device decodes the video to be recognized uploaded by the video information uploading module, the GPU and the CPU can be combined to extract multiple ordered image frames. When each ordered image frame is input into a preset face information detection model for face information detection, the GPU and the CPU can be combined to detect and recognize each ordered image frame, so as to obtain the label of each detected face information, thereby achieving the purpose of label management.

[0077] It should be noted that when the preset face information detection model is used for face information detection and recognition, there will be many linear operations, such as operations on each face feature vector. The data related to linear operations can be processed on the GPU, so as to improve the operation speed. In addition, in the optimization of each existing neural network model, the feature extraction network and the complex loss function are carefully designed, and the correct rate of face recognition is improved by continuous iteration training. Compared with this, the method of using the existing preset face information detection model trained to convergence for face information detection is simple in algorithm, easy to understand, and widely applicable.

[0078] The video character recognition method provided by the embodiment of the present application greatly improves the rapid efficiency and reliable accuracy of face information detection by inputting each ordered image frame into the preset face information detection model for face information detection. Since the preset face information detection model is a face information detection model that is pre-trained to convergence, the problem of high computational complexity and limited application range caused by constructing a neural network model, designing a loss function and constantly iterative training is avoided. The accuracy of face information detection is improved, and the application range is also very wide.

[0079] Optionally, the feature vector is extracted after each detected face information is cropped and aligned, so as to obtain the face feature vector of each image frame in the to-be-recognized video. The specific implementation process can include:

[0080] Firstly, after each detected face information is cropped and aligned, the face information after cropping and alignment is obtained. Then, the face feature vector of each cropped and aligned face information is extracted by inputting the face feature vector into the preset face feature vector extraction model, so as to obtain the face feature vector of each image frame in the to-be-recognized video. The preset face feature vector extraction model is a face feature vector extraction model that is pre-trained to convergence.

[0081] Specifically, the terminal device can first crop and align each detected face information to obtain the face information after cropping and alignment. The cropping and alignment method can use the existing conventional cropping and alignment method. Then, the face feature vector of each cropped and aligned face information is extracted by inputting the face feature vector into the preset face feature vector extraction model. When the face feature vector of each cropped and aligned face information is input into the preset face feature vector extraction model for face feature vector extraction, GPU and CPU combination can also be used for face feature vector extraction to improve the operation speed. The method of using the existing pre-trained face feature vector extraction model for face feature vector extraction is simple in algorithm and easy to understand, and can be further widely applied.

[0082] It should be noted that the present application can use existing face detection algorithms such as RetinaFace, Multi-task Convolutional Neural Network (MTCNN) and other algorithms to locate face information to obtain coordinate information, and then use existing face recognition algorithms such as AdaFace, FaceNet and other algorithms to extract face feature vectors from the face information, and use formula (1) to determine whether the two extracted face feature vectors are similar:

[0083] p=ab T (1)

[0084] In formula (1), p represents a similarity degree of two face feature vectors a and b, and the superscript T is a transposition operation.

[0085] The video character recognition method provided by the embodiment of the present application can greatly improve the rapidity, efficiency, reliability and accuracy of extracting face feature vectors by using the preset face feature vector extraction model to extract face feature vectors from each cropped and aligned face information. Since the preset face feature vector extraction model is a face feature vector extraction model trained to convergence in advance, the problem of high computational complexity and limited application range caused by constructing a neural network model, designing a loss function and constantly iterative training can be avoided. The accuracy of face feature vector extraction is improved, and the application range is wide.

[0086] Optionally, the specific implementation process of adding the face feature vector to the similar face feature vector matrix to determine the target similar face feature vector matrix in step 120 can include:

[0087] In the case that the similar face feature vector matrix set is an empty set, the face feature vector is added to a newly created similar face feature vector matrix to determine the target similar face feature vector matrix. In the case that the similar face feature vector matrix set is a non-empty set, the face feature vector is added to the similar face feature vector matrix matched with the face feature vector or added to a newly created similar face feature vector matrix according to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set to determine the target similar face feature vector matrix.

[0088] Specifically, the terminal device can determine the target similarity and the class index value corresponding to the target maximum similarity between the face feature vector and the similar face feature vector matrix set R, that is, determine the target similarity and the index value of the target similarity between the face feature vector and the similar face feature vector matrix A i in the similar face feature vector matrix set R.

[0089] Based on this, the terminal device determines the target similarity and the index value of the target similarity between the current face feature vector a in each face feature vector and the similar face feature vector matrix A iThe maximum similarity p and the corresponding category index value g are analyzed and judged. When the set of similar face feature vector matrices R is determined to be an empty set, the face feature vector a is determined to be the similar face feature vector matrix A1 corresponding to the first category index value in the set of similar face feature vector matrices R, that is, A1 = a. When the set of similar face feature vector matrices R is determined to be a non-empty set, the face feature vector a is added to the similar face feature vector matrix in the set of similar face feature vector matrices R that matches the face feature vector a, or the face feature vector a is added to a newly created similar face feature vector matrix, thereby determining the target similar face feature vector matrix.

[0090] For example, you can refer to Figure 3 The diagram shown illustrates the process of determining the feature vector matrix of similar faces to the target. Figure 3 In this example, for two facial feature vectors [0.24, 0.23, ..., 0.92] and [0.12, 0.73, ..., 0.52] extracted from an ordered image frame, by analyzing the calculated target similarity and the corresponding category index value, [0.24, 0.23, ..., 0.92] can be added to the existing similar face feature vector matrix corresponding to the category index value, and [0.12, 0.73, ..., 0.52] can be used as a newly created similar face feature vector matrix corresponding to the category index value in the set R of similar face feature vector matrices.

[0091] The video person recognition method provided in this invention involves a terminal device determining a target similar face feature vector matrix by adding face feature vectors to the index value of a newly created similar face feature vector matrix when the set of similar face feature vector matrices is empty, or adding face feature vectors to a similar face feature vector matrix that matches the face feature vectors when the set of similar face feature vector matrices is not empty, or adding face feature vectors to a newly created similar face feature vector matrix. This method categorizes current face feature vectors based on face feature vectors already appearing in the video to be recognized within the set of similar face feature vector matrices. As this benchmark expands during the recognition process, the face features used for categorization become increasingly rich, effectively classifying low-quality faces correctly and significantly improving the accuracy and reliability of determining similar face feature vector matrices that match multiple face feature vectors.

[0092] Optionally, determining the target similarity between the face feature vector and each similar face feature vector matrix in the set of similar face feature vector matrices can be implemented in a specific way, including:

[0093] Firstly, the similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined; then the maximum similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined as the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set.

[0094] Specifically, for each similar face feature vector matrix, the terminal device first determines the similarity between the face feature vector and each similar face feature vector in the similar face feature vector matrix, and can determine k similarities when the similar face feature vector matrix contains k face feature vectors; then the average of the determined multiple similarities is calculated, for example, the average of the k similarities is calculated, and the result obtained after the average is calculated is the similarity between the face feature vector and the similar face feature vector matrix. In this way, the similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined. If there are n similarities, the maximum similarity in the n similarities is determined as the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set. This process can be implemented by using formulas (2)-(4):

[0095]

[0096] p = f max (P) (3)

[0097] g = f argmax (P) (4)

[0098] In formulas (2)-(4), f mean is an average operation, f max is a maximum operation, f argmax is a maximum index operation, A i ∈ R, y ∈ 1, 2, …, n, a is the current face feature vector, P i is the similarity between the face feature vector a and the similar face feature vector matrix A i of the similar face feature vector matrix set R, P is the similarity set between the face feature vector a and the similar face feature vector matrix set R, p is the target similarity in the similarity set P, and g is the category index value corresponding to the target similarity p.

[0099] The video character recognition method provided by the embodiment of the present application can effectively improve the reliability and accuracy of determining the target similarity between the face feature vector and the similar face feature vector matrix set, and provide reliable guarantee for the subsequent accurate recognition of the video character.

[0100] Optionally, according to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, the face feature vector is added to the similar face feature vector matrix matched with the face feature vector, or the face feature vector is added to the newly created similar face feature vector matrix, to determine the target similar face feature vector matrix, and the specific implementation process can include:

[0101] According to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, the size relationship between the target similarity and the first preset threshold is compared; in the case that the target similarity is greater than or equal to the first preset threshold, the face feature vector is added to the similar face feature vector matrix corresponding to the target maximum similarity, to determine the target similar face feature vector matrix; or in the case that the target similarity is less than the first preset threshold, the face feature vector is added to the newly created similar face feature vector matrix, to determine the target similar face feature vector matrix.

[0102] Specifically, for the current face feature vector a, the similar face feature vector matrix A in the similar face feature vector matrix set R i The target similarity p calculated by formula (2)-(4) and the category index value g corresponding to the target similarity p can be specifically analyzed and determined according to the following conditions:

[0103] Case one: if the similar face feature vector matrix set R is not an empty set and the target similarity p is greater than or equal to the first preset threshold, it is determined that the similar face feature vector matrix A corresponding to the gth category index value in the similar face feature vector matrix set R g (j×m, j∈1, 2, …, k) adds the one-dimensional face feature vector a, that is, A g [j+1]=a, and j=j+1; to determine the target similar face feature vector matrix A g ;

[0104] Scenario 2: If the set R of similar face feature vector matrices is not empty and the target similarity p is less than the first preset threshold, then it is determined to add a new similar face feature vector matrix A corresponding to the category index value to the set R of similar face feature vector matrices. n+1 In the similar face feature vector matrix A n+1 Add a facial feature vector a, i.e., A n+1 =a, and let n = n+1; thus determine the target similarity face feature vector matrix A. n .

[0105] It should be noted that if the set R of similar face feature vector matrices is not empty and the target similarity between face feature vector a and each similar face feature vector matrix in the set R is low, it is also necessary to add a similar face feature vector matrix corresponding to the category index value to the set R of similar face feature vector matrices and add this face feature vector a.

[0106] It should be noted that the terminal device can classify the face feature vectors of each image frame in the video to be identified into similar face feature vectors by applying equations (2) to (4) and the above analysis process to each face feature vector in the acquired video to be identified, until the construction of the similar face feature vector matrix set is completed. For example, refer to Figure 4 The diagram shown illustrates the process of constructing a set of similar face feature vector matrices. Figure 4 In this process, for multiple ordered image frames of the video to be recognized, the facial information in each ordered image frame is detected sequentially, and after cropping and alignment, the feature vector is extracted to obtain the facial feature vector corresponding to each facial information. Figure 4 The feature vectors shown are: [feature vector 1] corresponding to face information 1, [feature vector 2] corresponding to face information 2, [feature vector 3] corresponding to face information 3, [feature vector 4] corresponding to face information 4, [feature vector 5] corresponding to face information 5, [feature vector 6] corresponding to face information 6, ..., [feature vector n] corresponding to face information n, [feature vector n+1] corresponding to face information n+2, and [feature vector n+2] corresponding to face information n+2, ...; and similar face feature vectors are sequentially categorized to construct a set R of similar face feature vector matrices. Figure 4 A1 and A2 are shown.

[0107] The video character recognition method provided by the embodiment of the present application can determine whether the current face feature vector is added to the similar face feature vector matrix corresponding to the existing class index value in the similar face feature vector matrix set or is classified into the similar face feature vector matrix corresponding to the new class index value in the similar face feature vector matrix set by comparing the target similarity of the current face feature vector and the similar face feature vector matrix set with the first preset threshold size when the similar face feature vector matrix set is empty. The method of clustering the subsequent face feature vectors based on the previous face feature vector in the video to be recognized can ensure the correct clustering of the low-quality face information in the video to be recognized, thereby effectively improving the accuracy of the subsequent video character recognition.

[0108] Optionally, the specific implementation process of step 130 can include:

[0109] Firstly, in the case that the maximum dimension of the target similar face feature vector matrix is greater than or equal to the second preset threshold, the character feature similarity between each similar face feature vector in the target similar face feature vector matrix and each preset video character in the preset video character library is determined; further, based on the plurality of character feature similarities, the number of similar face feature vectors most similar to the character feature of the same preset video character in the preset video character library is determined; then, in the case that the number is greater than the third preset threshold, the same preset video character in the preset video character library is identified as the target video character in the video to be recognized.

[0110] Specifically, the terminal device classifies the face feature vectors of each image frame in the video to be identified into similar face feature vectors until the construction of the similar face feature vector matrix set is completed, that is, the face feature vectors of each image frame in the video to be identified are matched and classified into different target similar face feature vector matrices, at this time, each target similar face feature vector matrix in the similar face feature vector matrix set and each preset video character in the preset video character library are matched, that is, for each target similar face feature vector matrix, whether the maximum dimension of the target similar face feature vector matrix is less than the second preset threshold is judged in turn, if the maximum dimension of the target similar face feature vector matrix is less than the second preset threshold, it can be determined that the corresponding character in the video to be identified is a character such as a groupie, an audience, etc. which does not belong to the category of video character identification, at this time, it is determined that this kind of character is not identified, and the second preset threshold here can be set to a positive integer not greater than 4; otherwise, if the maximum dimension of the target similar face feature vector matrix is greater than or equal to the second preset threshold, the character feature similarity between each similar face feature vector in the target similar face feature vector matrix and each preset video character in the preset video character library is determined, and based on the determined multiple character feature similarities, the number s of similar face feature vectors most similar to the character feature of the same preset video character in the preset video character library is determined, and the index value of the same preset video character can also be determined, the calculation formulas are as shown in formulas (5) and (6):

[0111] s = f same_count (f argmax (A i Y T )) (5)

[0112] g = f index_count (f argmax (A i Y T )) (6)

[0113] In formulas (5) and (6), f same_count is an operation for finding the maximum number of character feature similarities, f index_count is an operation for finding the index value of the maximum number of character feature similarities, s and g represent that s similar face feature vectors in all similar face feature vectors in the target similar face feature vector matrix have the highest character feature similarity with the gth preset video character in the preset video character library Y, and f argmax is an operation for finding the maximum index value.

[0114] Based on this, the terminal device can determine the video character recognition result according to the size relationship between the number s and the third preset threshold value, that is, when the number s is less than or equal to the third preset threshold value, it can be determined that the character corresponding to the to-be-recognized video in the target similar face feature vector matrix is not recognized; otherwise, when the number s is greater than the third preset threshold value, it can be determined that the character corresponding to the to-be-recognized video in the target similar face feature vector matrix is the gth preset video character in the preset video character library Y. The third preset threshold value can be the maximum integer greater than j / 2, and j is the maximum dimension of the similar face feature vector matrix A i For example, for the case where the maximum dimension of the target similar face feature vector matrix is 50, if 26 similar face feature vectors refer to the gth preset video character in the preset video character library Y, then the character corresponding to the to-be-recognized video in the target similar face feature vector matrix is the gth preset video character in the preset video character library Y; otherwise, if 24 similar face feature vectors refer to the gth preset video character in the preset video character library Y, it is determined that the character corresponding to the to-be-recognized video in the target similar face feature vector matrix is not recognized.

[0115] It should be noted that, in order to improve the recognition accuracy, the present application adopts a voting method for video character recognition. In the recognition of the target similar face feature vector matrix corresponding to each class index value in the similar face feature vector matrix set, first, each similar face feature vector in the target similar face feature vector matrix corresponding to the class index value is matched with the preset video character library Y, so as to "vote" for the preset video character matched by each similar face feature vector. If the preset video character with the most votes exceeds the third preset threshold value, it is determined that all similar face feature vectors in the target similar face feature vector matrix corresponding to the class index value belong to the preset video character. Since all face feature vectors of the same character in the similar face feature vector matrix set have been classified into a similar face feature vector matrix, there may be low-quality face feature vectors in the class. If the low-quality face feature vectors are recognized alone, the recognition result will be incorrect to a large extent. By using the "voting" method, the low-quality face feature vectors in the class are subject to the recognition result of the high-quality face feature vectors, which effectively improves the recognition accuracy. At the same time, if the face in the class is not in the preset video character library, the "voting" will fail, preventing misrecognition.

[0116] For example, referring to the process diagram of identifying a target video character shown in Figure 5 The recognition result can be obtained as follows: the target video character in the to-be-recognized video is the 8th preset video character and the 20th preset video character in the preset video character library Y. The specific implementation process can be referred to the foregoing embodiments, which will not be described here.

[0117] The video character recognition method provided by the application, the terminal device firstly determines the character feature similarity of each similar face feature vector in the target similar face feature vector matrix and the character feature of each preset video character in the preset video character library in the case that the maximum dimension of the target similar face feature vector matrix is greater than or equal to the second preset threshold, and then determines the number of the character feature similarity most similar to the character feature of the same preset video character in the preset video character library from the plurality of character feature similarities; finally, in the case that the number is greater than the third preset threshold, the same preset video character in the preset video character library is identified as the target video character in the video to be recognized. In this way, by collecting the similar face feature vector matrices in the video to be recognized together and then using the voting method to identify the target video character for each type of target similar face feature vector matrix, the method can not only greatly reduce the identification of the characters that do not exist in the preset video character library and reduce the probability of misidentification, but also can greatly improve the identification accuracy of the video characters.

[0118] The video character recognition device provided by the application is described below, and the video character recognition device described below can be correspondingly referred to the video character recognition method described above.

[0119] Reference Figure 6 The structure schematic diagram of the video character recognition device provided by the application is shown in the figure, the video character recognition device 600 comprises: Figure 6

[0120] The acquisition module 610 is configured to acquire the face feature vectors of each image frame in the video to be recognized.

[0121] The determination module 620 is configured to, for each face feature vector, add the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, the similar face feature vector matrix being a matrix or a newly created matrix in a similar face feature vector matrix set, the similarity between the face feature vector and each face feature vector in the similar face feature vector matrix being greater than a preset threshold.

[0122] The identification module 630 is configured to match the similar face feature vector in the target similar face feature vector matrix with the preset video character in the preset video character library to identify the target video character in the video to be recognized.

[0123] Optionally, the acquisition module 610 can be specifically configured to decode the video to be recognized in sequence to acquire a plurality of ordered image frames of the video to be recognized, detect face information for each ordered image frame, and extract a feature vector after cutting and aligning based on each detected face information, thereby acquiring the face feature vectors of each image frame in the video to be recognized.

[0124] ​Optionally, the acquisition module 610 can further be specifically configured to input each ordered image frame into a preset face information detection model for face information detection, wherein the preset face information detection model is a face information detection model that is pre-trained to convergence.

[0125] Optionally, the acquisition module 610 can further be specifically configured to acquire each cropped and aligned face information based on each detected face information, input each cropped and aligned face information into a preset face feature vector extraction model for face feature vector extraction, and acquire face feature vectors of each image frame in the video to be recognized, wherein the preset face feature vector extraction model is a face feature vector extraction model that is pre-trained to convergence.

[0126] Optionally, the determination module 620 can be specifically configured to, in a case where the similar face feature vector matrix set is an empty set, add the face feature vector into a newly created similar face feature vector matrix to determine a target similar face feature vector matrix, and in a case where the similar face feature vector matrix set is a non-empty set, add the face feature vector into a similar face feature vector matrix matched with the face feature vector or a newly created similar face feature vector matrix according to a target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set to determine the target similar face feature vector matrix.

[0127] Optionally, the determination module 620 can further be specifically configured to determine, for each similar face feature vector matrix, a similarity between the face feature vector and each similar face feature vector in the similar face feature vector matrix, determine a mean value of each similarity as a similarity between the face feature vector and the similar face feature vector matrix, and determine a maximum similarity in the similarities between the face feature vector and each similar face feature vector matrix as a target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set.

[0128] Optionally, the determination module 620 can further be specifically configured to compare a size relationship between the target similarity and a first preset threshold according to the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, add the face feature vector into a similar face feature vector matrix corresponding to the target maximum similarity to determine the target similar face feature vector matrix in a case where the target similarity is greater than or equal to the first preset threshold, and add the face feature vector into a newly created similar face feature vector matrix to determine the target similar face feature vector matrix in a case where the target similarity is less than the first preset threshold.

[0129] Optionally, the identification module 630 can be specifically configured to determine the person feature similarity between each similar face feature vector in the target similar face feature vector matrix and each preset video character in the preset video character library when the maximum dimension of the target similar face feature vector matrix is greater than or equal to the second preset threshold; determine the number of similar face feature vectors most similar to the person feature of the same preset video character in the preset video character library based on the plurality of person feature similarities; and identify the same preset video character in the preset video character library as the target video character in the video to be identified when the number is greater than a third preset threshold.

[0130] Figure 7 An example of a schematic diagram of the physical structure of an electronic device is shown in FIG. 7, which can include a processor 710, a communications interface 720, a memory 730, and a communications bus 740. The processor 710, the communications interface 720, and the memory 730 can communicate with each other via the communications bus 740. The processor 710 can invoke a logical instruction in the memory 730 to execute a video character identification method, which includes: Figure 7

[0131] Obtaining a face feature vector of each image frame in a video to be identified;

[0132] For each face feature vector, adding the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, the similar face feature vector matrix being a matrix in a similar face feature vector matrix set having a similarity greater than a preset threshold with the face feature vector or a newly created matrix;

[0133] Matching the similar face feature vector in the target similar face feature vector matrix with a preset video character in a preset video character library to identify a target video character in the video to be identified.

[0134] ​Further, the logic instructions in the memory 730 described above can be implemented in the form of software functional units and sold or used as standalone products, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or partially contribute to the prior art, or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0135] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the video character recognition method provided by the above-mentioned methods, which comprises:

[0136] obtaining a face feature vector of each image frame in a to-be-recognized video;

[0137] for each face feature vector, adding the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, the similar face feature vector matrix being a matrix with a similarity greater than a preset threshold between the face feature vector and the face feature vector in the similar face feature vector matrix set or a newly created matrix;

[0138] matching the similar face feature vector in the target similar face feature vector matrix with a preset video character in a preset video character library to recognize a target video character in the to-be-recognized video.

[0139] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, the computer program is executed by a processor to implement the video character recognition method provided by the above-mentioned methods, which comprises:

[0140] obtaining a face feature vector of each image frame in a to-be-recognized video;

[0141] for each face feature vector, adding the face feature vector to a similar face feature vector matrix to determine a target similar face feature vector matrix, the similar face feature vector matrix being a matrix with a similarity greater than a preset threshold between the face feature vector and the face feature vector in the similar face feature vector matrix set or a newly created matrix;

[0142] The similar face feature vector in the target similar face feature vector matrix is matched with a preset video character in a preset video character library, and a target video character in a to-be-identified video is identified.

[0143] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0144] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course, can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0145] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A video person identification method, characterized in that, The method comprises the following steps: obtaining a face feature vector of each image frame in a to-be-identified video; the face feature vector is a face feature vector extracted from the image frame by a network model based on a convolutional neural network; for each face feature vector, if a similar face feature vector matrix set is empty, the face feature vector is added to a newly created similar face feature vector matrix to determine a target similar face feature vector matrix; if the similar face feature vector matrix set is not empty, according to a target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, the face feature vector is added to a similar face feature vector matrix matched with the face feature vector or a newly created similar face feature vector matrix to determine the target similar face feature vector matrix; the similar face feature vector matrix is a matrix in the similar face feature vector matrix set having a similarity greater than a preset threshold with the face feature vector or a newly created matrix; wherein, the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined, including: for each similar face feature vector matrix, determining a similarity between the face feature vector and each similar face feature vector in the similar face feature vector matrix; determining an average of the similarities as the similarity between the face feature vector and the similar face feature vector matrix; determining a maximum similarity in the similarities between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set as the target similarity; whether two face feature vectors extracted are similar is determined by formula (1): the target similarity calculation method is: ＝ (1) wherein, is a current face feature vector, is a face feature vector in the similar face feature vector matrix; if a maximum dimension of the target similar face feature vector matrix is greater than or equal to a second preset threshold, a character feature similarity between each similar face feature vector in the target similar face feature vector matrix and each preset video character in a preset video character library is determined; based on a plurality of character feature similarities, a number of similar face feature vectors most similar to a character feature of a same preset video character in the preset video character library is determined; if the number is greater than a third preset threshold, the same preset video character in the preset video character library is identified as a target video character in the to-be-identified video. （2） （3） （4） In formulas (2) to (4), is a mean operation, is a maximum operation, is a maximum index operation, , , is a current face feature vector, is a face feature vector and a similar face feature vector matrix set a similar face feature vector matrix of the similar face feature vector matrix set a similarity, is a face feature vector and a similarity set of the similar face feature vector matrix set a similarity set a target similarity in the similarity set a target similarity a target similarity a category index value corresponding to the target similarity the target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set is determined, the face feature vector is added to a similar face feature vector matrix matched with the face feature vector or a newly created similar face feature vector matrix, and the target similar face feature vector matrix is determined, including:

2. The video person recognition method of claim 1, wherein, ​ According to a target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, a size relationship between the target similarity and a first preset threshold is compared; In a case where the target similarity is greater than or equal to the first preset threshold, the face feature vector is added to the similar face feature vector matrix corresponding to the target maximum similarity, and the target similar face feature vector matrix is determined; In a case where the target similarity is less than the first preset threshold, the face feature vector is added to a newly created similar face feature vector matrix, and the target similar face feature vector matrix is determined.

3. The video person recognition method of claim 1 or 2, characterized in that, The face feature vector of each image frame in the video to be recognized is obtained, including: sequentially decoding the video to be recognized to obtain a plurality of ordered image frames of the video to be recognized; detecting face information for each ordered image frame; extracting a feature vector after cutting and aligning based on each detected face information, thereby obtaining the face feature vector of each image frame in the video to be recognized.

4. The video person recognition method of claim 3, wherein, The face information detection for each ordered image frame includes: inputting each ordered image frame into a preset face information detection model for face information detection; the preset face information detection model is a face information detection model pre-trained to convergence.

5. The video person recognition method of claim 3, wherein, The face feature vector of each image frame in the video to be recognized is obtained, including: after cutting and aligning based on each detected face information, obtaining each face information after cutting and aligning; inputting each face information after cutting and aligning into a preset face feature vector extraction model for face feature vector extraction, thereby obtaining the face feature vector of each image frame in the video to be recognized; the preset face feature vector extraction model is a face feature vector extraction model pre-trained to convergence.

6. A video person recognition apparatus, characterized by comprising: including: an acquisition module configured to obtain a face feature vector of each image frame in a video to be recognized; the the face feature vector is a face feature vector extracted from the image frames by using a convolutional neural network-based network model; a determination module configured to, for each face feature vector, in a case where a similar face feature vector matrix set is an empty set, add the face feature vector to a newly created similar face feature vector matrix to determine a target similar face feature vector matrix; in a case where the similar face feature vector matrix set is a non-empty set, according to a target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set, add the face feature vector to a similar face feature vector matrix matched with the face feature vector, or add the face feature vector to a newly created similar face feature vector matrix to determine the target similar face feature vector matrix, the similar face feature vector matrix being a matrix with a similarity greater than a preset threshold between the face feature vector in the similar face feature vector matrix set or a newly created matrix; The identification module is configured to, in a case where a maximum dimension of the target similar face feature vector matrix is greater than or equal to a second preset threshold, determine a character feature similarity between each similar face feature vector in the target similar face feature vector matrix and each preset video character in a preset video character library; and determine, based on a plurality of character feature similarities, a number of similar face feature vectors that are most similar to a character feature of a same preset video character in the preset video character library. In a case where the number is greater than a third preset threshold, the same preset video character in the preset video character library is identified as a target video character in the video to be identified. The determination module is further configured to, for each similar face feature vector matrix, determine a similarity between the face feature vector and each similar face feature vector in the similar face feature vector matrix; determine a mean value of each similarity as a similarity between the face feature vector and the similar face feature vector matrix; determine a maximum similarity in the similarities between the face feature vector and each similar face feature vector matrix as a target similarity between the face feature vector and each similar face feature vector matrix in the similar face feature vector matrix set; and determine, using formula (1), whether the two extracted face feature vectors are similar. ＝ (1) wherein, is a current face feature vector, is a face feature vector in the similar face feature vector matrix; The target similarity calculation manner is: （2） （3） （4） In formulas (2) to (4), is a mean operation, is a maximum operation, is a maximum index operation, , , is a current face feature vector, is a face feature vector and a similar face feature vector matrix set a similar face feature vector matrix of the similar face feature vector matrix set a similarity, is a face feature vector and a similarity set of the similar face feature vector matrix set a similarity set of the similar face feature vector matrix set a target similarity in the similarity set a target similarity a target similarity a corresponding category index value.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the video character identification method according to any one of claims 1 to 5 when executing the program.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the video character identification method according to any one of claims 1 to 5 when executed by the processor.

Images