Data retrieval method and device

Abstract

The embodiment of the invention provides a data retrieval method which comprises the following steps: acquiring face image data and name information of N persons, extracting a first face feature in the face image data, and constructing a face database based on the first face feature and the name information; obtaining a target video, and slicing the target video to obtain video slices of the target video; performing face recognition and face feature extraction on the video slices to obtain second face features; determining a corresponding target face in the face database based on the second face feature, and binding a video slice corresponding to the second face feature with the target face to obtain a binding relationship; and obtaining face query data, and determining a target video slice corresponding to the face query data based on the binding relationship and the face database. According to the technical scheme of the embodiment of the invention, the returned video data can be more accurate, and the user experience is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular, to a data retrieval method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] Face retrieval refers to the process of inputting face-related data (such as a person's name) and finding the corresponding data of the face that is most similar to the input data in the face database.

[0003] However, when current face retrieval searches for videos corresponding to faces, the returned videos usually contain a lot of information that users do not need, causing users to have to search for the desired video clips in the returned videos. The retrieval accuracy is low and the user experience is poor.

[0004] It should be noted that the above content is not necessarily prior art, nor is it intended to limit the scope of patent protection of this application. Summary of the invention

[0005] The embodiments of the present application provide a data retrieval method, apparatus, computer device, computer-readable storage medium, and computer program product to solve or alleviate one or more of the technical problems raised above.

[0006] One aspect of an embodiment of the present application provides a data retrieval method, the method comprising: Obtaining facial image data and name information of N persons, extracting a first facial feature from the facial image data, and building a facial database based on the first facial feature and the name information, where N is a positive integer; Acquire a target video, and slice the target video to obtain video slices of the target video; Performing face recognition and face feature extraction on the video slice to obtain a second face feature; Determine a target face corresponding to the face database based on the second face feature, and bind the video slice corresponding to the second face feature to the target face to obtain a binding relationship; Acquire face query data, and determine a target video slice corresponding to the face query data based on the binding relationship and the face database.

[0007] Optionally, slicing the target video to obtain video slices of the target video includes: Performing shot boundary detection on the frame image of the target video to determine M shot boundaries included in the frame image, where M is a natural number; The target video is sliced ​​based on the M shot boundaries to obtain video slices of the target video.

[0008] Optionally, performing face recognition and face feature extraction on the video slice to obtain a second face feature includes: Extracting key frames of the video slice; Face recognition and face feature extraction are performed according to the key frame to obtain a second face feature.

[0009] Optionally, determining a corresponding target face in the face database based on the second face feature includes: Clustering the second facial features of the same video slice; Determine a representative facial feature of the second facial feature based on a clustering result; Based on the representative facial features, a corresponding target face in the face database is determined.

[0010] Optionally, determining a corresponding target face in the face database based on the representative face feature includes: Determining the similarity between the representative facial feature and the first facial feature in the face database; A target face corresponding to the representative facial feature is determined based on the similarity.

[0011] Optionally, extracting a first facial feature from the facial image data, and constructing a facial database based on the first facial feature and the name information includes: assigning a character identifier to each of said characters; Extracting a facial feature vector from the facial image data as the first facial feature; The face database is constructed based on the name information, the character identifier and the first face feature, wherein a mapping relationship is formed between the character identifier and the name information, and between the character identifier and the first face feature.

[0012] Optionally, the method further comprises: In response to the face update data, determining in the face database a first person identifier corresponding to the face update data; The data corresponding to the first person identifier is updated based on the face update data.

[0013] Optionally, binding the video slice corresponding to the second facial feature to the target face includes: Determine a second person identifier of the target face; The video slice corresponding to the second facial feature and the second facial feature are associated with the second person identifier.

[0014] Optionally, acquiring face query data and determining a target video slice corresponding to the face query data includes: In a case where the face query data is a target person's name, determining a third person identifier corresponding to the target person's name based on the mapping relationship; A target video slice corresponding to the target person name is determined based on the third person identifier and the binding relationship.

[0015] Optionally, acquiring face query data and determining a target video slice corresponding to the face query data includes: When the face query data is a target face image, extracting target face features of the target face image; Determine a corresponding fourth person identifier based on the target facial feature, the second facial feature and the mapping relationship; A target video slice corresponding to the target facial image is determined based on the fourth person identifier and the binding relationship.

[0016] Another aspect of an embodiment of the present application provides a data retrieval device, the device comprising: A construction module, configured to obtain facial image data and name information of N persons, extract a first facial feature from the facial image data, and construct a facial database based on the first facial feature and the name information, wherein N is a positive integer; An acquisition module, used to acquire a target video, slice the target video, and obtain video slices of the target video; An extraction module, used for performing face recognition and face feature extraction on the video slice to obtain a second face feature; A binding module, configured to determine a corresponding target face in the face database based on the second face feature, and bind the video slice corresponding to the second face feature to the target face to obtain a binding relationship; The retrieval module is used to obtain face query data and determine the target video slice corresponding to the face query data based on the binding relationship and the face database.

[0017] Another aspect of an embodiment of the present application provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein: the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described above.

[0018] Another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method described above is implemented.

[0019] Another aspect of an embodiment of the present application provides a computer program product, including a computer program, which implements the method described above when executed by a processor.

[0020] The above technical solution adopted in the embodiment of the present application may have the following advantages: By acquiring the facial image data and name information of a face, extracting the first facial feature in the facial image data, and building a facial database based on the first facial feature and name information; acquiring a target video, slicing the target video, and obtaining a video slice of the target video; performing facial recognition and facial feature extraction on the video slice, and mentioning a second facial feature; determining the corresponding target face in the facial database based on the second facial feature, binding the video slice corresponding to the second facial feature to the target face, and obtaining a binding relationship; acquiring facial query data, and determining the target video slice corresponding to the facial query data based on the binding relationship and the facial database. It can be seen that the embodiment of the present application can return video slices related to faces, so that the returned video data is more accurate and more in line with user needs, convenient for users to use, and improves user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings exemplarily illustrate the embodiments and constitute a part of the specification, and together with the text description of the specification, are used to explain the exemplary implementation of the embodiments. The embodiments shown are for illustrative purposes only and do not limit the scope of the claims. In all drawings, the same reference numerals refer to similar but not necessarily identical elements.

[0022] Figure 1 A flowchart of a data retrieval method according to Embodiment 1 of the present application is schematically shown; Figure 2 Schematically shows Figure 2 Flow chart of sub-steps of step S102; Figure 3 Schematically shows Figure 2 Flow chart of sub-steps of step S104; Figure 4 Schematically shows Figure 2 Flow chart of sub-steps of step S106; Figure 5 Schematically shows Figure 4 Flow chart of sub-steps of step S404; Figure 6 Schematically shows Figure 2 Flow chart of sub-steps of step S100; Figure 7 The newly added process of the data retrieval method according to the first embodiment of the present application is schematically shown; Figure 8 Schematically shows Figure 2 Another sub-step flow chart of step S106; Fig. 9 Schematically shows Figure 2 Flow chart of sub-steps of step S108; Fig.10 Schematically shows Figure 2 Another sub-step flow chart of step S108; Fig.11 The schematic diagram shows a principle diagram of a data retrieval method according to the first embodiment of the present application; Fig.12 A block diagram schematically shows a data retrieval device according to the second embodiment of the present application; and Fig.13 The hardware architecture diagram of the computer device according to the third embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0024] It should be noted that the descriptions involving "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0025] In the description of the present application, it should be understood that the numerical labels before the steps do not indicate the order in which the steps are executed, but are only used to facilitate the description of the present application and to distinguish each step, and therefore should not be understood as a limitation on the present application.

[0026] First, the following terms are explained: Face recognition: The process of automatically identifying or verifying faces in images or videos using computer vision techniques.

[0027] Feature extraction: The process of extracting representative information from raw data. In face recognition, it usually refers to converting facial images into digital feature vectors.

[0028] Facial Landmarks: Key locations on the face, such as the corners of the eyes, the tip of the nose, the corners of the mouth, etc.

[0029] Affine transformation: A geometric transformation that preserves the proportions of parallel lines and straight lines, used for face alignment.

[0030] Bounding Box: In the field of computer vision and image processing, it is a rectangular area that represents an area of ​​interest in an image.

[0031] Shot boundary detection: also known as shot segmentation or shot switching detection, refers to automatically locating shot switching points in a video sequence and segmenting the video into a series of continuous shot segments. In other words, it is to find the dividing line between different scenes in the video.

[0032] K-means clustering: an unsupervised learning algorithm for grouping data.

[0033] Vector similarity retrieval: The process of finding other vectors in vector space that are most similar to a given vector.

[0034] Cosine distance: An indicator to measure the similarity between two vectors, often used to compare the similarity of feature vectors.

[0035] Secondly, in order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present application, the relevant technologies are described below: Face retrieval has a wide range of applications in security monitoring, identity authentication, social media, and the entertainment industry.

[0036] However, the applicant has learned that the following defects still exist in the related face retrieval methods: (1) When searching for videos corresponding to faces, current face search often returns videos that contain a lot of information that users do not need, which results in users having to search for the desired video clips again in the returned results. This results in low search accuracy and poor user experience. (2) Related video face recognition technologies usually use a frame-by-frame processing method, which has low computational efficiency and is difficult to handle massive video data; (3) Current face retrieval systems often only support a single retrieval method (such as only supporting name retrieval), which lacks flexibility; (4) In related technologies, updating the face database usually requires reprocessing all video data, which is time-consuming and resource-intensive. (5) Traditional methods often encounter duplicate recognition problems when processing multiple similar faces appearing in a video, which affects retrieval efficiency and accuracy.

[0037] To this end, the embodiment of the present application provides a data retrieval technology solution. In this technical solution: (1) by slicing the video and returning the video slices after retrieval, the returned video data can be made more accurate and the user experience can be improved; (2) by using video slicing and key frame extraction technology, the processing efficiency of large-scale video data can be significantly improved; (3) dual-mode retrieval can be performed based on pictures and names to improve the flexibility and applicability of the face retrieval system; (4) when updating the data in the face database, only the affected part of the data is updated to achieve incremental update and improve update efficiency; (5) by clustering and deduplicating faces, multiple similar faces in the video can be accurately processed to improve the accuracy of retrieval. See below for details.

[0038] The technical solutions of the present application are described below through multiple embodiments. It should be noted that these embodiments can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described here.

[0039] Embodiment 1 Figure 1 The flowchart of the data retrieval method according to the first embodiment of the present application is schematically shown.

[0040] like Figure 1 As shown, the data retrieval method may include steps S100 to S108, wherein: Step S100, obtaining facial image data and name information of N persons, extracting first facial features from the facial image data, and constructing a face database based on the first facial features and the name information, where N is a positive integer.

[0041] Step S102: acquiring a target video, and slicing the target video to obtain video slices of the target video.

[0042] Step S104, performing face recognition and face feature extraction on the video slice to obtain a second face feature.

[0043] Step S106, determining a corresponding target face in the face database based on the second facial feature, and binding the video slice corresponding to the second facial feature to the target face to obtain a binding relationship.

[0044] Step S108, obtaining face query data, and determining a target video slice corresponding to the face query data based on the binding relationship and the face database.

[0045] The data retrieval method provided in this embodiment obtains the facial image data and name information of a face, extracts the first facial feature in the facial image data, and builds a facial database based on the first facial feature and name information; obtains a target video, slices the target video, and obtains a video slice of the target video; performs facial recognition and facial feature extraction on the video slice, and obtains a second facial feature; determines the corresponding target face in the facial database based on the second facial feature, binds the video slice corresponding to the second facial feature to the target face, and obtains a binding relationship; obtains facial query data, and determines the target video slice corresponding to the facial query data based on the binding relationship and the facial database. It can be seen that the embodiment of the present application can return video slices related to faces, so that the returned video data is more accurate and more in line with user needs, which is convenient for users to use and improves user experience.

[0046] The following combination Figure 1 , each step in steps S100~S108 and other optional steps are explained in detail.

[0047] Step S100 , obtain facial image data and name information of N persons, extract the first facial feature in the facial image data, and build a face database based on the first facial feature and the name information, where N is a positive integer.

[0048] It should be noted that the facial image data obtained is obtained with the user's authorization and complies with copyright and other legal requirements.

[0049] Specifically, a large number of facial image data and name information of people can be collected; then, face recognition and facial feature extraction are performed on the collected facial image data to obtain a first facial feature corresponding to the facial image data; a mapping relationship is formed between the first facial feature and the corresponding name to construct a face database. Optionally, the name can include a real name, alias, pen name, former name, stage name, etc. In addition, the face database can also include a mapping relationship between the name and the corresponding facial image data.

[0050] When performing face recognition and face feature extraction on the collected face image data, a pre-trained machine learning model can be used for face recognition and face feature extraction. Specifically, a face detection and key point location model can be used to extract the face bounding box and facial key points, and then the identified facial key points are used to perform affine transformation according to predefined standard points for face alignment; a deep learning model is used to convert the aligned face images into low-dimensional feature vectors so that each identified face corresponds to a feature vector. For face image data containing multiple face images, feature vectors corresponding to the number of faces can be generated.

[0051] Step S102 , obtain a target video, slice the target video, and obtain video slices of the target video.

[0052] The target video refers to the video that needs to be sliced, and the number can be multiple. Specifically, a large number of target videos can be collected, and each target video can be sliced ​​to obtain a video slice of each target video. When slicing the target video, the video can be cut into a video slice based on a fixed time interval (such as 5 seconds); or the target in the target video can be tracked and sliced ​​when the target changes significantly. Some exemplary schemes are provided below.

[0053] In an optional embodiment, if Figure 2 As shown, step S102 may include: Step S200: performing shot boundary detection on the frame image of the target video to determine M shot boundaries included in the frame image, where M is a natural number.

[0054] Step S202: Slice the target video based on the M shot boundaries to obtain video slices of the target video.

[0055] When performing shot boundary detection on the frame images of the target video, the shot boundary can be determined by judging whether a shot switch occurs between adjacent frame images. The determination of shot switching can be performed by calculating the pixel difference between adjacent frame images of the target video, and determining that a shot switch occurs when the pixel difference between adjacent frame images exceeds a preset threshold; or, the histograms between adjacent frame images of the target video can be compared, and when the histogram difference is large, the shot switch occurs; or, the motion information between adjacent frame images can be analyzed, and when the motion information changes significantly, the shot switch occurs; or, a neural network model (such as a deep learning model) can be used to extract deep features of the frame images of the target video, and the extracted features can be used to determine whether a shot switch occurs.

[0056] In this embodiment, by performing shot boundary detection on the frame image of the target video, M shot boundaries included in the frame image are determined, and the target video is sliced ​​based on the M shot boundaries to obtain video slices of the target video. This can effectively slice the video and generate video slices with independent semantics, which facilitates the subsequent return of video slices with finer granularity and more accuracy, thereby improving the accuracy of retrieval.

[0057] Step S104 , perform face recognition and face feature extraction on the video slice to obtain a second face feature.

[0058] Specifically, the same or similar scheme as the above-mentioned face recognition and face feature extraction for face image data can be adopted to perform face recognition and face feature extraction on the frame image in the video slice to obtain the second face feature. Since the frame image of the video usually includes a lot of redundant information, in order to improve the processing efficiency and the accuracy of face recognition and face feature extraction, face recognition and face feature extraction can be performed on the more representative frame image in the video slice. Some exemplary schemes are provided below.

[0059] In an optional embodiment, if Figure 3 As shown, step S104 may include: Step S300: extracting key frames of the video slice.

[0060] Step S302: performing face recognition and face feature extraction according to the key frame to obtain a second face feature.

[0061] Specifically, for each video slice, the key frames of the video slice can be extracted at fixed time intervals. After obtaining the key frames of the video slice, the same or similar scheme as the above-mentioned face recognition and face feature extraction for facial image data is adopted to perform face recognition and face feature extraction on the key frames to obtain the second facial feature.

[0062] In this embodiment, by extracting key frames of video slices, face recognition and face feature extraction are performed based on the key frames to obtain a second face feature corresponding to the video slice. Based on finer-grained video slices, face recognition and feature extraction can be further performed through representative key frames, thereby significantly improving the processing efficiency of video data and effectively coping with the processing of large-scale video data.

[0063] Step S106 , based on the second facial feature, determine the corresponding target face in the face database, bind the video slice corresponding to the second facial feature to the target face, and obtain a binding relationship.

[0064] Specifically, the similarity between the second facial feature and the first facial feature in the face database can be calculated, and when the similarity reaches a threshold, the face corresponding to the first facial feature that reaches the threshold is determined to be the target face. For example, if the similarity between the second facial feature and the first facial feature A reaches a threshold, and the face corresponding to the first facial feature A is face a, then face a is determined to be the target face. After determining the target face, the video slice corresponding to the second facial feature can be bound to the target face to obtain a binding relationship. For example, the video slice corresponding to the second facial feature can be bound to the name corresponding to the target face to obtain a binding relationship.

[0065] In actual applications, there may be multiple similar faces in the same video slice, which may lead to the problem of repeated face recognition, affecting the efficiency and accuracy of retrieval. In order to improve the efficiency and accuracy of retrieval, face deduplication processing can be performed. The following are some exemplary solutions: In an optional embodiment, if Figure 4 As shown, step S106 may include: Step S400: clustering the second facial features of the same video slice.

[0066] Step S402: Determine representative facial features of the second facial features based on the clustering result.

[0067] Step S404: determining a corresponding target face in the face database based on the representative face features.

[0068] Specifically, a preset clustering algorithm can be used to cluster the second facial features of the same video slice, and the second facial features closest to the cluster center are selected as representative facial features according to the clustering results; then, the similarity between the representative facial features and the first facial features in the face database is calculated, and the target face is determined according to the similarity. The clustering algorithm can be a partition clustering algorithm or a hierarchical clustering algorithm, and preferably, a K-means clustering algorithm can be used for clustering.

[0069] In this embodiment, by clustering the second facial features of the same video slice, determining the representative facial features of the second facial features based on the clustering results, and determining the corresponding target faces in the face database based on the representative facial features, it is possible to effectively deduplicate faces in the same video slice, thereby improving the efficiency and accuracy of retrieval.

[0070] In an optional embodiment, if Figure 5 As shown, step S404 may include: Step S500, determining the similarity between the representative facial feature and the first facial feature in the face database.

[0071] Step S502: determining a target face corresponding to the representative facial feature based on the similarity.

[0072] It is understandable that one or more representative facial features can be determined based on the clustering results. Specifically, for each representative facial feature, the similarity between each representative facial feature and the first facial feature in the face database can be calculated, and when the similarity reaches a threshold, the face corresponding to the first facial feature corresponding to the threshold is determined to be the target face corresponding to the representative facial feature.

[0073] In this embodiment, by determining the similarity between the representative facial feature and the first facial feature in the face database, and determining the target face corresponding to the representative facial feature according to the similarity, the target face corresponding to the face in the video slice can be conveniently determined.

[0074] Step S108 , obtain face query data, and determine a target video slice corresponding to the face query data based on the binding relationship and the face database.

[0075] The face query data can be a name or a face image. In the case where the face query data is a name, the input name can be matched with the name in the face database, the corresponding face can be determined based on the matching result, and then the corresponding video slice can be determined as the target video slice based on the face and the binding relationship. In the case where the face query data is a face image, face recognition and feature extraction can be performed on the face image first, and then the similarity between the extracted face feature and the first face feature in the face database can be calculated, and the corresponding face can be determined based on the similarity, and then the corresponding video slice can be determined as the target video slice based on the face and the binding relationship.

[0076] In order to improve the efficiency of retrieval, more information can be added when constructing the face database, so as to facilitate retrieval. Some exemplary solutions are provided below.

[0077] In an optional embodiment, if Figure 6 As shown, step S100 may include: Step S600: assign a character identifier to each character.

[0078] Step S602: extracting a facial feature vector from the facial image data as the first facial feature.

[0079] Step S604: construct the face database based on the name information, the character identifier and the first face feature, wherein a mapping relationship is formed between the character identifier and the name information, and between the character identifier and the first face feature.

[0080] Specifically, a unique person ID can be generated for each person in the face database as a person identifier, wherein the person ID can be generated by a self-incrementing serial number, UUID or hash method. After extracting the first face feature of the face image data, the person identifier can be mapped to the name information, and the person identifier can be mapped to the first face feature. In this way, the person identifier, the name information and the first face feature can be mapped to each other, which is convenient for searching or converting the corresponding data. Among them, when the name information includes multiple names (such as full names, aliases and stage names), the person identifier and the name form a one-to-many mapping relationship. Due to different reasons such as shooting angles, expressions and lighting conditions, there may be multiple first face features. In this case, the person identifier and the first face feature can also form a one-to-many mapping relationship.

[0081] It is understandable that since there may be duplication of names, by generating a character identifier for each person, each person can be distinguished by a unique identifier, which facilitates subsequent retrieval.

[0082] In this embodiment, by assigning a character identifier to each character, constructing a mapping relationship between the character identifier and the name information in the face database, and constructing a mapping relationship between the character identifier and the first face feature, subsequent retrieval can be facilitated through the character identifier, thereby improving the efficiency of the retrieval and supporting dual-mode retrieval of names and images.

[0083] When the face database is updated, the affected part can be updated according to the person identifier to reduce the amount of calculation and achieve incremental update. Some exemplary solutions are provided below.

[0084] In an optional embodiment, if Figure 7 As shown, the data retrieval method of the embodiment of the present application may also include: Step S700: In response to the face update data, determine in the face database a first person identifier corresponding to the face update data.

[0085] Step S702: Update the data corresponding to the first person identifier based on the face update data.

[0086] The face update data may include face-related data such as name, face image data, and video slices.

[0087] The face update data can be the addition or deletion of the data corresponding to the original face in the face database, or the addition of the data corresponding to the face that does not exist in the face database. When the face update data is the addition or deletion of the data corresponding to the original face in the face database, it can be determined that the first person identifier corresponding to the face update data, and then the data corresponding to the first person identifier is added or deleted. When the face update data is the addition of data corresponding to the face that does not exist in the face database, the person identifier corresponding to the face update data can be added to the face database as the first person identifier, and then the face update data can be updated to the data corresponding to the first person identifier. In addition, when updating, asynchronous update can be used for updating.

[0088] In this embodiment, by responding to face update data, a first person identifier corresponding to the face update data is determined in the face database, and the data corresponding to the first person identifier is updated based on the face update data. The part that needs to be updated can be updated according to the person identifier, and there is no need to calculate the full amount of data, thereby reducing the update calculation amount and achieving incremental update.

[0089] In order to better realize the retrieval of video slices, the video slices can be associated with the person identifiers in the face database. Some exemplary schemes are provided below.

[0090] In an optional embodiment, if Figure 8 As shown, step S106 may include: Step S800, determining a second person identifier of the target face.

[0091] Step S802: Associating the video slice corresponding to the second facial feature and the second facial feature with the second person identifier.

[0092] When the face database includes a mapping relationship between the second facial feature and the person identifier, the second person identifier corresponding to the target face can be determined, and then the video slice corresponding to the second facial feature, the second facial feature and the second person identifier can be associated to form a correspondence between the video slice, the second facial feature and the second person identifier, which is convenient for subsequent search and return of the corresponding video slice based on the person identifier.

[0093] In this embodiment, by determining the second person identifier of the target face, and associating the video slice corresponding to the second facial feature and the second facial feature with the second person identifier, the video slice can be easily retrieved.

[0094] In an optional embodiment, if Fig. 9 As shown, step S108 may include: Step S900, when the face query data is a target name, determine a third person identifier corresponding to the target name based on the mapping relationship.

[0095] Step S902: determining a target video slice corresponding to the person's name based on the third person identifier and the binding relationship.

[0096] Specifically, when the face query data is a target person's name, the person identifier corresponding to the target name can be searched as the third person identifier based on the mapping relationship between the name and the person identifier; then, the video slice corresponding to the third person identifier is searched based on the binding relationship between the video slice and the person identifier, and the search result is returned to the user as the target video slice.

[0097] In this embodiment, when the face query data is a target name, a corresponding third person identifier is determined based on the mapping relationship between the name and the person identifier, and a target video slice corresponding to the target name is determined based on the third person identifier and the binding relationship. Therefore, retrieval of video slices can be achieved when the face query data is a name.

[0098] In an optional embodiment, if Fig.10 As shown, step S108 may include: Step S1000: when the face query data is a target face image, extract the target face features of the target face image.

[0099] Step S1002: determining a corresponding fourth person identifier based on the target facial feature, the second facial feature and the mapping relationship.

[0100] Step S1004: determining a target video slice corresponding to the target facial image based on the fourth person identifier and the binding relationship.

[0101] Specifically, when the face query data is a target face image, the target face features of the target face image can be extracted first; then, the similarity between the target face features and the second face features is calculated, and the closest second face features are determined based on the similarity; based on the closest second face features and the mapping relationship between the second face features and the character identifier, the corresponding character identifier is determined as the fourth character identifier, and then based on the binding relationship between the character identifier and the video slice, the corresponding video slice is determined as the target video slice and returned to the user.

[0102] In this embodiment, by extracting the target facial features of the target facial image when the face query data is a target facial image, determining the corresponding fourth person identifier based on the target facial features, the second facial features and the mapping relationship, and determining the target video slice based on the fourth person identifier and the binding relationship, the retrieval of the video slice can be achieved when the face query data is a face image.

[0103] In order to make this application easier to understand, the following Fig.11 Provide an example application. Fig.11 As shown in the figure, the retrieval of video slices corresponding to faces can be achieved through four core modules: face recognition and feature extraction module, celebrity library (i.e. face database) construction module, video processing and indexing module, and face retrieval implementation module: 1. Face recognition and feature extraction module: 1. Perform face detection and key point location on the input image, and then use the identified key points to perform affine transformation according to predefined standard points to achieve face alignment; 2. Use the pre-trained deep learning model to convert the aligned faces into low-dimensional feature vectors, with each recognized face corresponding to a feature vector; 3. If the input image contains multiple faces, a corresponding number of feature vectors are generated.

[0104] 2. Celebrity database construction module: 1. Collect information such as the person's name, alias, and photo through the Internet or internal data sources. At least three clear frontal photos of each person can be collected; 2. Assign a unique identifier (personID) to each person; 3. Establish a mapping relationship between person name (full name, alias, etc.) and personID, and establish a mapping relationship between personID and feature vector; 4. Perform face recognition and feature extraction on each person’s photo, and associate the extracted feature vector with the corresponding personID and store it.

[0105] 3. Video processing and indexing module: 1. Use shot boundary retrieval technology to segment the video into independent semantic slices; 2. Extract key frames for each video slice at fixed time intervals; 3. Perform face recognition and feature extraction on each key frame to generate a feature vector list; 4. Use the clustering algorithm to cluster the facial feature vectors in the same slice, select the feature vector closest to the cluster center as the representative of the corresponding cluster, and extract the most representative facial feature vector; 5. For each facial feature vector after deduplication, perform vector similarity search in the celebrity database; when the highest similarity of the retrieved feature vector exceeds the preset threshold, associate the corresponding personID with the video slice; 6. Save the facial feature vector extracted from each slice; when the celebrity database is updated, asynchronously recalculate and update the personID of the affected video slice.

[0106] 4. Face retrieval implementation module: 1. Retrieval based on name: You can search for the personID corresponding to the input name in the celebrity database, use the obtained personID to retrieve the associated video slices in the video slice database, and return a list of video slices containing the target person; 2. Image-based retrieval: extract facial features from the input image, perform vector similarity search in the celebrity database, obtain the most matching personID, use the obtained personID to retrieve the associated video slices in the video slice library, and return a list of video slices containing the target person.

[0107] In this exemplary application, through the four core modules of face recognition and feature extraction module, celebrity library construction module, video processing and indexing module and face retrieval implementation module, video slices can be retrieved and returned, making the returned video data more accurate and improving user experience; and the processing efficiency of video data can be significantly improved through video slicing and key frame extraction technology; at the same time, dual-mode retrieval based on pictures and names can be performed to improve the flexibility and applicability of the face retrieval system; in addition, incremental updates can be achieved to improve update efficiency; and by clustering and deduplicating faces, multiple similar faces can be effectively processed to improve retrieval accuracy.

[0108] Embodiment 2 Fig.12 The block diagram of the data retrieval device according to the second embodiment of the present application is schematically shown. The device can be divided into one or more program modules, one or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiment of the present application. The program module referred to in the embodiment of the present application refers to a series of computer program instruction segments that can complete specific functions. The following description will specifically introduce the functions of each program module in this embodiment. Fig.12 As shown, the apparatus 1100 may include: a construction module 1110, an acquisition module 1120, an extraction module 1130, a binding module 1140 and a retrieval module 1150, wherein: A construction module 1110 is used to obtain facial image data and name information of N persons, extract a first facial feature from the facial image data, and construct a face database based on the first facial feature and the name information, where N is a positive integer; The acquisition module 1120 is used to acquire a target video, slice the target video, and obtain video slices of the target video; An extraction module 1130 is used to perform face recognition and face feature extraction on the video slice to obtain a second face feature; A binding module 1140 is used to determine a corresponding target face in the face database based on the second face feature, and bind the video slice corresponding to the second face feature to the target face to obtain a binding relationship; The retrieval module 1150 is used to obtain face query data and determine a target video slice corresponding to the face query data based on the binding relationship and the face database.

[0109] As an optional embodiment, slicing the target video to obtain video slices of the target video includes: Performing shot boundary detection on the frame image of the target video to determine M shot boundaries included in the frame image, where M is a natural number; The target video is sliced ​​based on the M shot boundaries to obtain video slices of the target video.

[0110] As an optional embodiment, performing face recognition and face feature extraction on the video slice to obtain a second face feature includes: Extracting key frames of the video slice; Face recognition and face feature extraction are performed according to the key frame to obtain a second face feature.

[0111] As an optional embodiment, determining a corresponding target face in the face database based on the second face feature includes: Clustering the second facial features of the same video slice; Determine a representative facial feature of the second facial feature based on a clustering result; Based on the representative facial features, a corresponding target face in the face database is determined.

[0112] As an optional embodiment, determining a corresponding target face in the face database based on the representative face feature includes: Determining the similarity between the representative facial feature and the first facial feature in the face database; A target face corresponding to the representative facial feature is determined based on the similarity.

[0113] As an optional embodiment, extracting a first facial feature from the facial image data, and constructing a face database based on the first facial feature and the name information includes: assigning a character identifier to each of said characters; Extracting a facial feature vector from the facial image data as the first facial feature; The face database is constructed based on the name information, the character identifier and the first face feature, wherein a mapping relationship is formed between the character identifier and the name information, and between the character identifier and the first face feature.

[0114] As an optional embodiment, the device 1100 is further used for: In response to the face update data, determining in the face database a first person identifier corresponding to the face update data; The data corresponding to the first person identifier is updated based on the face update data.

[0115] As an optional embodiment, binding the video slice corresponding to the second facial feature to the target face includes: Determine a second person identifier of the target face; The video slice corresponding to the second facial feature and the second facial feature are associated with the second person identifier.

[0116] As an optional embodiment, obtaining face query data and determining a target video slice corresponding to the face query data includes: In a case where the face query data is a target person's name, determining a third person identifier corresponding to the target person's name based on the mapping relationship; A target video slice corresponding to the target person name is determined based on the third person identifier and the binding relationship.

[0117] As an optional embodiment, obtaining face query data and determining a target video slice corresponding to the face query data includes: When the face query data is a target face image, extracting target face features of the target face image; Determine a corresponding fourth person identifier based on the target facial feature, the second facial feature and the mapping relationship; A target video slice corresponding to the target facial image is determined based on the fourth person identifier and the binding relationship.

[0118] Embodiment 3 Fig.13The schematic diagram of the hardware architecture of a computer device 10000 suitable for implementing a data retrieval method according to the third embodiment of the present application is shown schematically. In some embodiments, the computer device 10000 may be a terminal device such as a smart phone, a wearable device, a tablet computer, a personal computer, a vehicle-mounted terminal, a game console, a virtual device, a workbench, a digital assistant, a set-top box, a robot, etc. In other embodiments, the computer device 10000 may be a rack server, a blade server, a tower server, or a cabinet server (including an independent server, or a server cluster composed of multiple servers), etc. Fig.13 As shown, the computer device 10000 includes but is not limited to: a memory 10010, a processor 10020, and a network interface 10030 that can communicate with each other through a system bus. Among them: The memory 10010 includes at least one type of computer-readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 10010 can be an internal storage module of the computer device 10000, such as a hard disk or memory of the computer device 10000. In other embodiments, the memory 10010 can also be an external storage device of the computer device 10000, such as a plug-in hard disk equipped on the computer device 10000, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10010 can also include both the internal storage module of the computer device 10000 and its external storage device. In this embodiment, the memory 10010 is generally used to store the operating system and various application software installed in the computer device 10000, such as program codes of the data retrieval method, etc. In addition, the memory 10010 can also be used to temporarily store various data that have been output or are to be output.

[0119] In some embodiments, the processor 10020 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other chips. The processor 10020 is generally used to control the overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is used to run the program code stored in the memory 10010 or process data.

[0120] The network interface 10030 may include a wireless network interface or a wired network interface, and the network interface 10030 is generally used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, and to establish a data transmission channel and a communication link between the computer device 10000 and the external terminal. The network may be a wireless or wired network such as an intranet, the Internet, the Global System of Mobile communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, etc.

[0121] It should be pointed out that Fig.13 Only a computer device having components 10010 - 10030 is shown, but it should be understood that implementation of all of the components shown is not a requirement, and more or fewer components may alternatively be implemented.

[0122] In this embodiment, the data retrieval method stored in the memory 10010 may also be divided into one or more program modules and executed by one or more processors (such as processor 10020) to complete the embodiment of the present application.

[0123] Embodiment 4 An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the data retrieval method in the embodiment when executed by a processor.

[0124] In this embodiment, the computer-readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the computer-readable storage medium can be an internal storage unit of a computer device, such as a hard disk or memory of the computer device. In other embodiments, the computer-readable storage medium can also be an external storage device of a computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the computer-readable storage medium can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the computer-readable storage medium is generally used to store an operating system and various application software installed on the computer device, such as the program code of the data retrieval method in the embodiment, etc. In addition, the computer-readable storage medium can also be used to temporarily store various types of data that have been output or are to be output.

[0125] Embodiment 5 An embodiment of the present application also provides a computer program product, including a computer program, which implements the method in the above embodiment when executed by a processor.

[0126] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present application can be implemented by general-purpose computer devices, they can be concentrated on a single computer device, or distributed on a network composed of multiple computer devices, optionally, they can be implemented by executable program codes of computer devices, so that they can be stored in a storage device and executed by the computer device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0127] It should be noted that the above are only preferred embodiments of the present application, and the patent protection scope of the present application is not limited thereto. Any equivalent structure or equivalent process transformation made using the contents of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data retrieval method, characterized in that: The method comprises: Obtaining facial image data and name information of N persons, extracting a first facial feature from the facial image data, and building a facial database based on the first facial feature and the name information, where N is a positive integer; Acquire a target video, and slice the target video to obtain video slices of the target video; Performing face recognition and face feature extraction on the video slice to obtain a second face feature; Determine a target face corresponding to the face database based on the second face feature, and bind the video slice corresponding to the second face feature to the target face to obtain a binding relationship; Acquire face query data, and determine a target video slice corresponding to the face query data based on the binding relationship and the face database.

2. The method according to claim 1, characterized in that Slicing the target video to obtain video slices of the target video includes: Performing shot boundary detection on the frame image of the target video to determine M shot boundaries included in the frame image, where M is a natural number; The target video is sliced ​​based on the M shot boundaries to obtain video slices of the target video.

3. The method according to claim 1, characterized in that Performing face recognition and face feature extraction on the video slice to obtain a second face feature includes: Extracting key frames of the video slice; Face recognition and face feature extraction are performed according to the key frame to obtain a second face feature.

4. The method according to claim 3, characterized in that Determining a target face corresponding to the face database based on the second face feature includes: Clustering the second facial features of the same video slice; Determine a representative facial feature of the second facial feature based on a clustering result; Based on the representative facial features, a corresponding target face in the face database is determined.

5. The method according to claim 4, characterized in that Determining a target face corresponding to the face database based on the representative face feature includes: Determining the similarity between the representative facial feature and the first facial feature in the face database; A target face corresponding to the representative facial feature is determined based on the similarity.

6. The method according to any one of claims 1 to 5, characterized in that: Extracting a first facial feature from the facial image data, and constructing a facial database based on the first facial feature and the name information, including: assigning a character identifier to each of said characters; Extracting a facial feature vector from the facial image data as the first facial feature; The face database is constructed based on the name information, the character identifier and the first face feature, wherein a mapping relationship is formed between the character identifier and the name information, and between the character identifier and the first face feature.

7. The method according to claim 6, characterized in that The method further comprises: In response to the face update data, determining in the face database a first person identifier corresponding to the face update data; The data corresponding to the first person identifier is updated based on the face update data.

8. The method according to claim 6, characterized in that Binding the video slice corresponding to the second facial feature to the target face includes: Determine a second person identifier of the target face; The video slice corresponding to the second facial feature and the second facial feature are associated with the second person identifier.

9. The method according to claim 8, characterized in that Acquiring face query data and determining a target video slice corresponding to the face query data includes: In a case where the face query data is a target person's name, determining a third person identifier corresponding to the target person's name based on the mapping relationship; A target video slice corresponding to the target person name is determined based on the third person identifier and the binding relationship.

10. The method according to claim 8, characterized in that Acquiring face query data and determining a target video slice corresponding to the face query data includes: When the face query data is a target face image, extracting target face features of the target face image; Determine a corresponding fourth person identifier based on the target facial feature, the second facial feature and the mapping relationship; A target video slice corresponding to the target facial image is determined based on the fourth person identifier and the binding relationship.

11. A data retrieval device, characterized in that: The device comprises: A construction module, configured to obtain facial image data and name information of N persons, extract a first facial feature from the facial image data, and construct a facial database based on the first facial feature and the name information, wherein N is a positive integer; An acquisition module, used to acquire a target video, slice the target video, and obtain video slices of the target video; An extraction module, used for performing face recognition and face feature extraction on the video slice to obtain a second face feature; A binding module, configured to determine a corresponding target face in the face database based on the second face feature, and bind the video slice corresponding to the second face feature to the target face to obtain a binding relationship; The retrieval module is used to obtain face query data and determine the target video slice corresponding to the face query data based on the binding relationship and the face database.

12. A computer device, characterized in that: include: at least one processor; and a memory communicatively coupled to the at least one processor; wherein: The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images