Face searching method based on multi-modal technology

Abstract

The invention relates to the technical field of face search, and particularly discloses a multi-modal technology-based face search method, which comprises the following steps of: performing basic attribute marking on a face image in a face database by adopting a deep learning-based data processing technology to generate an attribute description text of the face image; semantic analysis and matching analysis are performed on the target face search text description input by the user and the generated face image attribute description text, so that face image search based on text description is realized. In this way, face search can be supported on the basis of semantic information of text description, so that face screening work can be quickly carried out, and the method has positive significance for a non-graph scene.

Description

Technical Field

[0001] The present application relates to the field of face search technology, and more specifically, to a face search method based on multimodal technology. Background Art

[0002] As an important biometric identification method, face search technology has shown great potential and value in many fields such as public security, identity authentication, social media, etc. At present, mainstream face search solutions mainly rely on direct image input, that is, uploading a photo containing the target person's face to start the search process, using image processing technology to extract facial features from the photo, and comparing it with the pre-stored face database to return the closest result.

[0003] However, this face search method has certain limitations in practical applications. For example, in some specific scenarios (such as public security), it is often difficult to obtain high-quality photos of the target person, and only verbal descriptions or text records can be obtained, which makes the traditional image-based face search technology ineffective and limits the accuracy and efficiency of the search.

[0004] Therefore, a face search method based on multimodal technology is expected. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a face search method based on multimodal technology, which uses data processing technology based on deep learning to label basic attributes of face images in a face database, generate attribute description text of the face image, and then, through semantic parsing and matching analysis of the target face search text description input by the user and the generated face image attribute description text, thereby realizing face image search based on text description. In this way, face search based on semantic information of text description can be supported, so that face screening work can be carried out quickly, which has positive significance for scenes without pictures.

[0006] Accordingly, according to one aspect of the present application, a face search method based on multimodal technology is provided, which includes:

[0007] Obtain a target face search text description and extract a first candidate face image from a face database;

[0008] Performing local region fine-grained semantic feature extraction on the first candidate face image to obtain a set of semantic encoding feature vectors of the local region of the face;

[0009] Performing facial attribute description on the set of semantically encoded feature vectors of the local face region to obtain a first candidate facial image attribute description text;

[0010] Based on the semantic match between the target face search text description and the first candidate face image attribute description text, it is determined whether to use the first candidate face image as a search result.

[0011] In the above-mentioned face search method based on multimodal technology, fine-grained semantic features of local areas of the first candidate face image are extracted to obtain a set of semantic coding feature vectors of local areas of the face, including: inputting the first candidate face image into a face feature encoder based on the ViT model to obtain the set of semantic coding feature vectors of local areas of the face.

[0012] In the above-mentioned face search method based on multimodal technology, the set of semantically encoded feature vectors of the local area of ​​the face is used to describe the face attributes to obtain a first candidate face image attribute description text, including: inputting the set of semantically encoded feature vectors of the local area of ​​the face into a face attribute descriptor based on the LLM model to obtain the first candidate face image attribute description text.

[0013] In the above-mentioned face search method based on multimodal technology, the set of semantic coding feature vectors of local face areas is subjected to face attribute description to obtain a first candidate face image attribute description text, including: performing semantic enhancement based on feature field effect distribution on the set of semantic coding feature vectors of local face areas to obtain a set of enhanced semantic coding feature vectors of local face areas; inputting the set of enhanced semantic coding feature vectors of local face areas into a face attribute descriptor based on the LLM model to obtain the first candidate face image attribute description text.

[0014] In the above-mentioned face search method based on multimodal technology, the set of semantic coding feature vectors of local face areas is semantically enhanced based on feature field effect distribution to obtain a set of enhanced semantic coding feature vectors of local face areas, including: using a semantic reconstructor to feature reconstruct each semantic coding feature vector of local face areas in the set of semantic coding feature vectors of local face areas to obtain a set of reconstructed semantic coding feature vectors of local face areas; calculating the semantic depth implicit representation value of each reconstructed semantic coding feature vector of local face areas in the set of reconstructed semantic coding feature vectors of local face areas to obtain a set of semantic depth implicit representation values ​​of local face areas; based on the sorting of the set of semantic depth implicit representation values ​​of local face areas, the set of semantic coding feature vectors of local face areas is reordered to obtain a set of updated semantic coding feature vectors of local face areas; and the set of updated semantic coding feature vectors of local face areas is bidirectionally contextually enhanced to obtain the set of enhanced semantic coding feature vectors of local face areas.

[0015] In the above-mentioned face search method based on multimodal technology, a semantic reconstructor is used to perform feature reconstruction on each local face area semantic coding feature vector in the set of local face area semantic coding feature vectors to obtain a set of reconstructed local face area semantic coding feature vectors, including: using a first weight parameter matrix and a second weight parameter matrix to perform continuous linear transformation on the local face area semantic coding feature vectors to obtain the reconstructed local face area semantic coding feature vectors.

[0016] In the above-mentioned face search method based on multimodal technology, the semantic depth implicit representation value of each reconstructed face local area semantic coding feature vector in the set of reconstructed face local area semantic coding feature vectors is calculated to obtain a set of face local area semantic depth implicit representation values, including: calculating the square of the Euclidean norm of the reconstructed face local area semantic coding feature vector to obtain the initial face local area semantic depth implicit representation value; calculating the ratio of the sum of the initial face local area semantic depth implicit representation value and the difference between the initial face local area semantic depth implicit representation value to obtain the semantic depth representation modulation coefficient; calculating the logarithmic function value of the semantic depth representation modulation coefficient with base 2 to obtain the face local area semantic depth implicit representation value.

[0017] In the above-mentioned face search method based on multimodal technology, the set of updated face local area semantic coding feature vectors is subjected to bidirectional contextual semantic association enhancement to obtain the set of enhanced face local area semantic coding feature vectors, including: inputting the set of updated face local area semantic coding feature vectors into a first semantic encoder based on a forward LSTM model to obtain a set of forward face local area contextual semantic association coding feature vectors; inputting the set of updated face local area semantic coding feature vectors into a second semantic encoder based on a backward LSTM model to obtain a set of backward face local area contextual semantic association coding feature vectors; and fusing the set of forward face local area contextual semantic association coding feature vectors and the set of backward face local area contextual semantic association coding feature vectors to obtain the set of enhanced face local area semantic coding feature vectors.

[0018] In the above-mentioned face search method based on multimodal technology, whether to use the first candidate face image as the search result is determined based on the semantic match between the target face search text description and the first candidate face image attribute description text, including: semantically encoding the target face search text description and the first candidate face image attribute description text to obtain a target face search text description semantic encoding vector and a first candidate face image attribute description semantic encoding vector; calculating the cosine similarity between the target face search text description semantic encoding vector and the first candidate face image attribute description semantic encoding vector as the face search matching degree; in response to the face search matching degree being greater than or equal to a preset threshold, using the first candidate face image as the search result.

[0019] Compared with the prior art, the face search method based on multimodal technology provided by the present application uses data processing technology based on deep learning to label the basic attributes of face images in the face database, generate attribute description text of the face image, and then, through semantic parsing and matching analysis of the target face search text description input by the user and the generated face image attribute description text, realize the face image search based on text description. In this way, it is possible to support face search based on the semantic information of text description, so as to quickly carry out face screening work, which has positive significance for scenes without images. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 The present invention is a flowchart of a face search method based on multimodal technology according to an embodiment of the present application.

[0022] Figure 2 Schematic diagram of data flow of a face search method based on multimodal technology according to an embodiment of the present application.

[0023] Figure 3 This is a flowchart of step S131 in the face search method based on multimodal technology according to an embodiment of the present application.

[0024] Figure 4 Flow chart of step S140 in the face search method based on multimodal technology according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0026] In view of the problem that the current face search technology cannot enable the 0-sample search method and can only realize face search by searching images with images, this application proposes a face search method based on multimodal technology, which uses data processing technology based on deep learning to label the basic attributes of face images in the face database, generate attribute description text of the face image, and then perform semantic parsing and matching analysis on the target face search text description input by the user and the generated face image attribute description text, thereby realizing face image search based on text description. In this way, face search based on semantic information described in text can be supported, so that face screening work can be carried out quickly, which has positive significance for scenes without images.

[0027] Figure 1 The present invention is a flowchart of a face search method based on multimodal technology according to an embodiment of the present application. Figure 2 FIG. 1 is a data flow diagram of a face search method based on multimodal technology according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the face search method based on multimodal technology according to the embodiment of the present application, the steps include: S110, obtaining a target face search text description, and extracting a first candidate face image from a face database at the same time; S120, performing local area fine-grained semantic feature extraction on the first candidate face image to obtain a set of semantic coding feature vectors of local face areas; S130, performing face attribute description on the set of semantic coding feature vectors of local face areas to obtain a first candidate face image attribute description text; S140, determining whether to use the first candidate face image as a search result based on the semantic match between the target face search text description and the first candidate face image attribute description text.

[0028] In the above-mentioned face search method based on multimodal technology, the step S110 obtains the target face search text description and extracts the first candidate face image from the face database. It should be understood that the target face search text description is the user's query intention, and the first candidate face image is a potential matching object. The present application solves the face search problem in image-free scenarios by performing cross-modal query matching on the two. To simplify the description, the technical solution of the present application only elaborates on the data processing process of a single face image, but it should be understood that in actual applications, the method can be applied to each face image in the face database, thereby realizing a traversal query of the face database.

[0029] In the above-mentioned face search method based on multimodal technology, the step S120 performs fine-grained semantic feature extraction of the local area of ​​the first candidate face image to obtain a set of semantic encoding feature vectors of the local area of ​​the face. It should be understood that considering that the traditional convolutional neural network (CNN) is limited to the extraction of local features when processing face images, it fails to fully consider the global structure of the face image, which may lead to the loss of face features. Based on this, in order to improve the accuracy of feature extraction, in a specific example of the present application, the first candidate face image is input into a face feature encoder based on the ViT model to obtain the set of semantic encoding feature vectors of the local area of ​​the face. Those of ordinary skill in the art should know that the ViT model is based on the Transformer architecture and can effectively model the global context information in the image. Specifically, the granularity of feature extraction is first refined by segmenting the first candidate face image. Next, each image block is embedded and encoded and input into the Transformer architecture of the ViT model as a sequence. The self-attention mechanism is used to model the global dependencies of the image, so that the global context information of the image can be encoded into the feature representation of each image block, so as to better understand the overall structure and local details of the face image and generate a set of semantic encoding feature vectors of the local area of ​​the face.

[0030] In the above-mentioned face search method based on multimodal technology, the step S130 describes the face attributes of the set of semantically encoded feature vectors of the local face area to obtain the first candidate face image attribute description text. That is, in order to achieve cross-modal matching between the face image and the target face search text description, the present application further performs modal conversion on the face image features, and uses a large language model to describe the attributes of the face image features. In a specific example of the present application, the set of semantically encoded feature vectors of the local face area is input into the face attribute descriptor based on the LLM model to obtain the first candidate face image attribute description text. Those of ordinary skill in the art should know that the LLM model has powerful language understanding and generation capabilities. Given a prompt, it can generate text descriptions corresponding to face image features, including detailed descriptions of multiple basic attributes such as face age, gender, hairstyle, face shape, etc., to reflect the visual representation of face attributes.

[0031] In particular, in the technical solution of the present application, when the first candidate face image is input into a face feature encoder based on the ViT model, the first candidate face image is first segmented to obtain a set of first candidate face image blocks, and then after embedding and encoding each first candidate face image block in the set of the first candidate face image blocks, the VIT model is used to perform inter-face image block context semantic encoding on the set of embedding coding vectors of the first candidate face image blocks to obtain a set of semantic coding feature vectors of local face areas.

[0032] However, considering that the integrity and smoothness of the local facial features between local facial regions will be destroyed when the image blocks are segmented, at the same time, the facial feature encoder based on the ViT model is based on the Transformer architecture to perform global context semantic association encoding between local facial regions, but there are obvious differences in the degree of association between different local facial regions, that is, the facial feature encoder based on the ViT model has a weak performance in capturing local enhanced features of the face, thereby affecting the description accuracy of the first candidate facial image attribute description text obtained by inputting the facial attribute descriptor based on the LLM model.

[0033] Based on this, in a preferred example of the present application, before the set of semantically coded feature vectors of local face regions is input into the face attribute descriptor based on the LLM model, it is further semantically enhanced. Accordingly, the step S130 includes: S131, semantically enhancing the set of semantically coded feature vectors of local face regions based on feature field effect distribution to obtain a set of enhanced semantically coded feature vectors of local face regions; S132, inputting the set of enhanced semantically coded feature vectors of local face regions into the face attribute descriptor based on the LLM model to obtain the first candidate face image attribute description text.

[0034] Figure 3 FIG. 1 is a flowchart of step S131 in the face search method based on multimodal technology according to an embodiment of the present application. Figure 3As shown, the step S131 includes: S1311, using a semantic reconstructor to perform feature reconstruction on each of the face local area semantic coding feature vectors in the set of face local area semantic coding feature vectors to obtain a set of reconstructed face local area semantic coding feature vectors; S1312, calculating the semantic depth implicit representation value of each reconstructed face local area semantic coding feature vector in the set of reconstructed face local area semantic coding feature vectors to obtain a set of face local area semantic depth implicit representation values; S1313, based on the sorting of the set of face local area semantic depth implicit representation values, re-sorting the set of face local area semantic coding feature vectors to obtain a set of updated face local area semantic coding feature vectors; S1314, performing bidirectional contextual semantic association enhancement on the set of updated face local area semantic coding feature vectors to obtain the set of enhanced face local area semantic coding feature vectors.

[0035] In a specific example of the present application, the step S1311 includes: using a first weight parameter matrix and a second weight parameter matrix to perform a continuous linear transformation on the semantic coding feature vector of the local area of ​​the face to obtain the reconstructed semantic coding feature vector of the local area of ​​the face.

[0036] In a specific example of the present application, the step S1312 includes: calculating the square of the Euclidean norm of the reconstructed semantic coding feature vector of the local area of ​​the face to obtain an initial local area semantic depth implicit representation value of the face; calculating the ratio of the sum of the initial local area semantic depth implicit representation value of the face and the difference between the sum and the initial local area semantic depth implicit representation value of the face to obtain a semantic depth representation modulation coefficient; calculating the logarithmic function value of the semantic depth representation modulation coefficient with base 2 to obtain the implicit representation value of the semantic depth of the local area of ​​the face.

[0037] That is, firstly, a semantic reconstructor is used to perform domain transformation on each semantic coding feature vector of a local face region in the set of semantic coding feature vectors of the local face region, so as to unify the feature dimension and enhance its representation ability in a specific context, so as to generate a set of reconstructed semantic coding feature vectors of local face regions. Then, by calculating the semantic depth implicit representation value of each reconstructed semantic coding feature vector of the local face region, the information density or complexity carried by each local face region in the overall face image information is measured, and the importance and influence of each local face region in the overall information is quantified. Then, according to the semantic depth implicit representation value of each local face region, the feature order in the set is further adjusted, and the feature structure of the face image is reorganized to ensure that key information is given priority in the subsequent contextual semantic association enhancement encoding process.

[0038] In a specific example of the present application, the step S1314 includes: inputting the set of updated face local area semantic coding feature vectors into a first semantic encoder based on a forward LSTM model to obtain a set of forward face local area context semantic association coding feature vectors; inputting the set of updated face local area semantic coding feature vectors into a second semantic encoder based on a backward LSTM model to obtain a set of backward face local area context semantic association coding feature vectors; fusing the set of forward face local area context semantic association coding feature vectors and the set of backward face local area context semantic association coding feature vectors to obtain the set of enhanced face local area semantic coding feature vectors.

[0039] In a specific example of the present application, the set of the forward face local area context semantic association coding feature vectors and the set of the backward face local area context semantic association coding feature vectors are fused, including: using a first weighting parameter and a second weighting parameter to weighted modulate the set of the forward face local area context semantic association coding feature vectors and the set of the backward face local area context semantic association coding feature vectors respectively to obtain a set of weighted modulated forward face local area context semantic association coding feature vectors and a set of weighted modulated backward face local area context semantic association coding feature vectors; calculating the corresponding points between the set of weighted modulated forward face local area context semantic association coding feature vectors, the set of weighted modulated backward face local area context semantic association coding feature vectors and the bias item to obtain the set of enhanced face local area semantic coding feature vectors.

[0040] That is, on the basis of feature updating, a bidirectional long short-term memory (LSTM) model is used to encode the set of updated semantic encoding feature vectors of local facial regions from both the forward and backward directions. Among them, the forward LSTM captures the information flow pattern from front to back, while the backward LSTM operates in the opposite direction to identify the dependency from back to front. This dual-perspective encoding mechanism can capture the correlation dependency between local facial regions at different levels, and thereby strengthen the semantic feature expression of each local facial region. Finally, by integrating the encoding results of the bidirectional LSTM model to integrate the bidirectional contextual semantic information of each local facial region, a richer and more comprehensive set of enhanced semantic encoding feature vectors of local facial regions is formed, thereby achieving a deep understanding of facial features to improve the text generation effect of facial attribute description through the LLM model.

[0041] In summary, the step S131 includes: processing the set of semantic coding feature vectors of the local face region by the following feature enhancement formula to obtain a set of enhanced semantic coding feature vectors of the local face region, wherein the feature enhancement formula is:

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049] in, represents a set of semantically encoded feature vectors of the local face region, , , and Respectively represent the first, second, and third in the set of semantic coding feature vectors of the local face area and Semantic encoding feature vector of local area of ​​individual face, is the number of semantic coding feature vectors of the local face area, and denote the first weight parameter matrix and the second weight parameter matrix respectively, For the Reconstruct the semantic encoding feature vector of the local area of ​​the face, represents the calculation of the Euclidean norm of a vector, represents the logarithmic function with base 2, Indicates The semantic depth implicit representation value of the local area of ​​a person's face, Represents the set of updated semantic encoding feature vectors of local face regions, , , and respectively represent the first, second, and third in the set of updated face local area semantic coding feature vectors and Update the semantic encoding feature vector of the local face region, represents the forward LSTM model, represents the backward LSTM model, represents the set of forward face local area context semantic association encoding feature vectors, Represents the set of backward face local area context semantic association encoding feature vectors, and denote the first weighting parameter and the second weighting parameter respectively, is the bias term, The set of semantic encoding feature vectors for the enhanced local face area.

[0050] In the above-mentioned face search method based on multimodal technology, the step S140 determines whether to use the first candidate face image as a search result based on the semantic match between the target face search text description and the first candidate face image attribute description text. Figure 4 FIG. 1 is a flowchart of step S140 in the face search method based on multimodal technology according to an embodiment of the present application. Figure 4 As shown, the step S140 includes: S141, semantically encoding the target face search text description and the first candidate face image attribute description text to obtain a target face search text description semantic encoding vector and a first candidate face image attribute description semantic encoding vector; S142, calculating the cosine similarity between the target face search text description semantic encoding vector and the first candidate face image attribute description semantic encoding vector as a face search match; S143, in response to the face search match being greater than or equal to a preset threshold, taking the first candidate face image as the search result.

[0051] Specifically, the step S141 semantically encodes the target face search text description and the first candidate face image attribute description text to obtain a target face search text description semantic encoding vector and a first candidate face image attribute description semantic encoding vector. That is, in order to achieve semantic matching between the target face search text description and the first candidate face image attribute description text, the present application further uses a pre-trained language model to extract semantic features from the two, and encodes the two into comparable semantic vector forms. In a specific example of the present application, the Bert model is used to semantically encode the target face search text description and the first candidate face image attribute description text. It should be understood that the Bert model is based on a bidirectional Transformer architecture, which can simultaneously consider the contextual information on the left and right sides of each word unit in the text, thereby more accurately capturing the semantic meaning of the text, and generating the corresponding target face search text description semantic encoding vector and the first candidate face image attribute description semantic encoding vector.

[0052] Specifically, the step S142 calculates the cosine similarity between the semantic coding vector of the target face search text description and the semantic coding vector of the first candidate face image attribute description as the face search matching degree. That is, by calculating the cosine similarity between the semantic coding vector of the target face search text description and the semantic coding vector of the first candidate face image attribute description, the semantic similarity between the two is evaluated, thereby revealing the matching degree between the first candidate face image and the target face search text. It should be understood that the calculation of cosine similarity is based on the vector space model. If the directions of the two vectors are closer, the cosine value is closer to 1, indicating that the similarity is higher; conversely, if the directions are more deviated, the cosine value is closer to 0, indicating that the similarity is lower.

[0053] Specifically, in step S143, in response to the face search matching degree being greater than or equal to a preset threshold, the first candidate face image is used as the search result. It should be understood that if the matching degree between the first candidate face image and the target face search text exceeds the preset threshold, the two are considered to be semantically matched, so that the first candidate face image can be returned to the user as the search result. On the contrary, if it is lower than the threshold, it is necessary to continue searching for other candidate face images until a matching image that meets the conditions is found.

[0054] In summary, the face search method based on multimodal technology according to the embodiment of the present application is explained, which uses data processing technology based on deep learning to label the basic attributes of face images in the face database, generate attribute description text of the face image, and then, through semantic parsing and matching analysis of the target face search text description input by the user and the generated face image attribute description text, thereby realizing face image search based on text description. In this way, face search based on semantic information described in text can be supported, so that face screening work can be carried out quickly, which has positive significance for scenes without images.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A face search method based on multimodal technology, characterized in that: include: Obtain a target face search text description and extract a first candidate face image from a face database; Performing local region fine-grained semantic feature extraction on the first candidate face image to obtain a set of semantic encoding feature vectors of the local region of the face; Performing facial attribute description on the set of semantically encoded feature vectors of the local face region to obtain a first candidate facial image attribute description text; Based on the semantic match between the target face search text description and the first candidate face image attribute description text, it is determined whether to use the first candidate face image as a search result.

2. The face search method based on multimodal technology according to claim 1, characterized in that: Extracting fine-grained semantic features of local regions of the first candidate face image to obtain a set of semantic encoding feature vectors of local regions of the face includes: The first candidate face image is input into a face feature encoder based on the ViT model to obtain a set of semantic coding feature vectors of the local face area.

3. The face search method based on multimodal technology according to claim 2, characterized in that: Performing a facial attribute description on the set of semantically encoded feature vectors of the local face region to obtain a first candidate facial image attribute description text includes: The set of semantically encoded feature vectors of the local face area is input into a face attribute descriptor based on an LLM model to obtain the first candidate face image attribute description text.

4. The face search method based on multimodal technology according to claim 2, characterized in that: Performing a facial attribute description on the set of semantically encoded feature vectors of the local face region to obtain a first candidate facial image attribute description text includes: Performing semantic enhancement based on feature field effect distribution on the set of semantic coding feature vectors of the local area of ​​the human face to obtain a set of enhanced semantic coding feature vectors of the local area of ​​the human face; The set of enhanced face local area semantic coding feature vectors is input into a face attribute descriptor based on an LLM model to obtain the first candidate face image attribute description text.

5. The face search method based on multimodal technology according to claim 4, characterized in that: The set of semantically coded feature vectors of the local region of the face is subjected to semantic enhancement based on feature field effect distribution to obtain a set of enhanced semantically coded feature vectors of the local region of the face, comprising: Using a semantic reconstructor to perform feature reconstruction on each of the local face region semantic coding feature vectors in the set of local face region semantic coding feature vectors to obtain a set of reconstructed local face region semantic coding feature vectors; Calculating the semantic depth implicit representation value of each reconstructed human face local area semantic coding feature vector in the set of reconstructed human face local area semantic coding feature vectors to obtain a set of human face local area semantic depth implicit representation values; Based on the sorting of the set of implicit representation values ​​of the semantic depth of the local face region, re-sorting the set of semantic encoding feature vectors of the local face region to obtain an updated set of semantic encoding feature vectors of the local face region; The set of updated face local area semantic coding feature vectors is subjected to bidirectional context semantic association enhancement to obtain the set of enhanced face local area semantic coding feature vectors.

6. The face search method based on multimodal technology according to claim 5, characterized in that: Using a semantic reconstructor to perform feature reconstruction on each of the face local area semantic coding feature vectors in the set of face local area semantic coding feature vectors to obtain a set of reconstructed face local area semantic coding feature vectors, including: The semantic coding feature vector of the local area of ​​the human face is continuously linearly transformed using the first weight parameter matrix and the second weight parameter matrix to obtain the reconstructed semantic coding feature vector of the local area of ​​the human face.

7. The face search method based on multimodal technology according to claim 6, characterized in that: Calculating the semantic depth implicit representation value of each reconstructed face local area semantic coding feature vector in the set of reconstructed face local area semantic coding feature vectors to obtain a set of face local area semantic depth implicit representation values, including: Calculating the square of the Euclidean norm of the reconstructed face local region semantic coding feature vector to obtain an initial face local region semantic depth implicit representation value; Calculating a ratio of a sum of the semantic depth implicit representation value of the initial human face local area and a difference with the semantic depth implicit representation value of the initial human face local area to obtain a semantic depth representation modulation coefficient; The logarithmic function value with base 2 of the semantic depth representation modulation coefficient is calculated to obtain the semantic depth implicit representation value of the local area of ​​the face.

8. The face search method based on multimodal technology according to claim 7, characterized in that: Performing bidirectional context semantic association enhancement on the set of updated face local area semantic coding feature vectors to obtain the set of enhanced face local area semantic coding feature vectors, including: Inputting the set of updated face local area semantic coding feature vectors into a first semantic encoder based on a forward LSTM model to obtain a set of forward face local area context semantic association coding feature vectors; Inputting the set of updated face local area semantic coding feature vectors into a second semantic encoder based on a backward LSTM model to obtain a set of backward face local area context semantic association coding feature vectors; The set of forward face local area context semantic association coding feature vectors and the set of backward face local area context semantic association coding feature vectors are fused to obtain the set of enhanced face local area semantic coding feature vectors.

9. The face search method based on multimodal technology according to claim 3, characterized in that: Determining whether to use the first candidate face image as a search result based on a semantic match between the target face search text description and the first candidate face image attribute description text includes: Performing semantic coding on the target face search text description and the first candidate face image attribute description text to obtain a target face search text description semantic coding vector and a first candidate face image attribute description semantic coding vector; Calculating the cosine similarity between the target face search text description semantic coding vector and the first candidate face image attribute description semantic coding vector as the face search matching degree; In response to the face search matching degree being greater than or equal to a preset threshold, the first candidate face image is used as the search result.

Images