Figure portrait recognition method and device, storage medium and computer equipment

Abstract

According to the figure portrait recognition method and device, the storage medium and the computer equipment provided by the invention, the image data including the to-be-recognized figure acquired from multiple angles of the monitoring area can be acquired, and then the facial features, the static features, the dynamic features and the spatial-temporal features related to the to-be-recognized figure in the image data from different angles are extracted respectively; therefore, even if the facial features of the to-be-recognized person are partially shielded or the posture of the to-be-recognized person is changed, the features (such as static features, dynamic features and spatio-temporal features) of other dimensions can still provide effective recognition information, and the recognition accuracy of the to-be-recognized person is improved. Therefore, normal operation of the recognition system in a complex scene is guaranteed, misrecognition caused by similar feature interference and missing report caused by shielding and posture change can be effectively reduced, the reliability and stability of the recognition system are effectively improved, and the recognition precision is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, storage medium and computer equipment for character portrait recognition. Background Art

[0002] Currently, after the security monitoring system collects video stream data of the monitored area through the front-end camera, it will identify the human features in the video stream data and determine whether the person is on the whitelist based on the identification results.

[0003] When collecting video stream data in the monitoring area, existing technologies generally use cameras installed at fixed angles to collect video stream data. Due to the limitations of the camera's shooting angle, lighting conditions (such as side light, top light, backlight) and front obstructions (such as trees and buildings), the collected image data may be partially blocked, reflected or blurred, resulting in incomplete feature extraction of the target person; and when performing character feature recognition, traditional methods generally use methods based on template matching, feature point detection or deep learning (such as convolutional neural networks) to extract single facial features. However, due to image quality limitations, the accuracy and robustness of feature extraction are low, and single features are easily interfered with by similar features, resulting in poor matching effects in complex scenarios (such as occlusion and posture changes). Summary of the Invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defects that when monitoring personnel in the monitoring area in the existing technology, the collected image data may be partially occluded, reflected or blurred, resulting in low accuracy and robustness of feature extraction, and inability to adapt to complex scenes.

[0005] This application provides a method for character portrait recognition, the method comprising:

[0006] Acquire image data containing a person to be identified collected from multiple angles of the monitoring area;

[0007] Extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from image data at different angles respectively;

[0008] Fusing facial features, static features, dynamic features, and spatiotemporal features from different angles to obtain a first fused feature;

[0009] The first fusion feature is matched with the personnel features of the whitelist personnel stored in the database, and it is determined whether the person to be identified is a whitelist person according to the first matching result.

[0010] Optionally, extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from image data at different angles respectively includes:

[0011] Image data for each angle:

[0012] Extracting facial features related to the person to be identified from the image data at the angle using a preset facial feature extraction algorithm;

[0013] Extracting static features related to the person to be identified from the image data at the angle using a preset static feature extraction algorithm;

[0014] Using a preset video analysis technology to extract dynamic features related to the person to be identified from the image data at the angle;

[0015] The acquisition time and position information of the image data at the angle are obtained, and the acquisition time and the position information are used as spatiotemporal features related to the person to be identified.

[0016] Optionally, extracting facial features related to the person to be identified from the image data at the angle using a preset facial feature extraction algorithm includes:

[0017] Preprocessing the image data at the angle to obtain preprocessed image data;

[0018] Positioning key facial feature points in the preprocessed image data and obtaining a positioning result;

[0019] Extracting facial contours and texture information related to the face of the person to be identified from the preprocessed image data according to the positioning result;

[0020] The extracted facial features and texture information are used as facial features related to the person to be identified.

[0021] Optionally, extracting static features related to the person to be identified from the image data at the angle using a preset static feature extraction algorithm includes:

[0022] Extracting initial features related to the person to be identified from the image data at the angle, the initial features including at least clothing features, accessories features, and height and body shape features of the person to be identified;

[0023] identifying clothing colors and styles corresponding to the clothing features, and accessory colors and types corresponding to the accessory features;

[0024] Analyzing the proportion of the person to be identified in the image data at the angle according to the height and body shape characteristics, and determining the height and body shape corresponding to the height and body shape characteristics in combination with parameter information of the image acquisition device at the angle;

[0025] The clothing color and style, the accessories color and type, and the height and body shape are used as static features related to the person to be identified.

[0026] Optionally, the extracting dynamic features related to the person to be identified from the image data at the angle by using a preset video analysis technology includes:

[0027] Using a preset video analysis technology to extract the gait cycle, walking speed and habitual movements related to the person to be identified from the image data of the angle;

[0028] The gait cycle, the walking speed and the habitual movements are used as dynamic features related to the person to be identified.

[0029] Optionally, fusing facial features at different angles, static features, dynamic features, and spatiotemporal features to obtain a first fused feature includes:

[0030] The same type of features from different angles are spliced ​​together to obtain facial splicing features, static splicing features, dynamic splicing features, and spatiotemporal splicing features;

[0031] The facial splicing feature, the static splicing feature, the dynamic splicing feature and the spatiotemporal splicing feature are fused to obtain a first fused feature.

[0032] Optionally, matching the first fusion feature with the personnel features of whitelisted personnel stored in a database to obtain a first matching result includes:

[0033] After performing similarity matching between the first fused feature and the features of the whitelisted personnel stored in the database using a cosine similarity algorithm, a candidate set matching the first fused feature is screened from the database based on the similarity matching result;

[0034] The first fusion feature is finely matched with the personnel features in the candidate set using a dynamic time warping algorithm to obtain a first matching result.

[0035] Optionally, determining whether the person to be identified is a whitelisted person according to the first matching result includes:

[0036] The facial features, static features, dynamic features and spatiotemporal features at the same angle are fused to obtain the second fused features at different angles;

[0037] Matching the second fusion features at different angles with the personnel features of the whitelist personnel stored in the database respectively to obtain second matching results at different angles;

[0038] The first matching result is corrected according to the second matching result at different angles, and whether the person to be identified is a whitelist person is determined according to the corrected first matching result.

[0039] Optionally, before extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from the image data at different angles, the method further includes:

[0040] Detecting the image quality of image data at different angles, and determining the image to be completed in the image data at different angles according to the detection results;

[0041] After completing the missing features of the image to be completed, the steps of respectively extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from the image data at different angles are performed.

[0042] This application also provides a person portrait recognition device, comprising:

[0043] An image acquisition module is used to acquire image data containing the person to be identified collected from multiple angles of the monitoring area;

[0044] A feature extraction module is used to extract facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from image data at different angles;

[0045] A feature fusion module is used to fuse facial features, static features, dynamic features and spatiotemporal features from different angles to obtain a first fused feature;

[0046] The feature matching module is used to match the first fusion feature with the personnel features of the white-list personnel stored in the database, and determine whether the person to be identified is a white-list person based on the first matching result.

[0047] The present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the character portrait recognition method as described in any of the above embodiments.

[0048] The present application also provides a computer device, comprising: one or more processors, and a memory;

[0049] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the person portrait recognition method described in any one of the above embodiments are performed.

[0050] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0051] The person portrait recognition method, device, storage medium, and computer device provided in this application can obtain image data containing a person to be identified from multiple angles of a monitored area, and then extract facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from the image data at different angles. The facial features, static features, dynamic features, and spatiotemporal features from different angles are then fused to obtain a first fused feature. Finally, the first fused feature is matched with the person features of whitelisted persons stored in a database, and based on the first matching result, it is determined whether the person to be identified is a whitelisted person. In this process, the present application collects and analyzes the facial features, static features, dynamic features, and spatiotemporal features of the person to be identified to describe and identify the person to be identified from multiple dimensions. In this way, even if the facial features of the person to be identified are partially occluded or the posture changes, the features in other dimensions (such as static features, dynamic features, and spatiotemporal features) can still provide effective identification information, thereby ensuring the normal operation of the recognition system in complex scenarios, and effectively reducing false recognition caused by interference from similar features, as well as missed detections caused by occlusion and posture changes, thereby effectively improving the reliability and stability of the recognition system and improving recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0053] Figure 1 A flowchart of a method for identifying a person's portrait provided in an embodiment of the present application;

[0054] Figure 2 A diagram showing the business logic of the character portrait recognition process provided in the embodiment of this application;

[0055] Figure 3 A schematic diagram of a process for extracting multi-dimensional features from image data at each angle provided in an embodiment of the present application;

[0056] Figure 4A schematic diagram of the structure of a person portrait recognition device provided in an embodiment of the present application;

[0057] Figure 5 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0059] In one embodiment, Figure 1 As shown, Figure 1 A flowchart of a method for character portrait recognition provided in an embodiment of the present application is provided. The present application provides a method for character portrait recognition, which may include:

[0060] S110: Acquire image data containing a person to be identified that is collected from multiple angles in the monitoring area.

[0061] In this step, when obtaining image data of the monitored area, image data collected by cameras installed at multiple angles in the monitored area can be obtained. The image data contains the person to be identified. Therefore, the present application can perform portrait recognition of the person to be identified based on the image features in the image data at each angle.

[0062] In particular, the present application may pre-install multiple cameras in the monitoring area to capture the person from different angles and obtain the person's image data. The camera installation position and angle can be designed according to the actual situation to ensure that the person's face, body posture and surrounding environment information can be captured as comprehensively as possible.

[0063] In a specific implementation, the camera of the present application can be connected to an image acquisition device in the system, which is used to receive and store image data captured by the camera. The image acquisition device can be one or more computers with sufficient storage space and computing power to process large amounts of image data. Furthermore, the image acquisition device can also perform pre-processing on the image data, such as denoising and contrast enhancement, to improve the accuracy of subsequent feature extraction.

[0064] Furthermore, the image data containing the person to be identified collected by the present application can be continuous video data or static picture data. For video data, the present application can extract key frames from it and perform subsequent feature extraction and recognition processing on the key frames. For static picture data, the present application can directly perform feature extraction and recognition processing on it. The person to be identified contained in the image data of the present application can be the front, side, back, upper body, lower body, etc. of the person, which can be determined by the angle of the visual shooting and is not limited here.

[0065] S120: extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from the image data at different angles respectively.

[0066] In this step, after obtaining image data containing the person to be identified from multiple angles of the monitoring area through S110, the facial features, static features, dynamic features and spatiotemporal features related to the person to be identified in the image data at different angles can be extracted respectively. In this way, the multi-dimensional features of each angle can be spliced ​​and fused, and then the person portrait can be recognized based on the fused features.

[0067] Specifically, the present application can utilize one or more feature extraction algorithms to extract facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from image data at different angles. These feature extraction algorithms can be based on machine learning or deep learning, or other methods, and the specific selection can be based on actual circumstances and is not limited here.

[0068] For example, for facial feature extraction, the present application can use deep learning algorithms, such as convolutional neural networks (CNNs), to locate facial feature points and extract facial contours and texture information. For static feature extraction, the present application can use image segmentation and object recognition technology to extract information such as a person's clothing, accessories, height, and body shape. For dynamic feature extraction, the present application can use video analysis technology to extract information such as a person's gait, walking speed, and habitual movements. At the same time, the present application can also obtain the acquisition time and location information of the image data. As spatiotemporal features, the present application can time-tag and position-tag the acquired image data, and count the frequency of a person appearing in different areas at different time periods, thereby determining their frequently appearing time periods and high-frequency activity areas.

[0069] S130: Fusing facial features at different angles, static features, dynamic features, and spatiotemporal features to obtain a first fused feature.

[0070] In this step, after extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from the image data at different angles in S120, the present application can splice and fuse the facial features, static features, dynamic features, and spatiotemporal features extracted from different angles to obtain a first fused feature. This process can fully utilize the information in the multi-angle image data, improve the richness and robustness of the feature representation, and the fused feature vector can more comprehensively describe the characteristics of the person to be identified, thereby improving the accuracy of recognition.

[0071] When fusing facial features from different angles, static features, dynamic features, and spatiotemporal features, the present application may employ methods such as feature concatenation, feature weighting, or feature fusion algorithms. For example, since facial features may differ from angle to angle, facial features from different angles can be aligned and normalized before concatenation. Similar methods can be used for static features, dynamic features, and spatiotemporal features. After concatenating to obtain a preliminary feature vector, the present application may further process the preliminary feature vector using a feature fusion algorithm to improve the robustness and accuracy of the feature representation. For example, the present application may employ methods such as principal component analysis (PCA) or linear discriminant analysis (LDA) to reduce the dimensionality of the feature vector, remove redundant information, and retain the most representative features. Furthermore, the present application may employ feature weighting to weight features from different angles and types to reflect their importance and contribution to the recognition process. Through this process, the present application can obtain a more comprehensive, accurate, and robust feature representation, providing strong support for subsequent character portrait recognition.

[0072] S140: Match the first fusion feature with the personnel features of the whitelist personnel stored in the database, and determine whether the person to be identified is a whitelist personnel based on the first matching result.

[0073] In this step, facial features, static features, dynamic features and spatiotemporal features at different angles are fused through S130 to obtain a first fused feature. The present application can then match the first fused feature with the personnel features of the white-listed personnel stored in the database, and determine whether the person to be identified is a white-listed person based on the first matching result.

[0074] The whitelisted personnel in this application refer to personnel who have passed identity verification or been pre-authorized, and whose personal characteristics have been stored in the database. Therefore, this application can quickly determine whether the person to be identified is a whitelisted person by matching the first fused feature obtained by fusion with the personal characteristics of the whitelisted personnel.

[0075] Furthermore, when matching the first fused feature with the person features of the whitelisted persons, the present application may use various matching algorithms, such as the cosine similarity algorithm, the Euclidean distance algorithm, or the dynamic time warping algorithm, etc. These algorithms can determine whether two feature vectors belong to the same person based on the similarity or distance between the feature vectors.

[0076] Schematically, as Figure 2 As shown, Figure 2 A diagram showing the business logic of the character portrait recognition process provided in the embodiment of this application; Figure 2 In this application, image data captured by an IPC (Internet Protocol Camera) device from different angles can be obtained, and then facial features, static features, dynamic features, and spatiotemporal features can be extracted from the image data from different angles. Subsequently, the facial features, static features, dynamic features, and spatiotemporal features from different angles can be fused. After obtaining a first fused feature, the similarity or distance between the first fused feature and the personal features of a whitelisted person can be calculated. The best matching whitelisted person can then be determined based on a preset threshold or sorting result. If the similarity between the first fused feature and the personal features of a whitelisted person exceeds a preset threshold, or the sorting result ranks in the top N (N is a preset positive integer), the person to be identified can be considered to be that whitelisted person. Conversely, if the similarity between the first fused feature and the personal features of all whitelisted persons is below a preset threshold, or the sorting results are not in the top N, the person to be identified can be considered not to be on the whitelist, i.e., not a whitelisted person.

[0077] Finally, the present application can also use a storage module to store relevant data and results from the matching and identification process. The storage module of the present application uses a large-capacity storage device to ensure that a large amount of feature data can be stored. At the same time, in order to increase the speed of reading and writing data, the present application can also use an efficient database management system to manage and organize the data.

[0078] In the above embodiment, image data containing the person to be identified can be acquired from multiple angles of the monitored area. The facial features, static features, dynamic features, and spatiotemporal features associated with the person to be identified are then extracted from the image data at different angles. The facial features, static features, dynamic features, and spatiotemporal features from the different angles are then fused to obtain a first fused feature. Finally, the first fused feature is matched with the person features of whitelisted persons stored in a database, and a determination is made based on the first matching result whether the person to be identified is a whitelisted person. In this process, the present application collects and analyzes the facial features, static features, dynamic features, and spatiotemporal features of the person to be identified in order to describe and identify the person to be identified from multiple dimensions. This allows the identification system to operate normally in complex scenarios, even if the facial features of the person to be identified are partially obscured or their posture changes. Features in other dimensions (such as static features, dynamic features, and spatiotemporal features) can still provide effective identification information, thereby ensuring the normal operation of the identification system in complex scenarios. Furthermore, it can effectively reduce false recognition caused by interference from similar features, as well as missed detections caused by occlusion or posture changes, thereby effectively improving the reliability and stability of the identification system and enhancing recognition accuracy.

[0079] In one embodiment, Figure 3 As shown, Figure 3 A schematic diagram of a process for extracting multi-dimensional features from image data at each angle provided in an embodiment of the present application; S120, respectively extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from image data at different angles, may include:

[0080] Image data for each angle:

[0081] S121: Using a preset facial feature extraction algorithm, facial features related to the person to be identified are extracted from the image data at the angle.

[0082] S122: Extracting static features related to the person to be identified from the image data at the angle using a preset static feature extraction algorithm.

[0083] S123: Using a preset video analysis technology, extract dynamic features related to the person to be identified from the image data at the angle.

[0084] S124: Acquire the acquisition time and position information of the image data at the angle, and use the acquisition time and position information as spatiotemporal features related to the person to be identified.

[0085] In this embodiment, for image data from each angle, the present application uses detailed steps to extract multi-dimensional features related to the person to be identified. These multi-dimensional features include, but are not limited to, facial features such as facial contours, static features such as clothing and accessories, dynamic features such as walking posture, and spatiotemporal features such as frequently occurring time periods and high-frequency activity areas. The present application can fully utilize the multi-dimensional information in multi-angle image data to improve the richness and robustness of feature representation.

[0086] Specifically, when extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from image data at different angles, the present application can first utilize pre-defined facial feature extraction algorithms, such as convolutional neural networks (CNNs) within deep learning models, to accurately locate facial landmarks and extract the contours and texture information of the facial features. These facial features are crucial for identifying an individual. Secondly, the present application can use pre-defined static feature extraction algorithms, combined with image segmentation and object recognition techniques, to meticulously analyze static information such as the person's clothing, accessories, height, and body shape in the image. This information can assist in identification when facial features are limited. Furthermore, the present application can utilize pre-defined video analysis techniques, such as optical flow or trajectory analysis, to capture and extract dynamic features such as a person's gait, walking speed, and habitual movements. These dynamic features can reflect a person's behavioral patterns and further enhance recognition accuracy. Finally, the present application can obtain the acquisition time and location information of the image data. This information not only constitutes spatiotemporal features but can also reveal a person's activity patterns and frequented locations by counting the frequency of a person's appearance in different time periods and areas, providing additional contextual information for person portrait recognition.

[0087] Through these detailed and comprehensive feature extraction steps, this application can fully utilize the rich information in multi-angle image data, laying a solid foundation for subsequent character portrait recognition and matching.

[0088] In one embodiment, extracting facial features related to the person to be identified from the image data at the angle using a preset facial feature extraction algorithm in S121 may include:

[0089] S1211: Preprocess the image data of the angle to obtain preprocessed image data.

[0090] S1212: Positioning key facial feature points in the pre-processed image data and obtaining positioning results.

[0091] S1213: Extracting facial contours and texture information related to the face of the person to be identified from the preprocessed image data according to the positioning result.

[0092] S1214: Using the extracted facial features and texture information as facial features related to the person to be identified.

[0093] In this embodiment, when extracting facial features relevant to the person to be identified from image data at each angle, the present application may employ advanced facial feature extraction algorithms to extract biometric templates from the image data (e.g., facial contours and texture). For example, the present application may first preprocess the image, including operations such as denoising and contrast enhancement, to improve image quality. Subsequently, the present application may accurately locate key facial feature points in the preprocessed image data. These key feature points typically include biometrically significant areas such as the corners of the eyes, mouth, and nose. Using advanced localization algorithms, such as regression models from deep learning, the present application can accurately obtain the location information of these feature points. Next, based on the localization results, the present application may extract facial contours and texture information relevant to the face of the person to be identified from the preprocessed image data. This information is an important component of facial features and can reflect the unique facial characteristics of an individual. Finally, the present application may store and process the extracted facial contours and texture information as facial features relevant to the person to be identified.

[0094] It is understood that the facial features extracted by this application through the above process not only include the precise location and shape information of the facial features, but also include detailed features such as the texture and color of the facial skin. These features together constitute a comprehensive description of the face of the person to be identified. At the same time, because this application uses advanced facial feature extraction algorithms and deep learning technology, the facial feature extraction process is more efficient and accurate, and can adapt to the needs of facial feature extraction under different lighting, posture and occlusion conditions.

[0095] In one embodiment, extracting static features related to the person to be identified from the image data at the angle using a preset static feature extraction algorithm in S122 may include:

[0096] S1221: Extracting initial features related to the person to be identified from the image data of the angle, wherein the initial features include at least clothing features, accessories features, and height and body shape features of the person to be identified.

[0097] S1222: Identify the clothing color and style corresponding to the clothing feature, and the accessory color and type corresponding to the accessory feature.

[0098] S1223: Analyze the proportional relationship of the person to be identified in the image data of the angle according to the height and body shape features, and determine the height and body shape corresponding to the height and body shape features in combination with parameter information of the image acquisition device at the angle.

[0099] S1224: The clothing color and style, the accessories color and type, and the height and body shape are used as static features related to the person to be identified.

[0100] In this embodiment, when extracting static features related to the person to be identified from the image data at each angle, a preset static feature extraction algorithm can be obtained. This static feature extraction algorithm includes a combination of one or more extraction algorithms. Therefore, the present application can use the preset static feature extraction algorithm to comprehensively and accurately extract static features related to the person to be identified from the image data. These static features include, but are not limited to, clothing features, accessory features, and height and body shape features. These features can provide important auxiliary information for subsequent character portrait recognition and matching.

[0101] In order to achieve this goal, the present application can first extract the initial features related to the person to be identified in the image data. These initial features cover key information such as the clothing, accessories, height and body shape of the person to be identified. Subsequently, the present application can use advanced image recognition technology to further identify the clothing color and style corresponding to the clothing features, and the accessory color and type corresponding to the accessory features. This information can carefully depict the appearance characteristics of the person to be identified and enhance the accuracy of recognition. At the same time, the present application can also analyze the proportional relationship of the person to be identified in the image data based on the height and body shape characteristics, and combine the parameter information of the image acquisition device, such as the focal length of the camera, the shooting angle, etc., to determine the actual height and body shape corresponding to the height and body shape characteristics. This process fully takes into account the shooting conditions of the image data and the parameter differences of the equipment, and can more accurately restore the actual height and body shape characteristics of the person to be identified.

[0102] Through the static feature extraction method of this embodiment, the present application can comprehensively and accurately extract static features related to the person to be identified in the image data. These static features not only enrich the dimensions of the character portrait, but also provide strong support for subsequent character portrait recognition and matching. At the same time, since the present application adopts advanced image recognition technology and deep learning algorithms, the static feature extraction process is more efficient and accurate, and can adapt to the static feature extraction needs under different lighting, posture and occlusion conditions. In addition, by combining the parameter information of the image acquisition device to correct the height and body shape, the present application can also more accurately restore the actual height and body shape characteristics of the person to be identified, further improving the accuracy and robustness of recognition.

[0103] In one embodiment, extracting dynamic features related to the person to be identified from the image data at the angle using a preset video analysis technology in S123 may include:

[0104] S1231: Using a preset video analysis technology, extract the gait cycle, walking speed and habitual movements related to the person to be identified from the image data of the angle.

[0105] S1232: Taking the gait cycle, the walking speed, and the habitual movements as dynamic features related to the person to be identified.

[0106] In this embodiment, when extracting dynamic features related to the person to be identified from the image data at each angle, pre-defined video analysis techniques, such as recurrent neural networks (RNNs) or long short-term memory networks (LSTMs) in deep learning, can be used to extract dynamic features such as the person's gait cycle, walking speed, and habitual movements (such as hand waving amplitude and head bowing frequency). This application can track the person's motion trajectory by analyzing multiple consecutive frames of images, calculating the gait cycle and walking speed. Simultaneously, the person's habitual movements during walking are observed, and dynamic feature vectors are generated by statistically analyzing the frequency and amplitude of these movements.

[0107] It is understandable that these dynamic feature vectors extracted in this application can reflect a person's walking habits and movement patterns, providing behavioral information for character portrait recognition. For example, the gait cycle can reveal a person's walking rhythm and stability, the walking speed can reflect the person's impatience or calm personality, and habitual movements can reveal the person's unconscious personal characteristics. Combining these dynamic features with facial features, static features, and spatiotemporal features can further improve the accuracy and reliability of character portrait recognition.

[0108] In one embodiment, in S130, facial features at different angles, static features, dynamic features, and spatiotemporal features are fused to obtain a first fused feature, which may include:

[0109] S131: Features of the same type from different angles are stitched together to obtain facial stitching features, static stitching features, dynamic stitching features, and spatiotemporal stitching features.

[0110] S132: Fusing the facial stitching feature, the static stitching feature, the dynamic stitching feature, and the spatiotemporal stitching feature to obtain a first fused feature.

[0111] In this embodiment, when fusing facial features, static features, dynamic features and spatiotemporal features from different angles, features of the same type can be spliced ​​first to obtain facial splicing features, static splicing features, dynamic splicing features and spatiotemporal splicing features, and then the splicing features of each dimension can be fused to obtain the first fused feature.

[0112] Specifically, the present application can stitch facial features collected from different angles to form facial stitching features; stitch static features from different angles to form static stitching features; stitch dynamic features from different angles to form dynamic stitching features; and stitch spatiotemporal features from different angles to form spatiotemporal stitching features. Furthermore, when stitching together features of the same type from different angles, the present application can stitch directly, stitch by weighted summation, or stitch by tensor fusion. The specific stitching method can be set according to the actual situation and is not limited here.

[0113] Subsequently, the present application can fuse the facial splicing features, static splicing features, dynamic splicing features and spatiotemporal splicing features obtained by the above splicing to obtain a first fused feature. This fusion process can use a feature fusion algorithm, such as the feature fusion layer in deep learning, which can organically integrate different types of features to form a comprehensive and accurate feature representation. Of course, the present application can also use other feature fusion algorithms, which can be selected according to actual conditions and are not limited here. The first fused feature obtained after fusion in the present application not only includes static information such as the face, clothing, accessories, etc. of the person to be identified, but also includes dynamic information such as gait, walking speed, habitual movements, and spatiotemporal information such as acquisition time and location, thereby providing strong support for subsequent character portrait recognition and matching.

[0114] Through the above feature extraction and fusion steps, this application can fully utilize the multi-dimensional information in multi-angle image data to construct a comprehensive and accurate character portrait feature representation. This feature representation not only improves the accuracy and reliability of character portrait recognition, but also provides a rich information foundation for subsequent character portrait applications.

[0115] In one embodiment, in S140, matching the first fusion feature with the features of the whitelisted personnel stored in the database to obtain a first matching result may include:

[0116] S141: After performing similarity matching between the first fusion feature and the personnel features of the whitelist personnel stored in the database using a cosine similarity algorithm, a candidate set matching the first fusion feature is screened from the database based on the similarity matching result.

[0117] S142: Use a dynamic time warping algorithm to finely match the first fusion feature with the personnel features in the candidate set to obtain a first matching result.

[0118] In this embodiment, when matching the first fused feature with the personal features of the whitelisted individuals stored in the database, a similarity matching algorithm can be directly used, or a weighted similarity algorithm can be used to calculate the matching degree between the target and the database personnel. The specific choice can be made based on the actual situation and is not limited here. When the weighted similarity algorithm is selected for matching in this application, a two-step matching strategy can be adopted to ensure the accuracy and efficiency of the match.

[0119] First, this application can use the cosine similarity algorithm for preliminary matching. The cosine similarity algorithm is a metric that measures the directional similarity between two vectors. By calculating the cosine value of the first fused feature and the features of each person on the whitelist in the database, this application can filter out the candidate set that is closest in direction to the first fused feature. This step effectively narrows the scope of subsequent matching and improves matching efficiency.

[0120] Subsequently, the present application can use the dynamic time warping algorithm to perform fine matching on the candidate set that has been preliminarily screened. The dynamic time warping algorithm is a method for measuring the similarity between two time series, which can handle the matching problem between time series of different lengths or speeds. In character portrait recognition, since dynamic features such as walking speed and gait cycle of different people may vary, the dynamic time warping algorithm can adapt well to such changes and achieve more precise matching. Through the dynamic time warping algorithm, the present application can further determine which person feature in the candidate set best matches the first fusion feature, thereby obtaining the first matching result.

[0121] This application utilizes a two-step matching strategy that ensures both accuracy and efficiency. The initial matching using the cosine similarity algorithm is fast and effective, rapidly narrowing the matching range; while the refined matching using the dynamic time warping algorithm ensures the accuracy and reliability of the final matching result. This strategy makes this application highly valuable in the field of person portrait recognition.

[0122] In one embodiment, determining whether the person to be identified is a whitelisted person according to the first matching result in S140 may include:

[0123] S143: Fusing facial features, static features, dynamic features, and spatiotemporal features at the same angle to obtain second fused features at different angles.

[0124] S144: Match the second fusion features at different angles with the personnel features of the whitelist personnel stored in the database respectively to obtain second matching results at different angles.

[0125] S145: Correcting the first matching result according to the second matching result at different angles, and determining whether the person to be identified is a whitelisted person according to the corrected first matching result.

[0126] In this embodiment, to further improve the accuracy and robustness of character portrait recognition, this application can also adopt a multi-angle feature fusion and matching strategy. Specifically, after obtaining the first matching result, this application does not directly determine whether the person to be identified is a whitelisted person based on this result, but further fuses and matches features from different angles to obtain more comprehensive and accurate matching information.

[0127] In one specific implementation, the present application can fuse facial features, static features, dynamic features, and spatiotemporal features from the same angle to obtain a second fused feature for that angle. This fusion process is similar to the acquisition of the first fused feature, employing the same feature fusion algorithm to organically integrate different types of features from the same angle to form a comprehensive and accurate feature representation. Next, the present application can match the second fused features from different angles against the personal features of whitelisted individuals stored in the database to obtain second matching results for different angles. This step utilizes a matching algorithm and strategy similar to the first matching process, ensuring matching accuracy and efficiency. Finally, the present application can modify the first matching result based on the second matching results from different angles. Because image data from different angles may contain different information, the second matching result may provide useful information not included in the first matching result. By comprehensively considering the matching results from different angles, the present application can further improve matching accuracy and reliability. Based on this, the present application can determine whether the person to be identified is a whitelisted individual based on the modified first matching result.

[0128] Through the above-mentioned multi-angle feature fusion and matching strategy, this application can fully utilize the information in multi-angle image data to further improve the accuracy and robustness of character portrait recognition. This strategy not only improves recognition accuracy but also provides a richer information foundation for subsequent character portrait applications, with broad application prospects and value.

[0129] In one embodiment, before extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from the image data at different angles in S120, the following steps may also be included:

[0130] S111: Detecting the image quality of the image data at different angles, and determining the image to be completed in the image data at different angles according to the detection result.

[0131] S112: After completing the missing features of the image to be completed, executing the steps of respectively extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from the image data at different angles.

[0132] In this embodiment, before extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified in image data from different angles, the present application can also add image quality detection and feature completion steps to improve the accuracy and completeness of feature extraction.

[0133] Specifically, the present application can utilize image quality detection technologies, such as image clarity detection, noise detection, and lighting detection, to comprehensively test image data from different angles. Through this step, the present application can identify images to be completed that have poor image quality, blur, noise, or insufficient lighting. For these images to be completed, the present application can use image restoration technology or feature completion algorithms to complete the missing features.

[0134] For example, for images with missing facial features, this application can use generative adversarial networks (GANs) or super-resolution technology in deep learning to repair and enhance facial features; for images with missing static features, this application can infer and complete features such as clothing and accessories based on contextual information or image data from other angles; for images with missing dynamic features, this application can use motion estimation and tracking algorithms to estimate and complete features such as gait and walking speed; for images with missing spatiotemporal features, this application can infer and complete spatiotemporal features based on acquisition time and location information.

[0135] Through the above-mentioned feature completion process, this application can ensure that the image data at different angles contains complete and accurate feature information, providing strong support for subsequent feature extraction and fusion. On this basis, this application further performs the steps of extracting facial features, static features, dynamic features, and spatiotemporal features related to the person to be identified from the image data at different angles, which can further improve the accuracy and completeness of feature extraction and provide stronger support for subsequent character portrait recognition and matching.

[0136] The following describes a person portrait recognition device provided in an embodiment of the present application. The person portrait recognition device described below and the person portrait recognition method described above can be referenced to each other.

[0137] In one embodiment, Figure 4 As shown, Figure 4This is a structural diagram of a person portrait recognition device provided in an embodiment of the present application. The present application also provides a person portrait recognition device, which may include an image acquisition module 210, a feature extraction module 220, a feature fusion module 230, and a feature matching module 240, specifically including the following:

[0138] The image acquisition module 210 is used to acquire image data containing the person to be identified, collected from multiple angles in the monitoring area.

[0139] The feature extraction module 220 is used to extract facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from image data at different angles.

[0140] The feature fusion module 230 is used to fuse facial features at different angles, static features, dynamic features and spatiotemporal features to obtain a first fused feature.

[0141] The feature matching module 240 is configured to match the first fusion feature with the personal features of whitelisted persons stored in a database, and determine whether the person to be identified is a whitelisted person based on a first matching result.

[0142] In the above embodiment, image data containing the person to be identified can be acquired from multiple angles of the monitored area. The facial features, static features, dynamic features, and spatiotemporal features associated with the person to be identified are then extracted from the image data at different angles. The facial features, static features, dynamic features, and spatiotemporal features from the different angles are then fused to obtain a first fused feature. Finally, the first fused feature is matched with the person features of whitelisted persons stored in a database, and a determination is made based on the first matching result whether the person to be identified is a whitelisted person. In this process, the present application collects and analyzes the facial features, static features, dynamic features, and spatiotemporal features of the person to be identified in order to describe and identify the person to be identified from multiple dimensions. This allows the identification system to operate normally in complex scenarios, even if the facial features of the person to be identified are partially obscured or their posture changes. Features in other dimensions (such as static features, dynamic features, and spatiotemporal features) can still provide effective identification information, thereby ensuring the normal operation of the identification system in complex scenarios. Furthermore, it can effectively reduce false recognition caused by interference from similar features, as well as missed detections caused by occlusion or posture changes, thereby effectively improving the reliability and stability of the identification system and enhancing recognition accuracy.

[0143] In one embodiment, the present application also provides a computer-readable storage medium, which stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the character portrait recognition method described in any of the above embodiments.

[0144] In one embodiment, the present application further provides a computer device, including: one or more processors, and a memory.

[0145] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the person portrait recognition method described in any one of the above embodiments are performed.

[0146] Schematically, as Figure 5 As shown, Figure 5 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 can be provided as a server. Figure 5 Computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by memory 301 for storing instructions executable by processing component 302, such as an application. The application stored in memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, processing component 302 is configured to execute the instructions to perform the person portrait recognition method of any of the above-mentioned embodiments.

[0147] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or the like.

[0148] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0149] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0150] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0151] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for character portrait recognition, characterized in that: The method comprises: Acquire image data containing a person to be identified collected from multiple angles of the monitoring area; Extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from image data at different angles respectively; Fusing facial features, static features, dynamic features, and spatiotemporal features from different angles to obtain a first fused feature; The first fusion feature is matched with the personnel features of the whitelist personnel stored in the database, and it is determined whether the person to be identified is a whitelist person according to the first matching result.

2. The method for character portrait recognition according to claim 1, wherein: The extracting of facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from the image data at different angles includes: Image data for each angle: Extracting facial features related to the person to be identified from the image data at the angle using a preset facial feature extraction algorithm; Extracting static features related to the person to be identified from the image data at the angle using a preset static feature extraction algorithm; Using a preset video analysis technology to extract dynamic features related to the person to be identified from the image data at the angle; The acquisition time and position information of the image data at the angle are obtained, and the acquisition time and the position information are used as spatiotemporal features related to the person to be identified.

3. The method for character portrait recognition according to claim 2, wherein: The method of extracting facial features related to the person to be identified from the image data at the angle by using a preset facial feature extraction algorithm includes: Preprocessing the image data at the angle to obtain preprocessed image data; Positioning key facial feature points in the preprocessed image data and obtaining a positioning result; Extracting facial contours and texture information related to the face of the person to be identified from the preprocessed image data according to the positioning result; The extracted facial features and texture information are used as facial features related to the person to be identified.

4. The method for character portrait recognition according to claim 2, wherein: The step of extracting static features related to the person to be identified from the image data at the angle using a preset static feature extraction algorithm includes: Extracting initial features related to the person to be identified from the image data at the angle, the initial features including at least clothing features, accessories features, and height and body shape features of the person to be identified; identifying clothing colors and styles corresponding to the clothing features, and accessory colors and types corresponding to the accessory features; Analyzing the proportion of the person to be identified in the image data at the angle according to the height and body shape characteristics, and determining the height and body shape corresponding to the height and body shape characteristics in combination with parameter information of the image acquisition device at the angle; The clothing color and style, the accessories color and type, and the height and body shape are used as static features related to the person to be identified.

5. The method for character portrait recognition according to claim 2, wherein: The method of extracting dynamic features related to the person to be identified from the image data at the angle by using a preset video analysis technology includes: Using a preset video analysis technology to extract the gait cycle, walking speed and habitual movements related to the person to be identified from the image data of the angle; The gait cycle, the walking speed and the habitual movements are used as dynamic features related to the person to be identified.

6. The method for character portrait recognition according to claim 1, wherein: The facial features, static features, dynamic features and spatiotemporal features at different angles are fused to obtain a first fused feature, including: The same type of features from different angles are spliced ​​together to obtain facial splicing features, static splicing features, dynamic splicing features, and spatiotemporal splicing features; The facial splicing feature, the static splicing feature, the dynamic splicing feature and the spatiotemporal splicing feature are fused to obtain a first fused feature.

7. The method for character portrait recognition according to claim 1, wherein: The matching of the first fusion feature with the personnel features of the whitelist personnel stored in the database to obtain a first matching result includes: After performing similarity matching between the first fused feature and the features of the whitelisted personnel stored in the database using a cosine similarity algorithm, a candidate set matching the first fused feature is screened from the database based on the similarity matching result; The first fusion feature is finely matched with the personnel features in the candidate set using a dynamic time warping algorithm to obtain a first matching result.

8. The method for character portrait recognition according to claim 1, wherein: Determining whether the person to be identified is a whitelisted person based on the first matching result includes: The facial features, static features, dynamic features and spatiotemporal features at the same angle are fused to obtain the second fused features at different angles; Matching the second fusion features at different angles with the personnel features of the whitelist personnel stored in the database respectively to obtain second matching results at different angles; The first matching result is corrected according to the second matching result at different angles, and whether the person to be identified is a whitelist person is determined according to the corrected first matching result.

9. The method for character portrait recognition according to any one of claims 1 to 8, characterized in that: Before extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from the image data at different angles, the method further includes: Detecting the image quality of image data at different angles, and determining the image to be completed in the image data at different angles according to the detection results; After completing the missing features of the image to be completed, the steps of respectively extracting facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from the image data at different angles are performed.

10. A person portrait recognition device, characterized in that: include: An image acquisition module is used to acquire image data containing the person to be identified collected from multiple angles of the monitoring area; A feature extraction module is used to extract facial features, static features, dynamic features and spatiotemporal features related to the person to be identified from image data at different angles; A feature fusion module is used to fuse facial features, static features, dynamic features and spatiotemporal features from different angles to obtain a first fused feature; The feature matching module is used to match the first fusion feature with the personnel features of the white-list personnel stored in the database, and determine whether the person to be identified is a white-list person based on the first matching result.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the human portrait recognition method according to any one of claims 1 to 9.

12. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the person portrait recognition method according to any one of claims 1 to 9 are performed.

Images