Human body detection and face recognition system and method

Abstract

The invention discloses a human body detection and face recognition system and method, and belongs to the technical field of video stream real-time monitoring and face recognition, and the method comprises the steps: 1, carrying out the motion detection of an obtained video stream transmitted by a camera; 2, executing human body detection according to a motion area obtained by motion detection; step 3, executing face feature extraction in a local face database; step 4, executing multi-image fusion in the video stream; step 5, performing face detection on the detected human body area; step 6, performing face recognition after face detection is performed; the face feature extraction and recognition method is further optimized, so that the detection and recognition precision and the real-time performance of the system in practical application and the adaptive capacity in a complex scene are remarkably improved, and the robustness is good.

Description

Technical Field

[0001] The present invention belongs to the field of human body detection and human face technology, and also relates to the field of video stream real-time monitoring and human face recognition technology, and specifically relates to a human body detection and human face recognition system and method. Background Art

[0002] As mentioned in the prior art solution with the patent publication number "CN212724129U", with the rapid development of computer vision and artificial intelligence technology, human detection and face recognition technology based on image processing and deep learning has been widely used in security monitoring, smart communities, smart access control and other fields. At present, the mainstream human detection and face recognition systems mainly rely on traditional image processing methods or deep learning models based on large-scale neural networks. However, the existing technology still has certain deficiencies in detection accuracy, real-time performance and processing of complex scenes. Traditional image processing-based methods (such as Haar features, HOG features) often cannot accurately detect and identify faces and bodies when facing complex backgrounds, lighting changes or dynamic scenes. In addition, although the deep learning method improves the detection accuracy, it has high requirements on the system's computing resources and is difficult to achieve ideal results in embedded or real-time application scenarios. At present, YOLO (You Only Look Once) is an efficient target detection algorithm that can achieve real-time detection while maintaining high accuracy, while OpenCV's background subtraction algorithm has good dynamic background adaptability in the field of motion detection. Combining these two technologies can effectively solve the shortcomings of traditional methods in complex environments and provide a more efficient solution for human detection and face recognition. However, the existing related systems are not sufficiently optimized in the processing flow in practical applications, and the recognition accuracy still needs to be improved, especially in the lack of more real-time and robust solutions for the integration of human motion detection and face recognition. Summary of the invention

[0003] In order to solve the defects in the prior art, the present invention proposes a human body detection and face recognition system and method, which fully utilizes the high efficiency performance of OpenCV in image processing and combines the deep learning advantages of YOLOv5 in the field of target detection, and further optimizes the extraction and recognition method of facial features, thereby significantly improving the detection and recognition accuracy, real-time performance and adaptability of the system in complex scenarios in practical applications, and has good robustness.

[0004] The present invention uses the following technical solutions.

[0005] A human body detection and face recognition method, comprising: Step 1: Perform motion detection on the original video stream acquired from the camera; Step 2: Perform human body detection based on the motion area obtained by motion detection; Step 3: Perform facial feature extraction in the local face database; Step 4: Perform multi-image fusion in the video stream; Step 5: Perform face detection on the detected human body area: Step 6: Perform face recognition after performing face detection.

[0006] Furthermore, step 1 specifically includes: Step 1-1: Perform background modeling on the acquired video stream to obtain a background model; Step 1-2: Perform foreground extraction on the acquired video stream according to the background model to obtain the foreground; Step 1-3: Perform noise removal on the foreground; Step 1-4: Perform recognition on the moving area of ​​the foreground.

[0007] Furthermore, step 1-1 specifically includes: Firstly, a background model is established for the video frames of the acquired video stream, and the static scenes in the video frames are regarded as the background model. Then, the MOG2 algorithm is used to dynamically adjust and update the background model to filter out unnecessary changes in the background model.

[0008] Furthermore, in step 1-1, the MOG2 algorithm updates the background model using the following formula:

[0009] in Indicates that the horizontal coordinate of the background model at time t+1 is And the vertical coordinate is The background model value of the pixel point, The horizontal coordinate of the background model at time t is And the vertical axis is The background model value of the pixel point, Indicates that the horizontal coordinate of the current video frame is And the vertical coordinate is The pixel value of the pixel point, is the learning rate.

[0010] Furthermore, in step 1-2, if the difference between the current video frame of the video stream and the background model is greater than the set threshold T, the pixel is marked as foreground, as follows:

[0011] in, express Under the condition of , the horizontal coordinate of the current video frame is And the vertical coordinate is The pixel point is the foreground pixel point, otherwise it is not the foreground pixel point.

[0012] Furthermore, in steps 1-3, a Gaussian blur method is applied to smooth the foreground image; in addition, a morphological operation is used to remove detected noise to make the foreground image contour clearer. The specific morphological operations are as follows:

[0013] Among them, E represents the corrosion operation, D represents the expansion operation, and K is the structural element. for The collection formed, is the foreground image after morphological operation.

[0014] Furthermore, steps 1-4 specifically include: Step 1-4-1: Execute the background subtractor, that is, use the background subtractor in OpenCV to extract the moving area of ​​the background; Step 1-4-2: Contour detection, that is, finding the contour of the moving area of ​​the background through the cv2.findContours function in OpenCV; Step 1-4-3: Filter small areas, that is, filter out smaller areas below the set area size by calculating the area of ​​the contour; Step 1-4-4: Display the motion area, that is, draw the border of the detected motion area on the original video frame.

[0015] Furthermore, in step 2, it specifically includes: Step 2-1: Model loading and configuration, that is, first load the pre-trained YOLOv5 model, and configure the input size, category label, and confidence threshold parameters of the YOLOv5 model; Step 2-2: Motion area processing, that is, passing the extracted motion area as input to the YOLOv5 model, which will analyze the input motion area and generate detection results; Step 2-3: Human body recognition, that is, the YOLOv5 model uses its convolutional neural network structure to analyze image information in the moving area in real time and detect human bodies in the video stream. Each recognized human body will be marked and its location information will be returned, including the bounding box coordinates and category confidence; Step 2-4: Result screening, that is, after obtaining all the test results, the reliable test results are screened out according to the set confidence threshold, and the test results below the confidence threshold will be discarded; Step 2-5: Result labeling, that is, the bounding box of each recognized human body will be drawn in the original video frame and the category label will be attached.

[0016] Furthermore, in step 3, it specifically includes: Step 3-1: Data preparation, that is, first load each face image from the local face database; Step 3-2: Face area detection, that is, in each face image, the detected face area is cropped using a rectangular area and Indicates that In are the coordinates of the upper left corner of the face area, In They are the coordinates of the lower right corner of the face area; Step 3-3: Feature coding generation, that is, using the feature extraction function in the face_recognition library to process the cropped face image; Step 3-4: Feature vector storage, that is, storing the extracted feature vector of each face together with its corresponding identity label into the local face feature library.

[0017] Furthermore, in step 4, it specifically includes: Step 4-1: ORB feature point detection, that is, first use the ORB algorithm to detect the key feature points in each frame image and extract their descriptors; Step 4-2: Feature point matching, that is, matching the descriptors of multiple frames of images in pairs, using a brute force matching algorithm, and selecting the first N matching points with the shortest distance as the optimal matching point set S={s i},s i That is Matching points; Step 4-3: Image alignment, that is, according to the matched feature points, the RANSAC algorithm is used to calculate the homography matrix between the images, so as to perform affine transformation of the images and align the face areas of different frames; the homography matrix formula is as follows:

[0018] in and are the coordinates of the corresponding matching points in the two frames of images, and the images are aligned by affine transformation through the homography matrix H; Step 4-4: Weighted image fusion, i.e. aligned images The fusion is performed by weighted averaging, and the aligned multi-frame images are superimposed frame by frame. The fusion is performed by the following weighted averaging formula:

[0019] in, is the fused image, N is the total number of image frames, is the weight of the i-th frame image, is the i-th frame image after alignment.

[0020] Furthermore, in step 5, it specifically includes: Step 5-1: Human body region extraction, that is, first extract each detected human body region from the video frame according to the bounding box information returned by the YOLOv5 model; Step 5-2: Call the face detection model, that is, use the built-in face detection model in the face_recognition library to analyze the extracted human body area; Step 5-3: Face position detection, that is, for each input human body region, the face detection model is applied to its image data to identify and locate the face. The detection result will return the coordinates of the face, including the four corners of the bounding box; Step 5-4: multi-face processing, that is, if multiple faces are detected in a human body region, a corresponding bounding box will be generated for each face; Step 5-5: Result storage and feedback, that is, finally storing the extracted face position and the corresponding bounding box information in the memory, preparing for subsequent face feature extraction and recognition.

[0021] Further, in step 6, it specifically includes: Step 6-1: Feature vector comparison, that is, obtaining the 128-dimensional feature vector of the currently detected face and comparing it with the feature vectors in the locally stored known face feature library; Step 6-2: Calculate the similarity, that is, use the Euclidean distance algorithm to calculate the distance between the newly detected face feature vector and each known face feature vector in the feature library ; Step 6-3: Matching result judgment, that is, according to the calculated distance value, the known facial features that are most similar to the newly detected facial features will be determined; if the Euclidean distance is less than the preset threshold, it is considered that the detected face matches an identity in the feature library, and the label of the identity will be returned; otherwise, it is marked as an "unknown" identity; Step 6-4: Output the recognition result, that is, the recognition result will be output in the end. The recognition result includes the identity information of the detected face and the similarity with the matching in the feature library.

[0022] A human body detection and face recognition system, comprising: The camera and the processing terminal connected to it, the camera is used to collect video streams and transmit them to the processing terminal in real time to perform human body detection and face recognition; The modules running on the processing terminal include: A motion detection module, which is used to perform motion detection on the video stream acquired from the camera; A human body detection module, which is used to perform human body detection according to the motion area obtained by motion detection; A feature extraction module, which is used to perform face feature extraction in a local face database; Image fusion module, which is used to perform multi-image fusion in the video stream: The face detection module is used to perform face detection on the detected human body area: The face recognition module is used to perform face recognition after performing face detection.

[0023] The beneficial effects of the present invention are that, compared with the prior art, the technical effects of the present invention include: It includes: Step 1: performing motion detection on the video stream acquired from the camera; Step 2: performing human body detection based on the motion area acquired through motion detection; Step 3: performing facial feature extraction in a local face database; Step 4: performing multi-image fusion in the video stream; Step 5: performing face detection on the detected human body area: Step 6: performing face recognition after performing face detection; it makes full use of the efficient performance of OpenCV in image processing, and combines the deep learning advantages of YOLOv5 in the field of target detection, further optimizes the extraction and recognition methods of facial features, thereby significantly improving the detection and recognition accuracy, real-time performance and adaptability of the system in complex scenarios in practical applications, and has good robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of the human body detection and face recognition method described in the present invention; Figure 2 It is a flow chart of face recognition of the human body detection and face recognition system described in the present invention. DETAILED DESCRIPTION

[0025] Currently, YOLO (You Only Look Once) is an efficient target detection algorithm that can achieve real-time detection while maintaining high accuracy, while OpenCV's background subtraction algorithm has good dynamic background adaptability in the field of motion detection. Combining these two technologies can effectively solve the shortcomings of traditional methods in complex environments and provide a more efficient solution for human detection and face recognition. However, the existing related systems are insufficient in optimizing the processing flow in practical applications, and the recognition accuracy still needs to be improved, especially in the lack of more real-time and robust solutions for the integration of human motion detection and face recognition.

[0026] In order to solve the problems of low detection accuracy, poor real-time performance and insufficient adaptability to complex scenes in the prior art, the present invention proposes an integrated solution based on OpenCV and YOLOv5. This solution makes full use of the high efficiency of OpenCV in image processing, and combines the deep learning advantages of YOLOv5 in the field of target detection, further optimizes the extraction and recognition method of facial features, thereby significantly improving the detection and recognition accuracy, real-time performance and adaptability of the system in complex scenes in practical applications, and has good robustness.

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the following will be combined with the drawings in the embodiments of the present invention to clearly and completely express the technical solution of the present invention. The embodiments expressed in this application are only some embodiments of the present invention, not all embodiments. According to the spirit of the present invention, other embodiments obtained by technicians in this field without making creative work are all within the protection scope of the present invention.

[0028] like Figure 1 As shown, a human body detection and face recognition method according to the present invention comprises: The present invention adopts a motion detection method based on background modeling, and detects the motion area of ​​the acquired real-time original video stream through the background subtraction algorithm (Background Subtraction) of OpenCV. Specifically, the MOG2 (Mixture of Gaussians) background subtraction algorithm is used, which can dynamically update the background model to adapt to light changes and scene dynamic changes.

[0029] In a preferred but non-limiting embodiment of the present invention, step 1 specifically comprises: Step 1-1: Perform background modeling on the acquired original video stream to obtain a background model; In a preferred but non-limiting embodiment of the present invention, step 1-1 specifically comprises: First, a background model is established for the video frames of the acquired video stream, and the static scenes in the video frames are regarded as the background model. As time goes by, the MOG2 algorithm can be used to dynamically adjust to update the background model and filter out unnecessary changes in the background model, such as changes in ambient light or the movement of small objects.

[0030] In a preferred but non-limiting embodiment of the present invention, in step 1-1, the MOG2 algorithm updates the background model by the following formula:

[0031] in Indicates that the horizontal coordinate of the background model at time t+1 is And the vertical coordinate is The background model value of the pixel point, The horizontal coordinate of the background model at time t is And the vertical coordinate is The background model value of the pixel point, Indicates that the horizontal coordinate of the current video frame is And the vertical coordinate is The pixel value of the pixel point, is the learning rate, which is used to balance the impact of new image information on the background model and is usually set to a small value to smooth the update process.

[0032] Step 1-2: Perform foreground extraction on the acquired video stream according to the background model to obtain the foreground; When the new frame is compared with the background model, the algorithm will consider the area that deviates significantly from the background as the foreground (i.e. the moving object). Through the binarization operation, the outline and moving area of ​​the foreground can be extracted.

[0033] In a preferred but non-limiting embodiment of the present invention, in step 1-2, if the difference between the current video frame of the video stream and the background model is greater than a set threshold T, the pixel is marked as a foreground, as follows:

[0034] in, express Under the condition of , the horizontal coordinate of the current video frame is And the vertical coordinate is The pixel point is the foreground pixel point, otherwise it is not the foreground pixel point.

[0035] Step 1-3: Perform noise removal on the foreground; In a preferred but non-limiting embodiment of the present invention, in steps 1-3, in order to improve the accuracy of motion detection, a Gaussian Blur method is applied to smooth the foreground image and reduce the influence of noise; in addition, morphological operations (such as corrosion and dilation) are used to remove detected noise points to make the foreground image contour clearer. The specific morphological operations are as follows:

[0036] Among them, E represents the corrosion operation, D represents the expansion operation, and K is the structural element. for The collection formed, It is the foreground image after morphological operation, which is used to eliminate noise in detection and make the foreground clearer.

[0037] Step 1-4: Perform recognition on the moving area of ​​the foreground.

[0038] After acquiring the foreground image, the size, shape and position of the moving area will be analyzed to determine the target area that needs further processing. By setting the size threshold of the dynamic area, unnecessary small movements, such as leaves blown by the wind, can be filtered out, so as to focus on more significant moving objects.

[0039] This background subtraction motion detection method can effectively identify moving objects from complex scenes, ensuring that only the moving areas are subsequently analyzed and processed, reducing unnecessary computational overhead. This process lays the foundation for subsequent YOLOv5 human detection and face recognition.

[0040] In a preferred but non-limiting embodiment of the present invention, steps 1-4 specifically include: Step 1-4-1: Execute the background subtractor, that is, use the background subtractor in OpenCV (such as MOG2) to extract the motion area of ​​the background; Step 1-4-2: Contour detection, that is, finding the contour of the moving area of ​​the background through the cv2.findContours function in OpenCV.

[0041] Step 1-4-3: Filter small areas, that is, filter out smaller areas below the set area size by calculating the area of ​​the contour, ensuring that only the larger motion area is focused on; Step 1-4-4: Display the motion area, that is, draw the border of the detected motion area on the original video frame.

[0042] The pseudo code flow of steps 1-4 is shown in Table 1: Table 1

[0043] Step 2: Perform human body detection based on the motion area obtained by motion detection; In a preferred but non-limiting embodiment of the present invention, in step 2, it specifically includes: after completing the extraction of the motion area, the YOLOv5 deep learning model is used to perform human body detection and recognition on the detected motion area. This step includes the following key links: Step 2-1: Model loading and configuration, that is, first load the pre-trained YOLOv5 model to ensure that it has the ability to recognize human bodies. According to specific task requirements, the input size, category label, confidence threshold and other parameters of the YOLOv5 model will be configured to optimize the detection effect; Step 2-2: Motion area processing, that is, passing the extracted motion area as input to the YOLOv5 model, which will analyze the input motion area and generate detection results; Step 2-3: Human body recognition, that is, the YOLOv5 model uses its convolutional neural network (CNN) structure to analyze image information in the moving area in real time and detect human bodies in the video stream. Each recognized human body will be marked and its location information will be returned, including the bounding box coordinates (upper left and lower right corners) and category confidence; Step 2-4: Result screening, that is, after obtaining all the test results, the reliable test results are screened out according to the set confidence threshold. The test results below the confidence threshold will be discarded to reduce false positives and false detections; Step 2-5: Result labeling, that is, the bounding box of each human body identified in the original video frame will be drawn and the category label will be attached. This not only intuitively demonstrates the detection effect, but also lays the foundation for subsequent face recognition.

[0044] Through the above steps, the human body area in the video stream is effectively identified, providing accurate positioning information for subsequent recognition tasks.

[0045] The specific implementation is as follows: 1. YOLOv5 model loading: Load the YOLOv5 pre-trained model and use the custom trained weight file; 2. Region extraction: Apply YOLOv5 to the motion area for human detection; 3. Result processing: Filter out human detection results with confidence higher than the set threshold (such as 0.6) and mark them on the image.

[0046] Step 3: Perform facial feature extraction in the local face database; In a preferred but non-limiting embodiment of the present invention, in step 3, it specifically includes: using the face_recognition library to perform feature encoding on the locally stored face data to construct a face feature library. The specific steps are as follows: Step 3-1: Data preparation, that is, first load each face image from the local face database; each face image should contain a clear face, and these face images have been labeled with corresponding identity information.

[0047] Step 3-2: Face area detection, that is, in each face image, the detected face area is cropped using a rectangular area and Indicates that In are the coordinates of the upper left corner of the face area, In are the coordinates of the lower right corner of the face area.

[0048] Step 3-3: Feature encoding generation, that is, using the feature extraction function in the face_recognition library to process the cropped face image; the face_recognition library uses a deep learning algorithm to analyze various features of the cropped face image (such as eye spacing, face shape, skin color, etc.) and generate a 128-dimensional feature vector for each face. The feature vector formula is:

[0049] Each f i Represents the face image feature points.

[0050] Step 3-4: Feature vector storage, that is, storing the extracted feature vector of each face together with its corresponding identity tag (name or ID) in the local face feature library. The process is as follows: ,in The first The identity label corresponding to the feature points, It is the face feature library.

[0051] Step 3-5: Feature library construction is completed, that is, through this feature extraction step, an efficient face feature library is established, in which each feature vector can be used to compare with the face detected in real time. This lays the foundation for subsequent face recognition and matching.

[0052] Through the above steps, local face data can be effectively converted into face features that can be used for identification, providing support for efficient and accurate face recognition.

[0053] The specific implementation is as follows: 1. Use the face_recognition library: extract the detected facial feature code.

[0054] 2. Face location: Use the face_locations function to get the location of the face in the image.

[0055] 3. Extract encoding: Use the face_encodings function to extract face encoding for subsequent recognition.

[0056] Step 4: Perform multi-image fusion in the video stream; In a preferred but non-limiting embodiment of the present invention, in step 4, it specifically includes: when capturing multiple frames of images from a video stream, using a multi-image fusion technology to align and fuse the multiple frames of face images to improve the stability and accuracy of face recognition. The specific process is as follows: Step 4-1: ORB feature point detection, that is, first use the ORB (Oriented FAST and Rotated BRIEF) algorithm to detect the key feature points in each frame image and extract its descriptor. ORB can efficiently process image features with rotation and scale changes; Step 4-2: Feature point matching, that is, matching the descriptors of multiple frames of images in pairs, using the brute force matching (BFMatcher) algorithm, and selecting the first N matching points with the shortest distance as the optimal matching point set S={s i},s i That is Matching points; Step 4-3: Image alignment, that is, according to the matched feature points, the RANSAC algorithm (random sampling consistency) is used to calculate the homography matrix between the images, so as to perform affine transformation of the images and align the face areas of different frames; the homography matrix formula is as follows:

[0057] in and are the coordinates of the corresponding matching points in the two frames of images, and the images are aligned by affine transformation through the homography matrix H; Step 4-4: Weighted image fusion, i.e. aligned images The fusion is performed by weighted averaging and superimposing frame by frame. The fused image can reduce the noise and error in the single-frame image, making the final extracted facial features more accurate. Especially when processing images with large changes in expression, angle or light, the fused image can significantly improve the accuracy and robustness of face recognition. The aligned multi-frame images are fused according to the following weighted average formula:

[0058] in, is the fused image, N is the total number of image frames, is the weight of the i-th frame image, is the i-th frame image after alignment, which usually takes the same value.

[0059] The specific implementation is as follows: 1. Align images: Use ORB feature detection to align images.

[0060] 2. Fusion image: Multiple images are weighted averaged to generate a fused image.

[0061] Step 5: Perform face detection on the detected human body area: In a preferred but non-limiting embodiment of the present invention, in step 5, it specifically includes: after successfully identifying the human body area in the video stream, further face detection will be performed on these detected human body areas using the face_recognition library, and the specific steps are as follows: Step 5-1: Human body region extraction, that is, first extract each detected human body region from the video frame according to the bounding box information returned by the YOLOv5 model in the previous step; these regions serve as input for subsequent face detection.

[0062] Step 5-2: Face detection model call, that is, using the built-in face detection model in the face_recognition library, usually based on deep learning methods such as HOG (Histogram of Oriented Gradients) or CNN (Convolutional Neural Network), to analyze the extracted human body area; Step 5-3: Face position detection, that is, for each input human body region, the face detection model is applied to its image data to identify and locate the face. The detection result will return the coordinates of the face, including the four corners of the bounding box (top, right, bottom, left); Step 5-4: Multi-face processing, that is, if multiple faces are detected in a human body area, a corresponding bounding box will be generated for each face; this process ensures that multiple faces appearing in complex scenes can be accurately identified.

[0063] Step 5-5: Result storage and feedback, that is, finally storing the extracted face position and the corresponding bounding box information in the memory, ready for subsequent face feature extraction and recognition. This information will be used to mark the face position in the original video frame.

[0064] Through this process, an in-depth analysis of the detected human body area is achieved, and the face position is successfully extracted, laying a solid foundation for the next step of facial feature recognition.

[0065] The specific implementation is as follows: 1. Use HOG model: Use the HOG model in the face_recognition library for face detection.

[0066] 2. Handle exceptions: handle possible abnormal situations to ensure the stability of face detection.

[0067] Step 6: Perform face recognition after performing face detection.

[0068] In a preferred but non-limiting embodiment of the present invention, in step 6, it specifically includes: after successfully performing face detection, identity recognition will be performed. In order to further improve the recognition accuracy, the present invention uses Euclidean distance for further judgment. The specific steps are as follows: Step 6-1: Feature vector comparison, that is, the 128-dimensional feature vector of the currently detected face is obtained and compared with the feature vectors in the locally stored known face feature library; each feature vector corresponds to a label of a known identity.

[0069] Step 6-2: Calculate the similarity, that is, use the Euclidean distance algorithm to calculate the distance between the newly detected face feature vector and each known face feature vector in the feature library. ; Euclidean distance measures the similarity between feature vectors by calculating the straight-line distance between two points. The smaller the distance, the higher the similarity between the two feature vectors.

[0070]

[0071] in, is the feature vector of the i-th face, is the feature vector of the i-th known face in the feature library. When the distance value is small, it indicates that the two feature vectors have high similarity.

[0072] Step 6-3: Matching result judgment, that is, according to the calculated distance value, the known facial features that are most similar to the newly detected facial features will be determined; if the Euclidean distance is less than the preset threshold (for example, 0.6), the detected face is considered to match an identity in the feature library, and the label of the identity will be returned; otherwise, it is marked as an "unknown" identity; Step 6-4: Output the recognition result, that is, the recognition result will be output in the end, which includes the identity information of the detected face and the similarity with the matching in the feature library. This process can not only realize real-time identity recognition, but also provide support for subsequent decision-making based on the similarity.

[0073] Through this step, the system can effectively identify newly detected faces and confirm their identities, ensuring security and accuracy in application scenarios.

[0074] The specific implementation is as follows: 1. Calculate face distance: Use face_distance to calculate the distance between the recognized face and the known faces.

[0075] 2. Face matching: Face matching is performed based on the minimum distance, and a threshold is set to determine whether the match is successful.

[0076] Result drawing: Draw the face frame and the corresponding name on the fused image.

[0077] like Figure 2 As shown, a human body detection and face recognition system according to the present invention comprises: The camera and the processing terminal connected to it, the camera is used to collect video streams and transmit them to the processing terminal in real time to perform human body detection and face recognition; The modules running on the processing terminal include: A motion detection module, which is used to perform motion detection on the video stream acquired from the camera; A human body detection module, which is used to perform human body detection according to the motion area obtained by motion detection; A feature extraction module, which is used to perform face feature extraction in a local face database; Image fusion module, which is used to perform multi-image fusion in the video stream: The face detection module is used to perform face detection on the detected human body area: The face recognition module is used to perform face recognition after performing face detection. The processing terminal can be a computer.

[0078] The beneficial effects of the present invention are that, compared with the prior art, the technical effects of the present invention include: It makes full use of the high-efficiency performance of OpenCV in image processing and combines the deep learning advantages of YOLOv5 in the field of target detection to further optimize the extraction and recognition methods of facial features, thereby significantly improving the system's detection and recognition accuracy, real-time performance, and adaptability to complex scenarios in practical applications, with good robustness.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the protection space of the claims of the present invention.

Claims

1. A human body detection and face recognition method, characterized in that: include: Step 1: Perform motion detection on the original video stream acquired from the camera; Step 2: Perform human body detection based on the motion area obtained by motion detection; Step 3: Perform facial feature extraction in the local face database; Step 4: Perform multi-image fusion in the video stream; Step 5: Perform face detection on the detected human body area: Step 6: Perform face recognition after performing face detection.

2. The human body detection and face recognition method according to claim 1, characterized in that: Step 1 specifically includes: Step 1-1: Perform background modeling on the acquired video stream to obtain a background model; Step 1-2: Perform foreground extraction on the acquired video stream according to the background model to obtain the foreground; Step 1-3: Perform noise removal on the foreground; Step 1-4: Perform recognition on the moving area of ​​the foreground.

3. The human body detection and face recognition method according to claim 2, characterized in that: Step 1-1 specifically includes: First, a background model is established for the video frames of the acquired video stream, and the static scenes in the video frames are regarded as the background model. Then, the MOG2 algorithm is used to dynamically adjust and update the background model to filter out unnecessary changes in the background model. In step 1-1, the MOG2 algorithm updates the background model using the following formula: ; in Indicates at time t The horizontal coordinate of the background model at time +1 is And the vertical axis is The background model value of the pixel point, Indicates at time t The horizontal coordinate of the background model at time And the vertical axis is The background model value of the pixel point, Indicates that the horizontal coordinate of the current video frame is And the vertical axis is The pixel value of the pixel point, is the learning rate.

4. The human body detection and face recognition method according to claim 3, characterized in that: In step 1-2, if the difference between the current video frame of the video stream and the background model is greater than the set threshold T Then mark the pixel as foreground, as follows: ; in, express Under the condition of , the horizontal coordinate of the current video frame is And the vertical axis is The pixel point is the foreground pixel point, otherwise it is not the foreground pixel point.

5. The human body detection and face recognition method according to claim 4, characterized in that: In steps 1-3, the Gaussian blur method is applied to smooth the foreground image; in addition, morphological operations are used to remove detected noise to make the foreground image contour clearer. The specific morphological operations are as follows: ;in, E represents the corrosion operation, D represents the expansion operation, K is the structural element, for The collection formed, is the foreground image after morphological operation.

6. The human body detection and face recognition method according to claim 5, characterized in that: Steps 1-4 specifically include: Step 1-4-1: Execute the background subtractor, that is, use the background subtractor in OpenCV to extract the moving area of ​​the background; Step 1-4-2: Contour detection, that is, finding the contour of the moving area of ​​the background through the cv2.findContours function in OpenCV; Step 1-4-3: Filter small areas, that is, filter out smaller areas below the set area size by calculating the area of ​​the contour; Step 1-4-4: Display the motion area, that is, draw the border of the detected motion area on the original video frame.

7. The human body detection and face recognition method according to claim 6, characterized in that: In step 2, it specifically includes: Step 2-1: Model loading and configuration, that is, first load the pre-trained YOLOv5 model, and configure the input size, category label, and confidence threshold parameters of the YOLOv5 model; Step 2-2: Motion area processing, that is, passing the extracted motion area as input to the YOLOv5 model, which will analyze the input motion area and generate detection results; Step 2-3: Human body recognition, that is, the YOLOv5 model uses its convolutional neural network structure to analyze image information in the moving area in real time and detect human bodies in the video stream. Each recognized human body will be marked and its location information will be returned, including the bounding box coordinates and category confidence; Step 2-4: Result screening, that is, after obtaining all the test results, the reliable test results are screened out according to the set confidence threshold, and the test results below the confidence threshold will be discarded; Step 2-5: Result labeling, that is, the bounding box of each recognized human body will be drawn in the original video frame and the category label will be attached.

8. The method for human body detection and face recognition according to claim 7, characterized in that: In step 3, it specifically includes: Step 3-1: Data preparation, that is, first load each face image from the local face database; Step 3-2: Face area detection, that is, in each face image, the detected face area is cropped using a rectangular area and Indicates that In are the coordinates of the upper left corner of the face area, In are the coordinates of the lower right corner of the face area; Step 3-3: Feature coding generation, that is, using the feature extraction function in the face_recognition library to process the cropped face image; Step 3-4: Feature vector storage, that is, storing the extracted feature vector of each face together with its corresponding identity label into the local face feature library.

9. The human body detection and face recognition method according to claim 8, characterized in that: In step 4, it specifically includes: Step 4-1: ORB feature point detection, that is, first use the ORB algorithm to detect the key feature points in each frame image and extract their descriptors; Step 4-2: Feature point matching, that is, matching the descriptors of multiple frames of images in pairs, using a brute force matching algorithm, and selecting the one with the shortest distance. N The optimal matching point set S ={s i },s i That is Matching points; Step 4-3: Image alignment, that is, according to the matched feature points, the RANSAC algorithm is used to calculate the homography matrix between the images, so as to perform affine transformation of the images and align the face areas of different frames; the homography matrix formula is as follows: ; in and They are the coordinates of the corresponding matching points in the two frames of images, respectively. H Align the images by affine transformation; Step 4-4: Weighted image fusion, i.e. aligned images The fusion is performed by weighted averaging, and the aligned multi-frame images are superimposed frame by frame. The fusion is performed by the following weighted averaging formula: ; in, is the fused image, N is the total number of image frames, For the The weight of the frame image, After alignment Frame image; In step 5, it specifically includes: Step 5-1: Human body region extraction, that is, first extract each detected human body region from the video frame according to the bounding box information returned by the YOLOv5 model; Step 5-2: Call the face detection model, that is, use the built-in face detection model in the face_recognition library to analyze the extracted human body area; Step 5-3: Face position detection, that is, for each input human body region, the face detection model is applied to its image data to identify and locate the face. The detection result will return the coordinates of the face, including the four corners of the bounding box; Step 5-4: multi-face processing, that is, if multiple faces are detected in a human body region, a corresponding bounding box will be generated for each face; Step 5-5: Result storage and feedback, that is, finally storing the extracted face position and the corresponding bounding box information in the memory, preparing for subsequent face feature extraction and recognition; In step 6, it specifically includes: Step 6-1: Feature vector comparison, that is, obtaining the 128-dimensional feature vector of the currently detected face and comparing it with the feature vectors in the locally stored known face feature library; Step 6-2: Calculate the similarity, that is, use the Euclidean distance algorithm to calculate the distance between the newly detected face feature vector and each known face feature vector in the feature library ; Step 6-3: Matching result judgment, that is, according to the calculated distance value, the known facial features that are most similar to the newly detected facial features will be determined; if the Euclidean distance is less than the preset threshold, it is considered that the detected face matches an identity in the feature library, and the label of the identity will be returned; otherwise, it is marked as an "unknown" identity; Step 6-4: Output the recognition result, that is, the recognition result will be output in the end. The recognition result includes the identity information of the detected face and the similarity with the matching in the feature library.

10. A human body detection and face recognition system, characterized in that: include: The camera and the processing terminal connected to it, the camera is used to collect video streams and transmit them to the processing terminal in real time to perform human body detection and face recognition; The modules running on the processing terminal include: A motion detection module, which is used to perform motion detection on the video stream acquired from the camera; A human body detection module, which is used to perform human body detection according to the motion area obtained by motion detection; A feature extraction module, which is used to perform face feature extraction in a local face database; Image fusion module, which is used to perform multi-image fusion in the video stream: The face detection module is used to perform face detection on the detected human body area: The face recognition module is used to perform face recognition after performing face detection.

Images