Face recognition tracking method and equipment based on holder

Abstract

The invention discloses a face recognition tracking method and equipment based on a holder, and relates to the technical field of face recognition. Receiving a figure locking instruction, determining a target figure, obtaining a first video stream, and carrying out figure feature detection on the first video stream to obtain facial features and motion features; if the facial features of the first video stream cannot be recognized, determining a central area according to the motion features and zooming to obtain a second video stream; and performing path prediction on the target person according to the second video stream to obtain a predicted path, and performing pre-regulation and control on the holder according to the predicted path to perform real-time tracking on the target person. Acquiring a video stream, and detecting face and motion features of a person; and if the facial features cannot be recognized, a new video stream is obtained through motion feature zooming, the path of the target person is predicted, and the holder is pre-regulated and controlled, so that real-time accurate tracking is realized, and the tracking efficiency and the system adaptability are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of face recognition, and in particular relates to a face recognition tracking method and equipment based on a pan / tilt platform. Background Art

[0002] The gimbal is a device that can rotate in all directions and stably carry the camera equipment. It realizes flexible adjustment of the camera angle and stable shooting through precise mechanical structure and control system. In the application of face recognition and tracking technology, the gimbal has become an indispensable key component. Through the high-precision control and rapid response capabilities of the gimbal, combined with advanced face recognition algorithms, dynamic tracking and precise positioning of the target person's face are achieved, thereby improving the practicality and efficiency of the face recognition and tracking system.

[0003] Existing PTZ control systems often have problems with poor timeliness and low tracking efficiency when facing complex and ever-changing surveillance scenarios. Traditional tracking methods often rely on single facial feature recognition. When the facial features of the target person cannot be clearly identified due to factors such as occlusion, angle or lighting, the system will find it difficult to continue tracking effectively, causing the PTZ control to become sluggish and unable to quickly adjust the viewing angle to lock the target, resulting in tracking failure or delay. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that when the facial features of the target person cannot be clearly identified due to factors such as occlusion, angle or lighting, the system will find it difficult to continue to track effectively, causing the control of the pan-tilt head to become sluggish and unable to quickly adjust the viewing angle to lock the target, resulting in tracking failure or delay, and a pan-tilt-based face recognition tracking method and device are proposed.

[0005] In a first aspect of the present invention, a face recognition and tracking method based on a pan / tilt platform is first proposed, the method comprising:

[0006] Receiving a person locking instruction and determining a target person, acquiring a first video stream and performing person feature detection on the first video stream to obtain facial features and motion features;

[0007] If the first video stream cannot identify facial features, determining a central area according to the motion features and performing zooming to obtain a second video stream;

[0008] A path prediction is performed on the target person according to the second video stream to obtain a predicted path, and the pan / tilt is pre-adjusted according to the predicted path to track the target person in real time.

[0009] Optionally, performing character feature detection on the first video stream to obtain facial features and behavioral features includes:

[0010] Extracting frames from the first video stream to obtain an image set, and performing character feature detection on images in the image set to determine a facial contour in the target image;

[0011] Facial features in the image set are extracted according to facial contours to obtain a facial comparison map, and the target person is located according to the facial comparison map.

[0012] Optionally, performing character feature detection on the first video stream to obtain facial features and motion features further includes:

[0013] Positioning the facial features of the target person as feature areas, and determining the person outline of the target person according to the feature areas;

[0014] According to the character outline, the first video stream is subjected to motion feature recognition through an optical flow algorithm to obtain gait features and behavior features; the motion features include: gait features and behavior features.

[0015] Optionally, performing path prediction on the target person according to the second video stream to obtain a predicted path includes:

[0016] Acquire a historical video stream data set, extract motion features from the historical video stream data set according to a time series to obtain a historical feature motion feature trajectory, and calculate the optical flow in the historical video stream data set by using the optical flow algorithm to obtain a historical optical flow graph;

[0017] Inputting the historical feature motion feature trajectory and the historical optical flow map into a convolutional neural network for training, and updating the weights of the convolutional neural network model through a back-propagation algorithm to obtain a path prediction model;

[0018] The characteristic motion feature trajectory and the optical flow map of the second video stream are obtained, and the characteristic motion feature trajectory and the optical flow map are input into a path prediction model to obtain a predicted path.

[0019] Optionally, pre-regulating the gimbal according to the predicted path includes:

[0020] Acquire the image size and focal length, determine the center coordinates according to the image size, and determine the offset coordinates of the target person according to the predicted path;

[0021] An offset is calculated according to the center coordinates and the offset coordinates, and a rotation angle is calculated according to the offset and the focal length;

[0022] The rotation angle is converted into a pan / tilt control instruction and sent to the pan / tilt, so that the pan / tilt rotates according to the rotation angle.

[0023] In a second aspect of the present invention, a face recognition and tracking device based on a gimbal is provided, comprising:

[0024] Locking instruction execution module, second video stream acquisition module and PTZ pre-control module:

[0025] The locking instruction execution module is used to receive a person locking instruction and determine a target person, obtain a first video stream and perform person feature detection on the first video stream to obtain facial features and motion features;

[0026] The second video stream acquisition module is used to determine the center area according to the motion features and perform zooming to acquire the second video stream if the first video stream cannot recognize the facial features;

[0027] The pan / tilt pre-control module is used to predict the path of the target person according to the second video stream to obtain a predicted path, and pre-control the pan / tilt according to the predicted path to track the target person in real time.

[0028] Optionally, the locking instruction execution module includes: a facial contour detection module and a facial comparison image positioning module:

[0029] The facial contour detection module is used to extract frames from the first video stream to obtain an image set, and perform character feature detection on the images in the image set to determine the facial contour in the target image;

[0030] The facial comparison image positioning module is used to extract facial features from the image set according to facial contours to obtain a facial comparison image, and to locate the target person according to the facial comparison image.

[0031] Optionally, the locking instruction execution module further includes: a human outline recognition module and a motion feature recognition module:

[0032] The character outline recognition module is used to locate the facial features of the target person as a feature area, and determine the character outline of the target person according to the feature area;

[0033] The motion feature recognition module is used to perform motion feature recognition on the first video stream according to the character outline through an optical flow algorithm to obtain gait features and behavior features; the motion features include: gait features and behavior features.

[0034] Optionally, the device further includes: a historical data analysis module, a model training module and a prediction output module:

[0035] The historical data analysis module is used to obtain a historical video stream data set, extract motion features from the historical video stream data set according to a time series to obtain a historical feature motion feature trajectory, and calculate the optical flow in the historical video stream data set through the optical flow algorithm to obtain a historical optical flow graph;

[0036] The model training module is used to input the historical feature motion feature trajectory and the historical optical flow map into a convolutional neural network for training, and to update the weights of the convolutional neural network model through a back propagation algorithm to obtain a path prediction model;

[0037] The prediction output module is used to obtain the characteristic motion feature trajectory and the optical flow map of the second video stream, and input the characteristic motion feature trajectory and the optical flow map into the path prediction model to obtain the predicted path.

[0038] Optionally, the gimbal pre-control module includes: an offset coordinate acquisition module, a rotation angle calculation module and a control instruction sending module:

[0039] The offset coordinate acquisition module is used to acquire the image size and focal length, determine the center coordinates according to the image size, and determine the offset coordinates of the target person according to the predicted path;

[0040] The rotation angle calculation module is used to calculate the offset according to the center coordinates and the offset coordinates, and calculate the rotation angle according to the offset and the focal length;

[0041] The control instruction sending module is used to convert the rotation angle into a pan / tilt control instruction and send the instruction to the pan / tilt, so that the pan / tilt rotates according to the rotation angle.

[0042] Beneficial effects of the present invention:

[0043] The present invention proposes a face recognition and tracking method based on a pan-tilt head. By receiving a person locking instruction and determining a target person, a first video stream is obtained and the first video stream is subjected to person feature detection to obtain facial features and motion features; if the first video stream cannot identify facial features, the central area is determined according to the motion features and zoomed to obtain a second video stream; the path of the target person is predicted according to the second video stream to obtain a predicted path, and the pan-tilt head is pre-adjusted according to the predicted path to track the target person in real time. The video stream is obtained, and the facial and motion features of the person are detected. If the facial features cannot be identified, a new video stream is obtained by zooming the motion features, the path of the target person is predicted and the pan-tilt head is pre-adjusted to achieve real-time and accurate tracking, and improve the tracking efficiency and system adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention will be further described below in conjunction with the accompanying drawings.

[0045] Figure 1 A flow chart of a face recognition and tracking method based on a pan-tilt platform is provided for an embodiment of the present invention;

[0046] Figure 2 A structural schematic diagram of a face recognition and tracking device based on a pan-tilt platform is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the present invention, the description of "first", "second", etc. is only used for descriptive purposes, and cannot be understood as indicating or implying its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0048] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0049] The embodiment of the present invention provides a face recognition and tracking method based on a pan / tilt system. Figure 1 , Figure 1 A flow chart of a face recognition and tracking method based on a pan-tilt system is provided in an embodiment of the present invention. The method comprises the following steps:

[0050] S101, receiving a person locking instruction and determining a target person, acquiring a first video stream and performing person feature detection on the first video stream to obtain facial features and motion features;

[0051] S102, if the facial features cannot be recognized in the first video stream, determine the central area according to the motion features and zoom in to obtain the second video stream;

[0052] S103, predicting the path of the target person according to the second video stream to obtain a predicted path, and pre-adjusting the pan / tilt according to the predicted path to track the target person in real time.

[0053] Based on the face recognition and tracking method based on the pan-tilt system provided by the embodiment of the present invention, after the target person is determined by receiving the person locking instruction, the first video stream is obtained and the person feature detection is performed to obtain the facial features and motion features. If the facial features cannot be identified, the center area is determined by the motion features and the second video stream is obtained by zooming; the path of the target person is predicted by the second video stream, and the pan-tilt system is pre-regulated according to the prediction results, so as to achieve real-time and accurate tracking of the target person, improve the tracking efficiency, and enhance the adaptability and practicality of the system to complex environments.

[0054] In one implementation, the target device includes a gimbal and a camera, which receive a person locking instruction and determine the target person, that is, the user manually selects the person on the screen as the target person for locking and tracking; after determining the target person, the system will first obtain the video of the target person and perform person feature detection, and the person feature detection includes facial features and motion features. By identifying facial features and motion features, the target person can be accurately locked for real-time tracking, wherein the identification of facial features has a small amount of calculation and has a high accuracy. When the system cannot identify the facial features of the target person (the target person is facing away from the camera), the motion features are used to determine the central area, and the target person is locked through the motion features. When the face cannot be recognized, the person can also be tracked through the motion features.

[0055] In one implementation, when the first video stream cannot clearly identify facial features, the system can determine the central area of ​​the target person based on the motion features (the determination of the central area helps to obtain the motion features later), and zoom to obtain a clearer second video stream, which enhances the system's ability to adapt to complex environments and ensures that it can still track through motion features when facial features cannot be identified. The zoom operation can magnify the target area, improve image clarity, and provide data support for subsequent analysis and identification.

[0056] In one implementation, when a target person is locked through facial recognition or motion characteristics, a second video stream is shot, and the path of the target person is predicted through the second video stream. By predicting the movement path of the person, the gimbal can pre-adjust its position and angle (that is, pre-control) to ensure that the person always remains in the center of the monitoring screen or in a specific area; the frequent adjustment and rotation of the gimbal during the tracking process can be reduced, thereby improving the tracking efficiency.

[0057] In one implementation, the path of the target person is predicted through the second video stream, and the pan / tilt is pre-adjusted according to the predicted path, so that the system can achieve real-time tracking of the target person. This step improves the initiative and accuracy of tracking and avoids tracking delays or losses caused by the movement of the target person. Path prediction can determine the possible direction of the target person in advance, and the pre-adjustment of the pan / tilt can ensure that the camera is always aimed at the target person, achieving continuous and stable tracking; the above method not only improves the tracking efficiency, but also enhances the practicality and reliability of the system.

[0058] In one embodiment, step S101 includes:

[0059] Extract frames from the first video stream to obtain an image set, and perform character feature detection on the images in the image set to determine the facial contour in the target image;

[0060] The facial features in the image set are extracted according to the facial contour to obtain a facial comparison map, and the target person is located according to the facial comparison map.

[0061] In one implementation, by extracting frames from the first video stream to obtain an image set, and performing character feature detection on each image in the image set, the facial contour in the target image can be accurately determined. Facial features are extracted from the facial contour to generate a facial comparison map, which records various features of the target person's face and provides a basis for calculating facial similarity. Using the facial comparison map, the system can efficiently locate the target person and determine whether the person in the image is the same person. This method has significant advantages in processing video stream data, tracking people, and identity verification, and improves the accuracy and efficiency of recognition.

[0062] In one implementation, when the system can continuously detect the face of the target person, it periodically obtains the target person's facial image and compares the facial image with the facial comparison map, that is, the shape, size, position and relative relationship between the eyes, nose, mouth and other facial features. Compare the similarity of these features between the facial image and the facial comparison map. Similarity calculation methods include: cosine similarity, Euclidean distance, etc.; when determining whether they are the same person, if the similarity of the two face images is greater than the similarity threshold, they are determined to be the same person; if the similarity is less than the similarity threshold, they are determined to be not the same person.

[0063] In one implementation, the face of the target person is used as the face tracking technology to track the face as a whole coordinate. This is done by using a face detection algorithm in consecutive video frames to identify the face area, then extracting facial feature points and generating a facial comparison map or feature vector, and then calculating the similarity of the facial comparison maps or vectors between different frames to determine the position and motion trajectory of the face, thereby achieving real-time tracking of the face as a whole coordinate.

[0064] In one embodiment, step S101 further includes:

[0065] The facial features of the target person are used as feature areas for positioning, and the outline of the target person is determined according to the feature areas;

[0066] According to the outline of the person, the motion feature recognition of the first video stream is performed through the optical flow algorithm to obtain the gait feature and the behavior feature; the motion feature includes: gait feature and behavior feature.

[0067] In one implementation, the human figure outline refers to the entire outline of the human body composed of the outer contours of the head, body and legs; the optical flow algorithm can capture the dynamic changes of the human body during walking or activities, and has a high accuracy in detecting gait characteristics and behavioral characteristics; the optical flow algorithm does not need to know any information about the scene in advance to detect moving objects, and has a certain degree of robustness.

[0068] In one implementation, by using the facial features of the target person as the feature area for precise positioning, the position of the target person in the video can be quickly determined, and a clear outline of the person can be drawn accordingly. The outline of the person can provide a solid foundation for in-depth identification of the gait features (e.g., walking speed, step size) and behavioral features (e.g., waving, jumping, etc.) of the target person in the first video stream using the optical flow algorithm, thereby ensuring the accuracy and comprehensiveness of the motion feature analysis and effectively improving the efficiency and quality of video analysis.

[0069] In one embodiment, performing path prediction on the target person according to the second video stream to obtain a predicted path includes:

[0070] Obtain a historical video stream data set, extract motion features from the historical video stream data set according to the time series to obtain a historical feature motion feature trajectory, and calculate the optical flow in the historical video stream data set through an optical flow algorithm to obtain a historical optical flow graph;

[0071] The historical feature motion feature trajectory and historical optical flow map are input into the convolutional neural network for training, and the weight of the convolutional neural network model is updated through the back-propagation algorithm to obtain the path prediction model;

[0072] The second video stream is obtained to perform feature motion feature trajectory and optical flow map, and the feature motion feature trajectory and optical flow map are input into the path prediction model to obtain the predicted path.

[0073] In one implementation, by collecting a large amount of historical video stream data sets, the motion patterns and behavioral characteristics of people in different scenes can be fully captured. By extracting motion features from the historical video stream data sets in time series, it is possible to generate historical feature motion feature trajectories, which is helpful for analyzing the motion patterns of the target person; the optical flow algorithm is used to calculate the optical flow in the historical video stream data sets to obtain a historical optical flow map, which can reflect the motion changes of the target person between consecutive frames; optical flow is a two-dimensional vector field that describes the speed and direction of pixel movement in an image, and the optical flow map is a visual representation of this vector field. The optical flow map intuitively displays the motion of objects in the image by drawing optical flow vectors;

[0074] In one implementation, the historical feature motion feature trajectory and historical optical flow map are input into the convolutional neural network for training, which can fully utilize the advantages of the convolutional neural network in processing image and video data, automatically learn and extract deep features related to the target person's motion path prediction. By continuously updating the weights of the convolutional neural network model through the back propagation algorithm, the model structure can be gradually optimized to improve the accuracy and robustness of path prediction.

[0075] In one implementation, after acquiring the second video stream, the characteristic motion feature trajectory and optical flow map are extracted, and these features are input into the path prediction model to obtain the predicted path of the target person. The advantage of this step is that it can realize the real-time path prediction of the target person, and provide timely and accurate information support for subsequent tracking, monitoring or early warning tasks; because the path prediction model is trained based on a large amount of historical data, it has high generalization ability and adaptability, and can meet the path prediction needs in different scenarios.

[0076] In one embodiment, step S103 includes:

[0077] Obtain the image size and focal length, determine the center coordinates based on the image size, and determine the offset coordinates of the target person based on the predicted path;

[0078] The offset is calculated based on the center coordinates and the offset coordinates, and the rotation angle is calculated based on the offset and the focal length;

[0079] The rotation angle is converted into a gimbal control command and sent to the gimbal, so that the gimbal rotates according to the rotation angle.

[0080] In one implementation, the offset coordinates are calculated by taking the head of the target person as the positioning area.

[0081] In one implementation, for example: center coordinates (x0, y0), offset coordinates (x1, y1), horizontal offset at this time: Δx = x1-x0, vertical offset: Δy = y1-y0, if Δx is positive, it means the target object is on the right side of the image center; if Δx is negative, it means the target object is on the left side of the image center; if Δy is positive, it means the target object is below the image center; if Δy is negative, it means the target object is above the image center; horizontal rotation angle (in radians): Vertical rotation angle (in radians): Convert radians to degrees to obtain the rotation angle of the gimbal. For example, the focal length of the camera is f = 1000mm, the center coordinates are (x0, y0) = (320, 240), and the coordinates of the center point of the target object in the image are (x1, y1) = (350, 220). The horizontal offset is 350-320 = 30, and the target object is 30 pixels to the right of the center point of the image. The vertical offset is 220-240 = -20, and the target object is 20 pixels above the center point of the image. The horizontal rotation angle is 30 / 1000 = 0.03, and the vertical rotation angle is -20 / 1000 = -0.02. Convert radians to degrees, which is: θ x ≈0.03×57.3 ° =1.72 ° ,θ y ≈-0.02×57.3 ° =-1.15 ° , the camera needs to perform a horizontal counterclockwise rotation of about 1.72° and a vertical clockwise rotation of about 1.15° around its optical axis.

[0082] In one implementation, the calculated rotation angle is converted into an instruction format recognizable by the gimbal (e.g., PWM signal or serial port instruction), and then a control instruction is sent to the gimbal. After receiving the instruction, the gimbal drives the internal motor to perform the rotation operation. During the rotation process, the system needs to obtain the current position information of the gimbal in real time, and make corresponding adjustments if the actual position deviates from the expected position.

[0083] Based on the same inventive concept, the present invention also provides a face recognition and tracking device based on a pan-tilt platform. Figure 2 , Figure 2 A schematic diagram of the structure of a face recognition and tracking device based on a pan-tilt structure provided in an embodiment of the present invention includes: a locking instruction execution module, a second video stream acquisition module, and a pan-tilt structure pre-control module:

[0084] A locking instruction execution module is used to receive a person locking instruction and determine a target person, obtain a first video stream, and perform person feature detection on the first video stream to obtain facial features and motion features;

[0085] A second video stream acquisition module, used to determine the center area according to the motion features and perform zooming to acquire the second video stream if the first video stream cannot recognize the facial features;

[0086] The pan / tilt pre-control module is used to predict the path of the target person according to the second video stream to obtain a predicted path, and pre-control the pan / tilt according to the predicted path to track the target person in real time.

[0087] The face recognition and tracking device based on the pan-tilt system provided by the embodiment of the present invention determines the target person by receiving the person locking command, obtains the first video stream and performs person feature detection to obtain facial features and motion features. If the facial features cannot be identified, the center area is determined by the motion features and the second video stream is obtained by zooming; the path of the target person is predicted by the second video stream, and the pan-tilt system is pre-regulated according to the prediction results, thereby achieving real-time and accurate tracking of the target person, improving the tracking efficiency, and enhancing the system's adaptability and practicality to complex environments.

[0088] In one embodiment, the locking instruction execution module includes: a facial contour detection module and a facial comparison image positioning module:

[0089] A facial contour detection module is used to extract frames from the first video stream to obtain an image set, and perform character feature detection on the images in the image set to determine the facial contour in the target image;

[0090] The facial comparison map positioning module is used to extract facial features from the image set according to the facial contour to obtain a facial comparison map, and locate the target person according to the facial comparison map.

[0091] In one embodiment, the locking instruction execution module further includes: a human outline recognition module and a motion feature recognition module:

[0092] A character outline recognition module is used to locate the facial features of the target person as a feature area and determine the character outline of the target person based on the feature area;

[0093] The motion feature recognition module is used to perform motion feature recognition on the first video stream according to the character outline through an optical flow algorithm to obtain gait features and behavior features; the motion features include: gait features and behavior features.

[0094] In one embodiment, the device further includes: a historical data analysis module, a model training module and a prediction output module:

[0095] The historical data analysis module is used to obtain a historical video stream data set, extract motion features from the historical video stream data set according to the time series to obtain a historical feature motion feature trajectory, and calculate the optical flow in the historical video stream data set through an optical flow algorithm to obtain a historical optical flow map;

[0096] The model training module is used to input the historical feature motion feature trajectory and the historical optical flow map into the convolutional neural network for training, and to update the weights of the convolutional neural network model through the back propagation algorithm to obtain the path prediction model;

[0097] The prediction output module is used to obtain the characteristic motion feature trajectory and the optical flow map of the second video stream, and input the characteristic motion feature trajectory and the optical flow map into the path prediction model to obtain the predicted path.

[0098] In one embodiment, the gimbal pre-control module includes: an offset coordinate acquisition module, a rotation angle calculation module and a control instruction sending module:

[0099] An offset coordinate acquisition module is used to obtain the image size and focal length, determine the center coordinates according to the image size, and determine the offset coordinates of the target person according to the predicted path;

[0100] A rotation angle calculation module is used to calculate the offset according to the center coordinates and the offset coordinates, and to calculate the rotation angle according to the offset and the focal length;

[0101] The control instruction sending module is used to convert the rotation angle into a pan / tilt control instruction and send it to the pan / tilt so that the pan / tilt rotates according to the rotation angle.

[0102] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention. The user information and data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A face recognition and tracking method based on a pan / tilt system, characterized in that: The method comprises: Receiving a person locking instruction and determining a target person, acquiring a first video stream and performing person feature detection on the first video stream to obtain facial features and motion features; If the first video stream cannot identify facial features, determining a central area according to the motion features and performing zooming to obtain a second video stream; A path prediction is performed on the target person according to the second video stream to obtain a predicted path, and the pan / tilt is pre-adjusted according to the predicted path to track the target person in real time.

2. The face recognition and tracking method based on a pan-tilt platform according to claim 1, characterized in that: Performing character feature detection on the first video stream to obtain facial features and behavioral features includes: Extracting frames from the first video stream to obtain an image set, and performing character feature detection on images in the image set to determine a facial contour in the target image; Facial features in the image set are extracted according to facial contours to obtain a facial comparison map, and the target person is located according to the facial comparison map.

3. The face recognition and tracking method based on a pan-tilt platform according to claim 1, characterized in that: Performing character feature detection on the first video stream to obtain facial features and motion features also includes: Positioning the facial features of the target person as feature areas, and determining the person outline of the target person according to the feature areas; According to the character outline, the first video stream is subjected to motion feature recognition through an optical flow algorithm to obtain gait features and behavior features; the motion features include: gait features and behavior features.

4. The face recognition and tracking method based on a pan-tilt platform according to claim 1, characterized in that: Predicting the path of the target person according to the second video stream to obtain a predicted path includes: Acquire a historical video stream data set, extract motion features from the historical video stream data set according to a time series to obtain a historical feature motion feature trajectory, and calculate the optical flow in the historical video stream data set by using the optical flow algorithm to obtain a historical optical flow graph; Inputting the historical feature motion feature trajectory and the historical optical flow map into a convolutional neural network for training, and updating the weights of the convolutional neural network model through a back-propagation algorithm to obtain a path prediction model; The characteristic motion feature trajectory and the optical flow map of the second video stream are obtained, and the characteristic motion feature trajectory and the optical flow map are input into a path prediction model to obtain a predicted path.

5. The face recognition and tracking method based on a pan-tilt platform according to claim 1, characterized in that: Pre-controlling the gimbal according to the predicted path includes: Acquire the image size and focal length, determine the center coordinates according to the image size, and determine the offset coordinates of the target person according to the predicted path; An offset is calculated according to the center coordinates and the offset coordinates, and a rotation angle is calculated according to the offset and the focal length; The rotation angle is converted into a pan / tilt control instruction and sent to the pan / tilt, so that the pan / tilt rotates according to the rotation angle.

6. The face recognition and tracking device based on the pan-tilt system is characterized by: The device comprises: a locking instruction execution module, a second video stream acquisition module and a pan / tilt pre-regulation module: The locking instruction execution module is used to receive a person locking instruction and determine a target person, obtain a first video stream and perform person feature detection on the first video stream to obtain facial features and motion features; The second video stream acquisition module is used to determine the center area according to the motion features and perform zooming to acquire the second video stream if the first video stream cannot recognize the facial features; The pan / tilt pre-control module is used to predict the path of the target person according to the second video stream to obtain a predicted path, and pre-control the pan / tilt according to the predicted path to track the target person in real time.

7. The face recognition and tracking device based on a pan-tilt platform according to claim 6, characterized in that: The locking instruction execution module includes: a facial contour detection module and a facial comparison image positioning module: The facial contour detection module is used to extract frames from the first video stream to obtain an image set, and perform character feature detection on the images in the image set to determine the facial contour in the target image; The facial comparison image positioning module is used to extract facial features from the image set according to facial contours to obtain a facial comparison image, and to locate the target person according to the facial comparison image.

8. The face recognition and tracking device based on a pan-tilt platform according to claim 6, characterized in that: The locking instruction execution module also includes: a human outline recognition module and a motion feature recognition module: The character outline recognition module is used to locate the facial features of the target person as a feature area, and determine the character outline of the target person according to the feature area; The motion feature recognition module is used to perform motion feature recognition on the first video stream according to the character outline through an optical flow algorithm to obtain gait features and behavior features; the motion features include: gait features and behavior features.

9. The face recognition and tracking device based on a pan-tilt platform according to claim 6, characterized in that: The device also includes: a historical data analysis module, a model training module and a prediction output module: The historical data analysis module is used to obtain a historical video stream data set, extract motion features from the historical video stream data set according to a time series to obtain a historical feature motion feature trajectory, and calculate the optical flow in the historical video stream data set through the optical flow algorithm to obtain a historical optical flow graph; The model training module is used to input the historical feature motion feature trajectory and the historical optical flow map into a convolutional neural network for training, and to update the weights of the convolutional neural network model through a back propagation algorithm to obtain a path prediction model; The prediction output module is used to obtain the characteristic motion feature trajectory and the optical flow map of the second video stream, and input the characteristic motion feature trajectory and the optical flow map into the path prediction model to obtain the predicted path.

10. The face recognition and tracking device based on a pan-tilt platform according to claim 6, characterized in that: The pan / tilt pre-control module includes: an offset coordinate acquisition module, a rotation angle calculation module and a control instruction sending module: The offset coordinate acquisition module is used to acquire the image size and focal length, determine the center coordinates according to the image size, and determine the offset coordinates of the target person according to the predicted path; The rotation angle calculation module is used to calculate the offset according to the center coordinates and the offset coordinates, and calculate the rotation angle according to the offset and the focal length; The control instruction sending module is used to convert the rotation angle into a pan / tilt control instruction and send the instruction to the pan / tilt, so that the pan / tilt rotates according to the rotation angle.

Images