Camera and face information collection method based on camera face recognition

Abstract

A camera and a method for collecting face information based on camera face recognition, the method comprising: a main controller collects video data and caches it to a main memory, the main controller transmits segmented video data in the main memory to an image computing chip, the main controller receives face information and quality parameters of face images in the segmented video data analyzed by the image computing chip, the main controller captures face photos and stores and uploads them according to the face information and the quality parameters of the face images reaching preset indicators. The present invention undertakes simple but large amounts of face recognition work by adding an image computing chip, and uses a general processor to obtain and upload face photos according to the results of face recognition, which can replace the bandwidth waste caused by directly uploading the original video code stream for face recognition to a server, as well as the waste of server hardware resources, and is suitable for simultaneously monitoring and tracking more faces to improve monitoring effects.

Description

[0001] The present invention is a divisional application of the invention patent application with application number 201710557708.0 filed on July 10, 2017 and invention name “A method for collecting facial information based on camera face recognition”. Technical Field

[0002] The present invention relates to a method for collecting face information, and in particular to a camera and a method for collecting face information based on camera face recognition. Background Art

[0003] In today's society, for security reasons, video surveillance equipment is increasingly used in safe city monitoring, as well as monitoring of various places such as residential areas and factories. In most cases, the monitoring process is aimed at identifying people, and with the maturity of face recognition technology, it has become possible for artificial intelligence to automatically digest faces in surveillance video data. As surveillance equipment becomes more and more popular, and face recognition has higher requirements for video quality, the corresponding surveillance data is also increasing.

[0004] At present, in most cases, face recognition is used in monitoring systems by sending video data from the camera to the backend server, which then relies on its powerful hardware and complex software for intelligent analysis. The data throughput of the monitoring network is amazing; with more and more cameras, the storage capacity and computing power of the backend server are also put to the test, which greatly increases the cost; in addition, the monitoring environment is complex and diverse, and the parameter settings of the camera cannot be guaranteed to be intact in real time, which will lead to the inability to recognize and collect portraits. Using a remote server to analyze the image to determine the image quality and return the camera parameter modification requirements, and the camera will then adjust the camera to take pictures, which will put great pressure on the hardware cost and will not be enough time.

[0005] In recent years, algorithm improvements and hardware performance enhancements have made it possible for cameras to recognize faces. Traditionally, general-purpose processors based on common mobile architectures such as Cortex or X86-based Atom are not suitable for performing video processing and hardware management while performing large-scale simple calculations required for artificial intelligence. Summary of the invention

[0006] In order to overcome the problem that the existing monitoring system has high network overhead and seriously uneven hardware overhead distribution, the purpose of the present invention is to provide a face information collection method based on camera face recognition.

[0007] The method provided by the present invention is as follows:

[0008] A method for collecting facial information based on camera facial recognition, the method comprising: in the process of collecting video data by a camera, facial information recognition is performed on each frame of video data to obtain facial images of all persons in the current frame and quality parameters of the corresponding facial images, and finally a person whose facial information and quality parameters of the facial images of all persons in the current frame meet preset standards is determined, and a facial photo of the person is captured.

[0009] The facial information recognition is performed on each frame of video data to obtain the facial images of all the characters in the current frame and the corresponding quality parameters of the facial images. Specifically, the camera picks up the facial information according to the brightness distribution in one of the frames of the video data, and then determines the quality parameters of the facial image according to the brightness, clarity and the degree of matching of facial features under the front face in the facial information.

[0010] After the camera picks up the facial information according to the brightness distribution in one frame of the video data, it also includes: the camera determines whether the current facial image is a new person based on the comparison between the current facial information and the facial information picked up in several adjacent frames. If it is determined to be a new person, a facial ID is assigned to the current facial image; otherwise, a corresponding relationship is established between the current facial image and the existing facial ID.

[0011] The method of determining a person whose facial information and quality parameters of facial images among all persons in the current frame meet preset standards, and capturing a facial photo of the person, specifically includes: when it is determined that the person is a new person and the quality parameter of the facial image is greater than or equal to a preset threshold, the camera captures the facial photo and stores it.

[0012] After performing facial information recognition on each frame of video data to obtain facial images of all characters in the current frame and the corresponding facial image quality parameters, the method also includes: when it is determined that the person is a new person but the quality parameter of the facial image is less than a preset threshold, the camera adjusts the camera parameters according to the facial information, discards the facial image and continues to judge the quality parameters of the facial images generated by subsequent video data until the quality parameter of the facial image is greater than or equal to the preset threshold.

[0013] Determine a person whose facial information and quality parameters of the facial image meet the preset standard among all the persons in the current frame, and capture a facial photo of the person. Specifically, when it is determined that there is an original face and the quality parameter of the facial image is greater than or equal to a preset threshold, the camera determines that the quality parameter of the facial image or the degree of face frontality is greater than the original facial photo, and finally captures the facial photo and replaces the original facial photo.

[0014] After performing facial information recognition on each frame of video data to obtain facial images of all characters in the current frame and the corresponding quality parameters of the facial images, the method also includes: when it is determined that there is a face but the quality parameter of the facial image is less than a preset threshold, the camera adjusts the camera parameters according to the facial information, discards the facial image and continues to judge the quality parameters of the facial images generated by subsequent video data until the quality parameter of the facial image is greater than or equal to the preset threshold.

[0015] When the quality parameter of the face image is greater than or equal to a preset threshold, the method also includes: the camera records the exposure parameter when the quality parameter of the face image is greater than or equal to the threshold, and then determines that the camera has not detected face information within a preset time, and adjusts the camera according to the exposure parameter recorded when the most recent face image parameter is greater than or equal to the preset threshold.

[0016] Determine a person whose facial information and quality parameters of the facial image meet the preset standards among all the people in the current frame, and capture a facial photo of the person. At the same time, the method also includes: the camera detects that the storage capacity is less than the warning value, and then deletes some facial photos. Specifically, the camera determines the priority of the face ID according to the access status, upload status and tracking status of the facial photos corresponding to the face ID and makes real-time adjustments. At the same time, the facial photos corresponding to the face ID are deleted according to the priority sorting and the face ID is cancelled. Finally, the deletion work is stopped when it is detected that the storage capacity is higher than the preset threshold.

[0017] According to the face information and the quality parameters of the face image reaching the preset indicators, the face photo is captured and uploaded. Specifically, the camera selects the face photo capture method according to the requirements of the remote server.

[0018] According to the face information and the quality parameters of the face image reaching the preset indicators, the face photo is captured and uploaded. Specifically, the camera selects the face photo uploading strategy according to the requirements of the remote server.

[0019] Compared with the prior art, the present invention adds an image computing chip to undertake simple but large amounts of face recognition work, and uses a general-purpose processor to obtain and upload face photos according to the results of face recognition, replacing the previous waste of bandwidth and server hardware resources caused by directly uploading the original video stream for face recognition to the server. It is suitable for simultaneously monitoring and tracking more faces to improve monitoring effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a method for collecting face information based on camera face recognition;

[0021] Figure 2 The present invention is a flowchart of a specific implementation method of a facial information collection method based on camera facial recognition. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0023] An embodiment of the present invention provides a face recognition method based on a camera, the steps comprising: a camera collects video data, while collecting data, performs face information recognition on each frame of video data to obtain face information of a face image and quality parameters of the face image, determines that the face image meets preset indicators, and takes a photo of the face that meets the preset indicators.

[0024] To implement the method, the camera includes a main controller for controlling camera operation, image encoding and decoding, communication, and the corresponding main memory, and also includes an image computing chip for using a face recognition algorithm to perform face image recognition and its corresponding memory and other external components. The camera can also be equipped with an external memory to store the captured photos to reduce the storage pressure of the main memory. On this basis, the specific steps of the method include:

[0025] Step 100: The main controller collects the video data of the sensor and caches it in the main memory.

[0026] Step 200: The main controller transmits the segmented video data to the image operation chip.

[0027] Step 300: The image computing chip analyzes the human faces in the segmented video data, recognizes the human faces and returns the human face information to the main controller.

[0028] Step 301: The image computing chip picks up the face information according to the brightness distribution of the face image in a certain frame in the received segmented video data.

[0029] Step 302: The image computing chip compares the facial information of the current detection with the facial information of the facial images detected in several adjacent frames to determine the similarity between the two, and determines whether the currently detected face is a new person based on the similarity and a set threshold value. Based on the judgment result, the facial information of the facial images that have appeared is matched with the corresponding facial ID, or a new facial ID is set for the new person.

[0030] Step 303: The image computing chip gives the quality parameters of the current face image according to the brightness and clarity of the current face image and the degree of matching with the facial features under the front face.

[0031] Step 304: Transmit the facial image and quality parameter information to the main controller.

[0032] Step 400: The main controller adjusts the environmental peripherals according to the facial information.

[0033] Step 401: The main controller determines whether the exposure parameters need to be adjusted and the required adjustment step length according to the quality parameters of the face image and the current brightness of the face.

[0034] Step 402: When the quality parameter of the face image reaches a set threshold, the main controller records the exposure parameters for a period of time.

[0035] Step 403: When no face is detected, the main controller sets the camera using the recorded exposure parameters.

[0036] Step 500: The main controller captures the face and stores it in the main memory or external memory.

[0037] Step 501: According to the face ID, when the quality parameter of the received face image of the new person is higher than the preset threshold, the main controller controls the camera to capture and store the face photo.

[0038] Specifically, the main controller controls the camera to capture a face image, a body image or an original image, or a free combination of the three, according to the settings.

[0039] Step 502: Based on the face ID, when the face image information of a saved face photo is received, the main controller determines that the current face image is not a frontal face, the current face image is larger than the frontal face area of ​​the saved face photo, and the quality parameter of the current face image is higher than the threshold, or the quality parameter of the current face image is higher than the saved face photo, then the face photo is captured and stored to replace the original face photo and the quality parameter of the face photo is recorded.

[0040] Step 503: Based on the face ID, when the face image information of the saved face photo is received, the main controller determines that when the saved face photo is a frontal face, if the quality parameter of the current face image is greater than that of the saved face, it will capture the face photo and store it to replace the original face photo and record the quality parameter of the face photo.

[0041] Step 600: Upload and clean up face photos.

[0042] Step 601: When the quality of the face photo exceeds a set value, the main controller uploads the face photo.

[0043] Specifically, the main controller preferentially uploads face photos whose photo quality parameters meet the upload standard and whose time intervals with the current photos are shorter, and face photos with higher quality parameters among all face photos.

[0044] Specifically, the main controller controls the camera to perform three uploading methods according to the settings, namely real-time uploading, uploading after the person leaves, and interval uploading. Real-time uploading is to upload the face photo immediately after the face photo changes, uploading after the person leaves is to upload the best photo after the person leaves the detection area, and interval uploading is to upload the best photo regularly when the face image corresponding to the face ID appears in the detection area.

[0045] Step 602: When the usage capacity of the main memory or external memory reaches the set value, the main controller sets the priority for each face ID according to the current access status, current upload status, current tracking status and last tracking time of the face photo corresponding to the face ID.

[0046] Specifically, the face photo currently being accessed has the highest priority, the face photo that has not yet been uploaded has a higher priority, the face photo currently being tracked has a higher priority, and the face photo with a longer tracking time interval with the current time has a lower priority.

[0047] Step 603: Based on the priority of the face ID, the face photos with lower priority are deleted and covered first, and the face ID corresponding to the covered face photo is cancelled.

[0048] Embodiment 1:

[0049] The camera records the detection area and stores it in the memory, and the ARM processor transmits the segmented video data to the FPGA computing chip through the BT1120 standard video data interface.

[0050] The FPGA computing chip caches the segmented video data in the memory of the FPGA computing chip and analyzes the segmented data frame by frame, picks up the brightness distribution of the face area in the image, captures the face image and records the face information. The FPGA computing chip compares the currently recorded face information with the face information recorded in the most recent frames to determine whether the face is consistent with the previously recorded face. If the face is determined to be a new face, a new face ID is assigned. If the face is determined to be an existing face, the currently recorded face information is associated with the face ID corresponding to the original face.

[0051] The FPGA computing chip gives the quality parameters of the current face image according to the brightness, clarity and matching degree of the facial features under the front face, and transmits the quality parameters and facial information to the ARM processor.

[0052] The ARM processor determines whether the exposure parameters and other camera parameters need to be adjusted based on the facial brightness and quality parameters of the current facial information, confirms the adjustment step, controls the camera to complete the modification of the shooting parameters, and monitors the facial information in the subsequent facial images transmitted by the FPGA computing chip until the brightness is sufficient and the quality parameters meet the standards.

[0053] Under the premise that the face image quality parameters reach the preset threshold, the ARM processor captures the face photo and stores it in the memory.

[0054] Specifically, the ARM processor extracts the face image in the cache as a face photo, and adds description information to the file, recording the shooting time and face ID.

[0055] The ARM processor uploads the facial photo stored in the memory to the remote server.

[0056] The advantages of this solution are: using face ID to determine the acquisition of facial images, facilitating accurate control and recording of the collection of facial information, and there is no need to acquire and upload a large number of images, greatly saving hardware costs.

[0057] Acquiring the photo shooting situation according to the facial information and quality parameters of the face photo and adjusting the camera parameters in real time is conducive to taking photos with higher quality.

[0058] Embodiment 2:

[0059] On the basis of the first embodiment, when the FPGA computing chip cannot detect any face, the ARM processor traces back to the most recent face photo that meets the standard and adjusts the exposure parameters for setting the camera.

[0060] This solution makes it easy to set the exposure state of the camera to the most reasonable state when there is no face in the detection area, so as to adjust to the appropriate exposure position more quickly after the face is detected again.

[0061] Embodiment three:

[0062] Based on the first embodiment, the ARM controller determines the face ID after receiving the face image information. When the face with the face ID has been captured and a photo is stored, the quality parameters and the frontal face area of ​​the corresponding stored face photo and the quality parameters and face area contained in the current face image information are compared. If the quality parameters are higher or the frontal face area is larger, the face photo is captured again and the original face photo is replaced.

[0063] If the stored face photo is a frontal face, only the quality parameter of the face photo and the quality parameter of the current face image are compared. If the quality parameter of the current face image is higher than the quality parameter of the stored face photo, a new face photo is captured and stored to replace the old face photo.

[0064] On average, photos with larger frontal face areas have higher quality parameters of face images, so this solution is suitable for screening out images and photos that can better identify people. This solution facilitates quick comparison and obtaining higher quality face photos, and facilitates the preservation of better quality face photos.

[0065] Embodiment 4:

[0066] On the basis of the third embodiment, the ARM processor also receives instructions from the remote server and adjusts the capture rules according to the requirements of the remote server.

[0067] Specifically, it includes but is not limited to the following picture styles:

[0068] The original image of the current scene, the image of the body to which the tracked face belongs, or the image of the face to which the tracked face belongs, or any combination of the three;

[0069] It also includes but is not limited to the following capture strategies:

[0070] When the quality parameter of the face image reaches the threshold, it is immediately captured and uploaded;

[0071] When the person to whom the face belongs leaves the detection area, the best face photo corresponding to the face is uploaded;

[0072] When the person to whom the face belongs appears in the detection area, the optimal face image is transmitted regularly.

[0073] This solution makes the camera more adaptable to daily monitoring tasks and retains more reasonable data when there is no video data for archiving.

[0074] Embodiment five:

[0075] On the basis of the first embodiment, a TF card or a ROM memory is added as an external storage connected to the ARM processor, and the ARM processor stores the face photo in the TF card or the ROM memory.

[0076] This solution optimizes the storage environment for facial photos to prevent the situation where photos are discarded due to untimely uploading or newly captured photos cannot be stored.

[0077] Embodiment six:

[0078] Based on the first or fifth embodiment, when the ARM processor detects that the storage space of the internal memory or the external memory is less than a preset threshold, the stored face photos are prioritized.

[0079] Specifically, based on the face ID, determine whether the face photo corresponding to the face ID is being accessed, set a high priority for the face photo being accessed, determine the upload status of the face photo corresponding to the face ID, set a low priority for the face photo that has been uploaded, determine the tracking status of the face image corresponding to the face ID, set a high priority for the face ID corresponding to the face image being tracked, and set the priorities from high to low in sequence from near to far according to the last tracking time.

[0080] The ARM processor deletes the face photo corresponding to the face ID with a lower priority in the memory or external storage according to the real-time changing priority information, and cancels the face ID. When the storage space in the memory or external storage exceeds a preset threshold, the deletion work stops.

[0081] This solution ensures more effective preservation of high-value facial photos under extremely poor network conditions while not hindering the normal progress of monitoring work.

[0082] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A camera, characterized in that: It includes sensors, main controllers and image computing chips, among which: The sensor is used to collect video data; The image computing chip is used to perform face information recognition on each frame of video data to obtain face information of face images of all characters in the current frame and quality parameters of the corresponding face images, and transmit the face information and the quality parameters to the main controller; The main controller is used to control the camera to capture a facial photo of a person when the facial information and quality parameters of the facial images of all the people in the current frame meet the preset standards; The main controller is also used to upload the face photo when the quality of the face photo exceeds a set value; The image computing chip is specifically used for: The face information is picked up according to the brightness distribution in one frame of the video data, and then the quality parameters of the face image are determined according to the brightness, clarity and matching degree of the face features under the front face in the face information; The image computing chip is also used for: After the image computing chip picks up the face information according to the brightness distribution in one frame of the video data, it determines whether the current face image is a new person according to the comparison between the current face information and the face information picked up in several adjacent frames. If it is determined to be a new person, a face ID is assigned to the current face image. Otherwise, a corresponding relationship is established between the current face image and the existing face ID. The main controller is specifically used for: When it is determined that the person is a new person and the quality parameter of the face image is greater than or equal to a preset threshold, controlling the camera to capture and store a face photo of the person; The main controller is specifically used for: When it is determined that there is an original face and the quality parameter of the face image is greater than or equal to a preset threshold, the quality parameter of the face image or the degree of face frontality is judged to be greater than the original face photo, and finally the camera is controlled to capture the face photo and replace the original face photo.

2. The camera according to claim 1, characterized in that: After the image computing chip performs facial information recognition on each frame of video data to obtain facial images of all characters in the current frame and the corresponding facial image quality parameters, the main controller is used to, when it is determined that a new person is present but the facial image quality parameter is less than a preset threshold, adjust the camera parameters according to the facial information, discard the facial image and continue to judge the quality parameters of facial images generated by subsequent video data until the facial image quality parameter is greater than or equal to the preset threshold.

3. The camera according to claim 1, characterized in that: After the image computing chip performs facial information recognition on each frame of video data to obtain the facial images of all characters in the current frame and the corresponding quality parameters of the facial images, the main controller is also used to: when it is determined that there is a face but the quality parameter of the facial image is less than a preset threshold, adjust the camera parameters according to the facial information, discard the facial image and continue to judge the quality parameters of the facial images generated by subsequent video data until the quality parameter of the facial image is greater than or equal to the preset threshold.

4. The camera according to claim 1 or 3, characterized in that: The main controller is also used for: When the quality parameter of the face image is greater than or equal to the preset threshold, the exposure parameter when the quality parameter of the face image is greater than or equal to the threshold is recorded. If it is then determined that the camera has not detected face information within the preset time, the camera is adjusted according to the exposure parameter recorded when the most recent face image parameter was greater than or equal to the preset threshold.

5. The camera according to claim 1, characterized in that: The main controller determines a person whose facial information and quality parameters of the facial images of all the persons in the current frame meet the preset standards, and captures a facial photo of the person. The main controller is also used to detect that the storage capacity is less than the warning value, and then delete some facial photos. Specifically, the main controller determines the priority of the face ID according to the access status, upload status and tracking status of the facial photos corresponding to the face ID and makes real-time adjustments. At the same time, the main controller deletes the facial photos corresponding to the face ID according to the priority sorting and cancels the face ID. Finally, the deletion work is stopped when it is detected that the storage capacity is higher than the preset threshold.

6. The camera according to claim 1, characterized in that: The main controller captures and uploads a face photo according to the face information and the quality parameters of the face image reaching the preset indicators. Specifically, the main controller selects a face photo uploading strategy according to the requirements of the remote server.

7. A method for collecting face information based on camera face recognition, characterized in that: include: Get video data; Performing face information recognition on each frame of video data to obtain face information of face images of all characters in the current frame and quality parameters of the corresponding face images; When determining a person whose facial information and quality parameters of facial images among all the persons in the current frame meet the preset standards, controlling the camera to capture a facial photo of the person; When the quality of the face photo exceeds a set value, uploading the face photo; Performing face recognition on each frame of video data to obtain face information of all characters in the current frame and the quality parameters of the corresponding face images, including: The face information is picked up according to the brightness distribution in one frame of the video data, and then the quality parameters of the face image are determined according to the brightness, clarity and matching degree of the face features under the front face in the face information; After picking up the face information according to the brightness distribution in one frame of the video data, it also includes: According to the comparison between the current face information and the face information picked up in several adjacent frames, it is judged whether the current face image is a new person. If it is determined to be a new person, a face ID is assigned to the current face image. Otherwise, a corresponding relationship is established between the current face image and the existing face ID; When determining a person whose facial information and quality parameters of facial images among all the persons in the current frame meet the preset standard, controlling the camera to capture a facial photo of the person, including: When it is determined that the person is a new person and the quality parameter of the face image is greater than or equal to a preset threshold, controlling the camera to capture and store a face photo of the person; When determining a person whose facial information and quality parameters of facial images among all the persons in the current frame meet the preset standard, controlling the camera to capture a facial photo of the person, including: When it is determined that there is an original face and the quality parameter of the face image is greater than or equal to a preset threshold, the quality parameter of the face image or the degree of face frontality is judged to be greater than the original face photo, and finally the camera is controlled to capture the face photo and replace the original face photo.

8. The method according to claim 7, characterized in that: After performing face information recognition on each frame of video data to obtain face information of face images of all characters in the current frame and quality parameters of the corresponding face images, the method further includes: When it is determined that the person is a new person but the quality parameter of the facial image is less than a preset threshold, the camera parameters are adjusted according to the facial information, and the facial image is discarded and the quality parameters of the facial images generated by subsequent video data are continued to be judged until the quality parameter of the facial image is greater than or equal to the preset threshold.

9. The method according to claim 7, characterized in that: After performing face information recognition on each frame of video data to obtain face information of face images of all characters in the current frame and quality parameters of the corresponding face images, the method further includes: When it is determined that there is a face but the quality parameter of the face image is less than a preset threshold, the camera parameters are adjusted according to the face information, the face image is discarded and the quality parameters of the face images generated by subsequent video data are continued to be judged until the quality parameter of the face image is greater than or equal to the preset threshold.

10. The method according to claim 7 or 8, characterized in that: When the quality parameter of the face image is greater than or equal to a preset threshold, the method further includes: The exposure parameters when the quality parameters of the face image are greater than or equal to the threshold are recorded. If it is determined that the camera has not detected face information within a preset time, the camera is adjusted according to the exposure parameters recorded when the most recent face image parameters were greater than or equal to the preset threshold.

11. The method according to claim 7, characterized in that: Determining a person whose face information and quality parameters of the face image meet the preset standard among all the persons in the current frame, and taking a face photo of the person, the method further includes: When it is detected that the storage capacity is less than the warning value, some face photos will be deleted. Specifically, the priority of the face ID is determined according to the access status, upload status and tracking status of the face photos corresponding to the face ID, and real-time adjustment is made. At the same time, the face photos corresponding to the face ID are deleted according to the priority sort and the face ID is cancelled. Finally, when it is detected that the storage capacity is higher than the preset threshold, the deletion work is stopped.

12. The method according to claim 7, characterized in that: According to the face information and face image quality parameters reaching the preset indicators, the face photo is captured and uploaded. Specifically, the face photo upload strategy is selected according to the remote server requirements.

Images