A camera and a method for collecting face information based on face recognition by the camera

Abstract

A camera and a method for collecting face information based on face recognition by the camera. The method is as follows: The main controller collects video data and caches it in the main memory. The main controller transmits the segmented video data in the main memory to the image operation chip. The main controller receives the face information and the quality parameters of the face image in the segmented video data parsed by the image operation chip. The main controller captures, stores, and uploads face photos according to the face information and the quality parameters of the face image reaching the preset indicators. By adding an image operation chip to undertake simple but large amounts of face recognition work, and using a general-purpose processor to obtain and upload face photos according to the results of face recognition, the present invention can replace the bandwidth waste caused by directly uploading the original video stream for the server to perform face recognition and the waste of hardware resources of the server, and is suitable for simultaneously monitoring and tracking more faces to improve the monitoring effect.

Description

[0001] This invention is a divisional application of the patent application with the application number 201710557708.0 and the invention title "A Method for Collecting Facial Information Based on Camera Facial Recognition" filed on July 10, 2017. Technical Field

[0002] This invention relates to a method for collecting facial information, and specifically to a camera and a method for collecting facial information based on camera facial recognition. Background Art

[0003] In today's society, for security reasons, video surveillance devices are increasingly applied to the surveillance of safe cities, as well as the surveillance of various places such as communities and factories. In most cases, the surveillance process is for human recognition. With the maturity of facial recognition technology, it has become possible for artificial intelligence to automatically process the facial images in the surveillance video data. As surveillance devices become more and more popular, and facial recognition has high requirements for video quality, the corresponding surveillance data is also getting larger and larger.

[0004] Currently, when using facial recognition in a surveillance system, in most cases, the method adopted is that the camera sends video data to the background server, and the background server then relies on its powerful hardware and complex software for intelligent analysis. In this way, the data throughput of the surveillance network is astonishing; under the condition of more and more cameras, the storage capacity and computing power of the background server are also severely tested, resulting in a great increase in costs; in addition, the surveillance environment is complex and diverse, and the parameter settings of the camera cannot guarantee real-time integrity, which may lead to the situation where people cannot be recognized and collected. And using a remote server to analyze images to judge the image quality and return requirements for modifying camera parameters, and then the camera adjusts the shooting, will cause great pressure on the hardware overhead and there is not enough time.

[0005] In recent years, the improvement of algorithms and the enhancement of hardware performance have made it possible for cameras to recognize faces. Traditionally, general-purpose processors using common mobile architectures such as cortex or atom based on X86 are not suitable for performing video processing and hardware management while also performing large-scale simple operations required by artificial intelligence. Summary of the Invention

[0006] In order to overcome the situation of large network overhead and serious uneven distribution of hardware overhead in the existing technology surveillance system, the purpose of this invention is to provide a method for collecting facial information based on camera facial recognition.

[0007] The method provided by this invention is as follows:

[0008] A method for collecting face information based on camera face recognition, the method being: during the process of the camera collecting video data, face information recognition is performed on each frame of video data to obtain the face images of all persons in the current frame and the quality parameters of the corresponding face images. Finally, the persons among all persons in the current frame whose face information and quality parameters of the face images meet the preset standards are determined, and a face photo of this person is captured.

[0009] The performing face information recognition on each frame of video data to obtain the face images of all persons in the current frame and the quality parameters of the corresponding face images is specifically: the camera picks up face information according to the brightness distribution in one of the frames of the video data, and then determines the quality parameters of the face images according to the brightness, clarity, and face feature matching degree of the face images in the face information.

[0010] After the camera picks up face information according to the brightness distribution in one of the frames of the video data, it further includes: the camera judges whether the current face image is a new person according to the comparison situation between the current face information and the face information picked up in several adjacent frames. If it is determined to be a new person, a face ID is assigned to the current face image. Otherwise, the current face image is associated with the existing face ID.

[0011] The determining the persons among all persons in the current frame whose face information and quality parameters of the face images meet the preset standards, and capturing a face photo of this person is specifically: when it is determined to be a new person and the quality parameter of the face image is greater than or equal to the preset threshold, the camera captures a face photo and stores it.

[0012] After performing face information recognition on each frame of video data to obtain the face images of all persons in the current frame and the quality parameters of the corresponding face images, the method further includes: when it is determined to be a new person but the quality parameter of the face image is less than the preset threshold, the camera adjusts the camera parameters according to the face information, and at the same time discards the face image and continues to judge the quality parameters of the face images generated by the subsequent video data until the quality parameter of the face image is greater than or equal to the preset threshold.

[0013] The determining the persons among all persons in the current frame whose face information and quality parameters of the face images meet the preset standards, and capturing a face photo of this person is specifically: when it is determined to be an original face and the quality parameter of the face image is greater than or equal to the preset threshold, the camera judges that the quality parameter of the face image or the face frontal degree is greater than the original face photo, and finally captures a face photo and replaces the original face photo.

[0014] After performing face information recognition on each frame of video data to obtain the face images of all persons within the current frame and the quality parameters of the corresponding face images, the method further includes: when it is determined that it is an original face but the quality parameter of the face image is less than a preset threshold, the camera adjusts the camera parameters according to the face information, and at the same time discards the face image and continues to judge the quality parameters of the face images generated by subsequent video data until the quality parameter of the face image is greater than or equal to the preset threshold.

[0015] When the quality parameter of the face image is greater than or equal to the preset threshold, the method further includes: the camera records the exposure parameter when the quality parameter of the face image is greater than or equal to the threshold, and then judges that no face information is detected by the camera within a preset time, and then adjusts the camera according to the exposure parameter recorded when the face image parameter was greater than or equal to the preset threshold last time.

[0016] Determine the persons among all persons within the current frame whose face information and quality parameters of the face images meet the preset standards, and capture face photos of these persons. At the same time, the method further includes: when the camera detects that the storage capacity is less than the warning value, it deletes some face photos. Specifically: the camera determines the priority of the face ID according to the access status, upload status and tracking status of the face photos corresponding to the face ID and makes real-time adjustments, and at the same time deletes the face photos corresponding to the face ID according to the priority ranking and cancels the face ID. Finally, when it is detected that the storage capacity is higher than the preset threshold, the deletion work stops.

[0017] Capture and upload face photos according to the face information and the quality parameters of the face images reaching the preset indicators. Specifically, the camera selects the capture method of face photos according to the requirements of the remote server.

[0018] According to the described method for collecting face information based on camera face recognition, capture and upload face photos according to the face information and the quality parameters of the face images reaching the preset indicators. Specifically, the camera selects the upload strategy of face photos according to the requirements of the remote server.

[0019] Compared with the prior art, the present invention adds an image operation chip to undertake simple but large amounts of face recognition work, and uses a general-purpose processor to obtain and upload face photos according to the results of face recognition, replacing the bandwidth waste caused by directly uploading the original video stream for the server to perform face recognition and the waste of hardware resources of the server, and is suitable for monitoring and tracking more faces simultaneously to improve the monitoring effect. Description of the Drawings

[0020] Figure 1 It is a flowchart of a method for collecting face information based on camera face recognition;

[0021] Figure 2It is a flowchart of a specific implementation of a face information collection method based on camera face recognition. Specific implementation

[0022] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0023] The embodiment of the present invention provides a face recognition method based on a camera. The steps include: the camera collects video data, and while collecting the data, performs face information recognition on each frame of video data to obtain the face information of the face image and the quality parameters of the face image, determines that the face image reaches a preset index, and takes a picture of the face that reaches the preset index.

[0024] To specifically implement this method, the camera internally includes a main controller for controlling the operations of the camera, image encoding and decoding, communication, etc., and a corresponding main memory. It also includes an image operation chip for performing face image recognition using a face recognition algorithm and its corresponding external components such as a memory. The camera can also be equipped with an external memory for storing the captured photos to reduce the storage pressure on the main memory. On this basis, the specific steps of this method include:

[0025] Step 100: The main controller collects the video data of the sensor and caches it in the main memory.

[0026] Step 200: The main controller transmits the segmented video data to the image operation chip.

[0027] Step 300: The image operation chip analyzes the face in the segmented video data, recognizes the face and returns the face information to the main controller.

[0028] Step 301: The image operation chip picks up the face information according to the brightness distribution of the face image in a certain frame of the received segmented video data.

[0029] Step 302: The image operation chip compares the current detected face information with the face information of the face images detected in several adjacent frames forward, judges the similarity between the two, and judges whether the currently detected face is a new face according to the similarity and the set threshold. According to the judgment result, the face information of the face image that has appeared is associated with the corresponding face ID, or a new face ID is set for the new face.

[0030] Step 303; The image operation chip gives the quality parameters of the current face image according to the brightness, clarity of the current face image and the matching degree with the face features under a frontal face.

[0031] Step 304: Transmit the face image and quality parameter information to the main controller.

[0032] Step 400: The main controller adjusts the environmental peripherals according to the face information.

[0033] Step 401: The main controller determines whether to adjust the exposure parameters and the required adjustment step size based on the quality parameters of the face image and the brightness of the current face.

[0034] Step 402: When the quality parameters of the face image reach the set threshold, the main controller records the exposure parameters for a period of time.

[0035] Step 403: When no face is detected, the main controller sets the camera using the recorded exposure parameters.

[0036] Step 500: The main controller captures a face and stores it in the main memory or external memory.

[0037] Step 501: According to the face ID, when the quality parameters of the face image of a new person are higher than the preset threshold, the main controller controls the camera to capture and store the face photo.

[0038] Specifically, the main controller controls the camera to capture face images, body images, or original images, or any free combination of the three according to the settings.

[0039] Step 502: According to the face ID, when receiving the face image information of the saved face photo, if the main controller determines that the current face image is not a frontal face, and the frontal face area of the current face image is larger than that of the saved face photo, and the quality parameters of the current face image are higher than the threshold, or the quality parameters of the current face image are higher than those of the saved face photo, then capture and store the face photo to replace the original face photo and record the quality parameters of the face photo.

[0040] Step 503: According to the face ID, when receiving the face image information of the saved face photo, if the main controller determines that the saved face photo is a frontal face and the quality parameters of the current face image are greater than those of the saved face, then capture and store the face photo to replace the original face photo and record the quality parameters of the face photo.

[0041] Step 600: Upload and clean up the face photos.

[0042] Step 601: When the quality of the face photo exceeds the set value, the main controller uploads the face photo.

[0043] Specifically, the main controller preferentially uploads the face photos whose photo quality parameters reach the upload standard and are closer to the current time, and the face photos with higher quality parameters among all face photos.

[0044] Specifically, the main controller controls the camera to perform three upload methods: real-time upload, upload after a person leaves, and interval upload according to the settings. Among them, real-time upload means uploading the face photo immediately after the face photo changes. Upload after a person leaves means uploading the optimal photo when the person leaves the detection area. Interval upload means regularly uploading the optimal photo when the face image corresponding to the face ID appears in the detection area.

[0045] Step 602: When the used capacity of the main memory or external memory reaches the set value, the main controller sets priorities for each face ID according to the current access and storage situation of the face photo corresponding to the face ID, the current upload situation, the current tracking situation, and the last tracking time.

[0046] Specifically, the face photo currently being accessed and stored has the highest priority. The face photo that has not been uploaded yet has a relatively high priority. The face photo currently being tracked has a relatively high priority. The face photo with a longer time interval between the last tracking time and the current time has a lower priority.

[0047] Step 603: According to the priorities of the face IDs, preferentially delete and overwrite the face photos with lower priorities, and cancel the face ID corresponding to the overwritten face photo.

[0048] Embodiment 1:

[0049] The camera records the video of the detection area where it is located and stores it in the memory. The ARM processor transmits the segmented video data to the FPGA operation chip through the BT1120 standard video data interface.

[0050] The FPGA operation chip caches the segmented video data in its own memory and analyzes each frame of the segmented data. It picks up the brightness distribution of the face area in the image, captures the face image, and records the face information. The FPGA operation chip compares the currently recorded face information with the face information recorded in the recent several frames to determine whether the face is consistent with the previously recorded face. If the face is determined to be a new face, a new face ID is assigned. If the face is determined to be an original face, the currently recorded face information is associated with the face ID corresponding to the original face.

[0051] The FPGA operation chip gives the quality parameter of the current face image according to the brightness, clarity of the current face information, and the matching degree of the face features under the frontal face, and transmits the quality parameter and the face information to the ARM processor.

[0052] The ARM processor determines whether it is necessary to adjust the exposure parameters and other camera parameters according to the face brightness and quality parameters of the current face information, confirms the adjustment step size, controls the camera to complete the modification of the shooting parameters, and at the same time monitors the face information in the subsequent face images transmitted by the FPGA operation chip until the brightness is sufficient and the quality parameters meet the standards.

[0053] On the premise that the face image quality parameters reach the preset threshold, the ARM processor captures a face photo and stores it in the memory.

[0054] Specifically, the ARM processor extracts the face image in the cache as the face photo, and at the same time adds the description information of the file, records the shooting time and the face ID.

[0055] The ARM processor uploads the face photo stored in the memory to the remote server.

[0056] The advantage of this solution is that it uses the face ID to judge the acquisition situation of the face image, which is convenient for accurately controlling and recording the collection status of the face information, and there is no need to acquire and upload a large number of pictures, which greatly saves the hardware cost.

[0057] Obtain the photo shooting situation according to the face information and quality parameters of the face photo, and adjust the camera parameters in real time, which is conducive to taking higher-quality photos.

[0058] Embodiment 2:

[0059] On the basis of Embodiment 1, when the FPGA operation chip fails to detect any face, the ARM processor traces back to the most recently taken face photo that meets the standards, and adjusts the exposure parameters for setting the camera.

[0060] This solution is convenient for setting the exposure state of the camera in the most reasonable state when there is no face in the detection area, which is convenient for quickly adjusting to the appropriate exposure position after detecting a face again.

[0061] Embodiment 3:

[0062] On the basis of Embodiment 1, after the ARM controller receives the face image information, it judges the face ID. When the face photo corresponding to the face ID has been captured and stored, compare the quality parameters of the stored face photo, the frontal face area, and the quality parameters and face area included in the current face image information. If the quality parameters are higher or the frontal face area is larger, capture the face photo again and replace the original face photo.

[0063] If the stored face photo is a frontal face, only compare the quality parameters of the face photo and the quality parameters of the current face image. If the quality parameters of the current face image are higher than those of the stored face photo, capture and store the new face photo to replace the old face photo.

[0064] On average, for photos with a larger frontal face area, the quality parameters of the face images are also relatively high. Therefore, this solution is suitable for screening out images and photos that can better identify people. This solution facilitates quick comparison and obtaining higher-quality face photos, and is convenient for storing face photos with better quality.

[0065] Example 4:

[0066] Based on Example 3, the ARM processor also receives instructions from the remote server and adjusts the capture rules according to the requirements of the remote server.

[0067] Specifically, it includes but is not limited to the following picture styles:

[0068] The original image of the current scene, the image of the human body to which the face belongs, or the face image of the tracked face, or any combination of the three;

[0069] It also includes but is not limited to the following capture strategies:

[0070] When the quality parameters of the face image reach the threshold, immediately capture and upload;

[0071] When the person to whom the face belongs leaves the detection area, upload the optimal face photo corresponding to this face;

[0072] Regularly transmit the optimal face image when the person to whom the face belongs appears in the detection area.

[0073] This solution makes the camera more adaptable to daily monitoring tasks and retains more reasonable data in the case of no video data for archiving.

[0074] Example 5:

[0075] Based on Example 1, add a TF card or ROM memory as an external storage to connect with the ARM processor, and the ARM processor stores the face photos in the TF card or ROM memory.

[0076] This solution optimizes the storage environment of the face photos, preventing the situation where photos are not uploaded in time and are discarded, or new captured photos cannot be stored.

[0077] Example 6:

[0078] Based on Example 1 or 5, when the ARM processor monitors that the storage space of the memory or external storage is less than the preset threshold, prioritize the stored face photos.

[0079] Specifically, according to the face ID, it is determined whether the face photo corresponding to the face ID is being accessed or stored. A high priority is set for the face photo being accessed or stored. The upload status of the face photo corresponding to the face ID is judged, and a low priority is set for the face photo that has been uploaded. The face image tracking status corresponding to the face ID is judged, and a high priority is set for the face ID corresponding to the face image being tracked. The priorities are set from high to low in sequence according to the last tracking time from near to far.

[0080] Based on the priority information that changes in real time, the ARM processor deletes the face photos corresponding to the face IDs with lower priorities in the memory or external storage, and cancels the corresponding face IDs. When the storage space of the memory or external storage is higher than a preset threshold, the deletion work is stopped.

[0081] This solution ensures the more effective preservation of high-value face photos and does not interfere with the normal progress of monitoring work under extremely poor network conditions.

[0082] As described above, only the specific preferred embodiments of the present invention are provided, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A camera, characterized in that: It includes a main controller and an image operation chip. Among them, the image operation chip is used to analyze each frame of video data in the video data collected by the camera using a face recognition algorithm, and obtain the face information of all the faces in the current frame and the quality parameters of the corresponding face images; the main controller is used to, when the quality parameter of the face image is less than a preset threshold, adjust the parameters of the camera according to the face information, and at the same time discard the face image and continue to judge the quality parameters of the face images generated by the subsequent video data until the quality parameter of the face image is greater than or equal to the preset threshold; the face image is a new face or an original face; the main controller is also used to, when the quality parameter of the face image reaches the preset threshold, control the camera to capture a face photo and upload it; the image operation chip is specifically used for: picking up the face information according to the brightness distribution in the current frame of the video data; determining the quality parameter of the face image according to the brightness, clarity and face feature matching degree of the face image in the face information; the main controller is also used for: recording the exposure parameter when the quality parameter of the face image is greater than or equal to the preset threshold; when no face is detected in the video data by the image operation chip, obtaining the exposure parameter of the camera used when capturing the face image with the quality parameter reaching the preset threshold for the last time; adjusting the parameters of the camera according to the exposure parameter; the image operation chip is also used for: after picking up the face information by the image operation chip, comparing the currently detected face information with the face information picked up in several adjacent frames forward, judging the similarity between the two, and judging whether the currently detected face image is a new face image according to the similarity and the set threshold. If it is determined to be a new face, assign a face ID to the current face image. Otherwise, establish a correspondence between the current face image and the existing face ID; the camera also includes a main memory, and the main controller is specifically used for: when it is determined that the face image in the video data is not a new face image and the quality parameter of the face image reaches the preset threshold, if the quality parameter of the face image in the video data is greater than the quality parameter of the original face image in the main memory, or the frontal degree of the face image in the video data is greater than the frontal degree of the original face image, then control the camera to capture a face photo and upload it; wherein, the face image in the video data and the original face image are face images of the same person.

2. The camera according to claim 1, characterized in that: the main controller is specifically used for: judging whether it is necessary to adjust the exposure parameter of the camera and the adjustment step size according to the quality parameter of the face image and the brightness of the current face; controlling the camera to complete the modification of the parameters.

3. The camera according to claim 1 or 2, characterized in that: the main controller is specifically used for: when it is judged that the face image in the video data is a new face image and the quality parameter of the face image reaches the preset threshold, controlling the camera to capture a face photo and upload it.

4. The camera according to claim 1 or 2, characterized in that: the camera also includes a main memory, and the main controller is also used for: Store the face photo in the main memory.

5. The camera according to claim 1, characterized in that, The main controller is further configured to: Delete the original face image in the main memory and store the face photo in the memory.

6. The camera according to claim 1 or 2, characterized in that: Specifically, the main controller is configured to: Select an upload strategy for the face photo according to the requirements of the remote server and upload the face photo according to the upload strategy.

7. The camera according to claim 1 or 2, characterized in that: The main controller is further configured to: Receive an instruction from the remote server and adjust the capture rule according to the instruction of the remote server; Specifically, the main controller is configured to: When the quality parameter of the face image reaches the preset threshold, the camera captures to obtain a face photo and uploads the face photo; Or, After the person to whom the face image belongs leaves the detection area, obtain the face image with the highest quality parameter corresponding to the person and upload the face image; Or, After the person to whom the face image belongs appears in the detection area, regularly upload the face image with the highest quality parameter corresponding to the person.

8. The camera according to claim 1 or 2, characterized in that: The main controller is further configured to: Determine the shooting time and face identifier of the face photo; Record the shooting time and the face identifier of the face photo.

9. The camera according to claim 1 or 2, characterized in that: The camera further includes an external memory, and the external memory is connected to the main controller, wherein The main controller is further configured to store the face photo in the external memory.

10. A method for collecting face information based on camera face recognition, characterized in that: Including: Using a face recognition algorithm, analyze each frame of video data collected by the camera to obtain the face information of the face images of all persons in the current frame and the quality parameters of the corresponding face images; When the quality parameter of the face image is less than the preset threshold, adjust the parameters of the camera according to the face information, and at the same time discard the face image and continue to judge the quality parameters of the face images generated by the subsequent video data until the quality parameter of the face image is greater than or equal to the preset threshold; the face image includes a new person or an original face; When the quality parameter of the face image reaches the preset threshold, control the camera to capture a face photo and upload it; The obtaining of the face information of the face images of all persons in the current frame and the quality parameters of the corresponding face images includes: Pick up the face information according to the brightness distribution in the current frame of the video data; Determine the quality parameter of the face image according to the brightness, clarity and face feature matching degree of the face image in the face information; The method further includes: Record the exposure parameter when the quality parameter of the face image is greater than or equal to the preset threshold; When no face is detected in the video data, obtain the exposure parameter of the camera used when obtaining the face image with the highest quality parameter that was captured most recently and reached the preset threshold; Adjust the exposure parameter according to the camera parameters; After using a face recognition algorithm to analyze each frame of video data collected by the camera to obtain the face information of the face images of all persons in the current frame and the quality parameters of the corresponding face images, it further includes: Compare the currently detected face information with the face information picked from several adjacent frames forward to determine the similarity between the two. Then, based on the similarity and a preset threshold, determine whether the currently detected face image is a face image of a new person. If it is determined to be a new person, assign a face ID to the current face image; otherwise, establish a correspondence between the current face image and the existing face ID. The camera further includes a main memory. When the quality parameter of the adjusted face image reaches the preset threshold, control the camera to capture a face photo and upload it, including: When it is determined that the face image in the video data is not a face image of a new person and the quality parameter of the face image reaches the preset threshold, if the quality parameter of the face image in the video data is greater than the quality parameter of the original face image in the main memory, or the frontal face degree of the face image in the video data is greater than the frontal face degree of the original face image, then control the camera to capture a face photo and upload it; Wherein, the face image in the video data and the original face image are face images of the same person.

11. The method according to claim 10, characterized in that: It further includes: Judge whether it is necessary to adjust the exposure parameter of the camera and the adjustment step size according to the quality parameter of the face image and the brightness of the current face; Control the camera to complete the modification of the parameters.

12. The method according to claim 10 or 11, characterized in that: When the quality parameter of the adjusted face image reaches the preset threshold, control the camera to capture a face photo and upload it, including: When it is judged that the face image in the video data is a face image of a new person and the quality parameter of the face image reaches the preset threshold, control the camera to capture a face photo and upload it.

13. The method according to claim 10 or 11, characterized in that: The camera further includes a main memory, and the method further includes: Store the face photo in the main memory.

14. The method according to claim 10, wherein: The method further includes: Delete the original face image in the main memory and store the face photo in the memory.

15. The method according to claim 10 or 11, characterized in that: Upload the face photo, including: Select an upload strategy for the face photo according to the requirements of the remote server and upload the face photo according to the upload strategy.

16. The method according to claim 10 or 11, characterized in that: The method further includes: Receive an instruction from the remote server and adjust the capture rule according to the instruction of the remote server; Upload the face photo, including: When the quality parameter of the face image reaches the preset threshold, the camera captures to obtain a face photo and uploads the face photo; Or, After the person to whom the face image belongs leaves the detection area, obtain the face image with the highest quality parameter corresponding to the person and upload the face image; Or, After the person to whom the face image belongs appears in the detection area, regularly upload the face image with the highest quality parameter corresponding to the person.

17. The method according to claim 10 or 11, characterized in that: The method further includes: Determine the shooting time and face identifier of the face photo; Record the shooting time and the face identifier of the face photo.

18. The method according to claim 10 or 11, characterized in that: The camera further includes an external memory, and the method further includes: Store the face photo in the external memory.

Images