A cell face posture detection system and method based on online reinforcement learning

Abstract

The application provides a cell face posture detection system and method based on online reinforcement learning, which comprises a monitoring device, an AI server and a front-end display device, the AI server internally deploying a video detection service module and an online reinforcement learning service module, the video detection service module comprising a video stream acquisition unit, a video stream decoding unit and a trained FacePose-RCNN model deployed in the video detection service module, the FacePose-RCNN model comprising a face detection network branch network Faster RCNN and a face posture recognition branch network FacePose-Net; the online reinforcement learning service module comprising an online checking unit, a face posture database, an online learning unit and an online updating unit; through the above system, continuous reinforcement learning of the face detection network model can be carried out online quickly after the newly added data of the face posture database exceeds a certain amount. The application continuously enhances the recognition ability of the network model to the face posture through online reinforcement learning, and improves the accuracy of face posture detection.

Description

Technical Field

[0001] This invention belongs to the field of intelligent community monitoring technology, and in particular relates to a community face pose detection system and method based on online reinforcement learning. Background Technology

[0002] Artificial intelligence is developing rapidly in society and is being widely applied to various fields of social production and life, including intelligent transportation, smart community security, and other smart Internet of Things applications. This is mainly due to the rise of deep neural network technology and the rapid development of GPU hardware. Deep neural network technology has achieved state-of-the-art results in many fields, including object detection, image classification, image segmentation, image recognition, and speech recognition.

[0003] With the increasing sophistication of residential communities, IoT devices, especially cameras and surveillance equipment, are widely deployed, generating massive amounts of surveillance video data daily. Community surveillance video data analysis has become a hot topic in the field of video analytics. Face pose detection, as a crucial task in community surveillance video analysis, is significant for face recognition, attention detection, behavior analysis, human-computer interaction, and tracking. Current face pose detection methods face challenges in community surveillance scenarios, such as complex backgrounds, variable camera angles, and diverse face poses. In practical applications, these methods suffer from low accuracy and numerous false detections. Therefore, a high-precision face pose detection method is urgently needed to improve the accuracy of face pose detection. Summary of the Invention

[0004] To address the aforementioned problems, the first aspect of this invention provides a cell face pose detection system based on online reinforcement learning, comprising a monitoring device, an AI server, and a front-end display device; the monitoring device is used to acquire cell video streams; the AI ​​server is equipped with a video detection service module and an online reinforcement learning service module; the video detection service module includes a video stream acquisition unit, a video stream decoding unit, and a trained FacePose-RCNN model deployed within the video detection service module; the video stream acquisition unit acquires the video stream acquired by the monitoring device and sends it to the video stream decoding unit for decoding; the FacePose-RCNN model performs face pose detection on the decoded video frames, and the FacePose-RCNN model includes a face detection network branch network Faster. The R-CNN and FacePose-Net face pose recognition subnetwork are used. The online reinforcement learning service module includes an online verification unit, a face pose database, an online learning unit, and an online update unit. The online verification unit is used to obtain the face pose detection results processed by the FacePose-RCNN model and display and verify them on the front-end display device. The verification results are sent to the face pose database. After the number of new data in the face pose database exceeds a certain amount, the online reinforcement learning training function of the online learning unit is executed to reinforce the FacePose-Net face pose recognition subnetwork with the new face pose data samples. After the training is completed, the update function of the online update unit is executed to update the trained model parameters into the FacePose-RCNN model.

[0005] In one possible design, the face detection network branch network Faster RCNN includes an RPN layer, a ROIAlign layer, a feature classification layer, and a bounding box regression layer;

[0006] The RPN layer is used to extract candidate boxes, the ROI Align layer is used to align the features of the candidate boxes, the feature classification layer is used to classify the candidate boxes to generate target categories, and the bounding box regression layer is used to regress the candidate boxes to generate face target boxes.

[0007] The FacePose-Net face pose recognition subnetwork includes an ROI alignment layer, a convolutional pooling layer, and a SoftMax classifier;

[0008] The ROI alignment layer is used to extract the basic features of the face on the basic feature layer. The convolutional pooling layer is used to perform convolutional pooling on the target features through a convolutional pooling layer with a large stride convolutional kernel to obtain the face pose features. The SoftMax classifier is used to perform reinforcement learning on the face pose features.

[0009] In one possible design, after the number of new data entries in the face pose database exceeds 1,000, the online reinforcement learning training function of the online learning unit is executed.

[0010] In one possible design, the training process of the trained FacePose-RCNN model deployed in the video detection service module is as follows:

[0011] Step 1: Data Collection; Collect historical facial recognition data from community surveillance cameras, and obtain and decode surveillance video streams from on-site monitoring equipment;

[0012] Step 2, data annotation; filter valid data and annotate facial targets and facial poses;

[0013] Step 3, Model Training: Use face target labeled data to train the FacePose-Net branch network for face pose recognition in the Faster RCNN model, and adjust the parameters of the intermediate training results until the optimal model parameters are obtained.

[0014] A second aspect of this invention also provides a cell face pose detection method based on online reinforcement learning, mainly comprising the following steps:

[0015] Step 1: Acquire the video stream from the monitoring device through the video stream acquisition unit and send it to the video stream decoding unit for decoding;

[0016] Step 2: Send the decoded video frames to the trained FacePose-RCNN face pose detection model for face pose detection;

[0017] Step 3: Send the face pose detection results from Step 2 to the online verification unit and display them on the front end. The front end display device will then display and verify the results, while simultaneously sending the verification results to the face pose database.

[0018] Step 4: When the number of new data added to the face pose database exceeds a certain amount, execute the online reinforcement learning training function of the online learning unit, and use the newly added face pose data samples to reinforce the Face Pose-Net face pose recognition branch network.

[0019] Step 5: After training is complete, execute the online update unit update function and update the trained model parameters into the FacePose-RCNN model.

[0020] In one possible design, the face detection network branch network Faster RCNN includes an RPN layer, a ROIAlign layer, a feature classification layer, and a bounding box regression layer;

[0021] The RPN layer is used to extract candidate boxes, the ROI Align layer is used to align the features of the candidate boxes, the feature classification layer is used to classify the candidate boxes to generate target categories, and the bounding box regression layer is used to regress the candidate boxes to generate face target boxes.

[0022] The FacePose-Net face pose recognition subnetwork includes an ROI alignment layer, a convolutional pooling layer, and a SoftMax classifier;

[0023] The ROI alignment layer is used to extract the basic features of the face on the basic feature layer. The convolutional pooling layer is used to perform convolutional pooling on the target features through a convolutional pooling layer with a large stride convolutional kernel to obtain the face pose features. The SoftMax classifier is used to perform reinforcement learning on the face pose features.

[0024] In one possible design, after the number of new data entries in the face pose database exceeds 1,000, the online reinforcement learning training function of the online learning unit is executed.

[0025] In one possible design, the training process of the trained FacePose-RCNN face pose detection model is as follows:

[0026] Step 1: Data Collection; Collect historical facial recognition data from community surveillance cameras, and obtain and decode surveillance video streams from on-site monitoring equipment;

[0027] Step 2, data annotation; filter valid data and annotate facial targets and facial poses;

[0028] Step 3, Model Training: Use face target labeled data to train the FacePose-Net branch network for face pose recognition in the Faster RCNN model, and adjust the parameters of the intermediate training results until the optimal model parameters are obtained.

[0029] A third aspect of the present invention also provides a computer-readable storage medium storing a program or instructions that, when executed by a processor, cause a computer to perform the method described in the second aspect.

[0030] The beneficial effects of this invention are as follows: This invention provides a cell face pose detection system and method based on online reinforcement learning. By continuously enhancing the network model's ability to recognize face poses through online reinforcement learning, the accuracy of face pose detection is improved. The face pose detection network proposed in this invention, FacePose-RCNN, adds a face pose recognition branch, FacePose-Net, to Faster RCNN, enabling face pose detection with minimal increase in computational cost. The FacePose-Net branch proposed in this invention extracts and classifies face pose features and can be easily transferred to other RCNN networks. The cell face pose detection method proposed in this invention, based on online reinforcement learning, allows for rapid and continuous online reinforcement learning of the face detection network model. Attached Figure Description

[0031] Figure 1 This is a structural block diagram of the face pose detection system for residential communities according to the present invention.

[0032] Figure 2 This is a diagram of the FacePose-RCNN network structure for face pose detection in this invention.

[0033] Figure 3 This is a network structure diagram of FacePose-Net, the face pose recognition network branch in this invention;

[0034] Figure 4 This is a flowchart of the cell face pose detection method based on online reinforcement learning according to the present invention. Detailed Implementation

[0035] The invention will be further described below with reference to specific embodiments.

[0036] Example 1:

[0037] This embodiment provides a community face pose detection system and method based on online reinforcement learning. The system includes monitoring equipment, an AI server, and a front-end display device. The specific structure of the system is as follows: Figure 1 As shown; the monitoring equipment can be common community monitoring equipment such as high-definition cameras, used to acquire video streams of the community; the AI ​​server is equipped with a video detection service module and an online reinforcement learning service module.

[0038] The video detection service module includes a video stream acquisition unit, a video stream decoding unit, and a trained FacePose-RCNN model deployed within the module. The video stream acquisition unit acquires the video stream from the monitoring device and sends it to the video stream decoding unit for decoding. The FacePose-RCNN model performs face pose detection on the decoded video frames. The FacePose-RCNN model includes a face detection branch network (Faster RCNN) and a face pose recognition branch network (FacePose-Net). The structure of the face detection branch network (Faster RCNN) is as follows: Figure 2 As shown, the network includes an RPN layer, an ROI Align layer, a feature classification layer, and a bounding box regression layer. The RPN layer extracts candidate bounding boxes, the ROI Align layer aligns the features of the candidate bounding boxes, the feature classification layer classifies the candidate bounding boxes to generate target categories, and the bounding box regression layer regresses the candidate bounding boxes to generate face target boxes. The structure of the FacePose-Net subnetwork for face pose recognition is as follows: Figure 3 As shown, it includes an ROI alignment layer, a convolutional pooling layer, and a SoftMax classifier. The ROI alignment layer is used to extract the basic features of the face on the basic feature layer. The convolutional pooling layer is used to perform convolutional pooling processing on the target features through a convolutional pooling layer with a large stride convolutional kernel to obtain the face pose features. The SoftMax classifier is used to perform reinforcement learning on the face pose features.

[0039] The online reinforcement learning service module includes an online verification unit, a face pose database, an online learning unit, and an online update unit. The online verification unit is used to obtain the face pose detection results processed by the FacePose-RCNN model and display and verify them on a front-end display device, which can be a PC terminal computer, and the verification is performed manually by property management personnel. The verification results are sent to the face pose database. After the number of new data in the face pose database exceeds a certain amount (e.g., 1000 images), the online reinforcement learning training function of the online learning unit is executed to reinforce the face pose recognition branch network FacePose-Net with the newly added face pose data samples. After the training is completed, the update function of the online update unit is executed to update the trained model parameters into the FacePose-RCNN model.

[0040] Based on the above system, this invention provides a cell face pose detection method based on online reinforcement learning, such as... Figure 4 As shown, it includes the following steps:

[0041] Step 1: Acquire the video stream from the monitoring device through the video stream acquisition unit and send it to the video stream decoding unit for decoding;

[0042] Step 2: Send the decoded video frames to the trained FacePose-RCNN face pose detection model for face pose detection;

[0043] Step 3: Send the face pose detection results from Step 2 to the online verification unit and display them on the front end. The front end display device will then display and verify the results, while simultaneously sending the verification results to the face pose database.

[0044] Step 4: When the number of new data added to the face pose database exceeds a certain amount, execute the online reinforcement learning training function of the online learning unit, and use the newly added face pose data samples to reinforce the Face Pose-Net face pose recognition branch network.

[0045] Step 5: After training is complete, execute the online update unit update function and update the trained model parameters into the FacePose-RCNN model.

[0046] The specific training process of the initial trained FacePose-RCNN face pose detection model is as follows:

[0047] S1, Data Acquisition: Collect historical facial recognition data from community surveillance cameras, and acquire and decode surveillance video streams from on-site monitoring equipment.

[0048] S2, Data Labeling: Filter valid data and label facial targets and facial poses. Facial pose classification labels and criteria include:

[0049]

[0050] S3, Model Training: The FacePose-Net branch network in the Faster RCNN model is trained using face target annotation data, and the intermediate training results are adjusted until the optimal model parameters are obtained.

[0051] Example 2:

[0052] The present invention also provides a non-volatile computer-readable storage medium storing a program or instructions that, when executed by a processor, cause a computer to perform the cell face pose detection method based on online reinforcement learning as described in Embodiment 1.

[0053] Specifically, a system, apparatus, or device may be provided equipped with a readable storage medium on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system, apparatus, or device reads and executes the instructions stored in the readable storage medium. In this case, the program code read from the readable medium itself can implement the functions of any of the embodiments described above, therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0054] The aforementioned storage media can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disks or optical disks (such as CD-ROM, CD-R, CD-RW, DVD-20ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tape, etc. The storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0055] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0056] It should be understood that the storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and storage medium can exist as discrete components in a terminal or server.

[0057] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0058] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0059] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. Although specific embodiments of the present invention have been described above, they are not intended to limit the protection scope of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A cell face pose detection system based on online reinforcement learning, characterized in that: The system comprises a monitoring device, an AI server and a front-end display device; the monitoring device is used to acquire a video stream of a cell; the AI server internally deploys a video detection service module and an online reinforcement learning service module; the video detection service module comprises a video stream acquisition unit, a video stream decoding unit and a trained FacePose-RCNN model deployed in the video detection service module; the video stream acquisition unit is used to acquire the video stream acquired by the monitoring device and send it to the video stream decoding unit for decoding, the FacePose-RCNN model is used to detect the face pose of the decoded video frame, and the FacePose-RCNN model comprises a face detection network branch Faster RCNN and a face pose recognition branch FacePose-Net; the online reinforcement learning service module comprises an online checking unit, a face pose database, an online learning unit and an online updating unit; the online checking unit is used to acquire the face pose detection result processed by the FacePose-RCNN model and display and check it on the front-end display device, the checking result is sent to the face pose database, and when the newly added data of the face pose database exceeds a set number, the online reinforcement learning training function of the online learning unit is executed to reinforce the training of the face pose recognition branch FacePose-Net with the newly added face pose data samples, and after the training is completed, the updating function of the online updating unit is executed, and the trained model parameters are updated to the FacePose-RCNN model; The face detection network branch Faster RCNN comprises an RPN layer, a ROI Align layer, a feature classification layer and a bounding box regression layer; The RPN layer is used to extract a candidate box, the ROI Align layer is used to perform feature alignment on the candidate box features, the feature classification layer is used to classify the candidate box to generate a target category, and the bounding box regression layer is used to regress the candidate box to generate a face target box; The face pose recognition branch FacePose-Net comprises a ROI alignment layer, a convolutional pooling layer and a SoftMax classifier; The ROI alignment layer is used to extract the basic features of the face on the basic feature layer, the convolutional pooling layer is used to perform convolutional pooling processing on the target features through a large-step convolution kernel to obtain face pose features, and the SoftMax classifier is used to perform reinforcement learning on the face pose features.

2. The cell face pose detection system based on online reinforcement learning of claim 1, wherein: When the newly added data of the face pose database exceeds 1000, the online reinforcement learning training function of the online learning unit is executed.

3. The cell face pose detection system based on online reinforcement learning of claim 1, wherein, The training process of the trained FacePose-RCNN model deployed in the video detection service module is as follows: Step one, data collection; collect historical monitoring face data of a cell, acquire monitoring video stream from a field monitoring device and decode it; Step two, data labeling; filter effective data and label the face target and face pose; Step three, model training; the face target annotation data is used for training the face posture recognition branch network FacePose-Net in the Faster RCNN model, and the intermediate training result is adjusted until the optimal model parameter is obtained.

4. A cell face posture detection method based on online reinforcement learning, characterized in that, Comprise the following steps: Step 1, the video stream acquisition unit of the monitoring device is obtained and sent to the video stream decoding unit for decoding; Step 2, the decoded video frame is sent to the trained face posture detection model FacePose-RCNN model for face posture detection; the FacePose-RCNN model comprises a face detection network branch network Faster RCNN and a face posture recognition branch network FacePose-Net; The Faster RCNN comprises an RPN layer, a ROI Align layer, a feature classification layer and a bounding box regression layer; The RPN layer is used for extracting the candidate frame, the ROI Align layer is used for aligning the candidate frame feature, the feature classification layer is used for classifying the candidate frame to generate the target category, and the bounding box regression layer is used for regressing the candidate frame to generate the face target frame; The FacePose-Net comprises a ROI alignment layer, a convolutional pooling layer and a SoftMax classifier; The ROI alignment layer is used for extracting the basic feature of the face on the basic feature layer, the convolutional pooling layer is used for performing convolutional pooling processing on the target feature through a large step convolution kernel to obtain the face posture feature, and the SoftMax classifier is used for reinforcement learning on the face posture feature; Step 3, the face posture detection result in step 2 is sent to the online checking unit, and displayed in the front end, and displayed and checked in the front end display device, and the checking result is sent to the face posture database; Step 4, when the new data in the face posture database exceeds a certain number, the online reinforcement learning training function of the online learning unit is executed, and the face posture recognition branch network FacePose-Net is reinforced trained with the new face posture data sample; Step 5, the update function of the online update unit is executed after training, and the model parameter after training is updated to the FacePose-RCNN model.

5. The cell face pose detection method based on online reinforcement learning according to claim 4, characterized in that: When the new data in the face posture database exceeds 1000, the online reinforcement learning training function of the online learning unit is executed.

6. The cell face pose detection method based on online reinforcement learning according to claim 4, characterized in that, The training process of the trained face posture detection model FacePose-RCNN model is as follows: Step one, data collection; collect historical monitoring face data in the community, obtain monitoring video stream from the field monitoring device and decode; Step 2, data annotation; filter effective data and annotate face target and face posture; Step 3, model training; the face target annotation data is used for training the face posture recognition branch network FacePose-Net in the Faster RCNN model, and the intermediate training result is adjusted until the optimal model parameter is obtained.

7. A computer-readable storage medium, characterized in that: The computer readable storage medium stores programs or instructions, and the programs or instructions are executed by the processor to make the computer execute the method in any one of claims 4 to 6.

Images