Personnel intrusion detection method and device, equipment, storage medium and computer program product

Abstract

The invention discloses a personnel intrusion detection method, device and equipment, a storage medium and a computer program product, and relates to the technical field of intelligent security protection, and the method comprises the steps: carrying out the face recognition of a to-be-recognized user, and judging whether the posture recognition of the to-be-recognized user is needed or not according to a face recognition result; if so, performing posture recognition on the to-be-recognized user, and judging whether the to-be-recognized user is an external user according to a posture recognition result; if yes, judging whether the user to be identified is in a target prohibited area of the current monitoring environment; if not, whether the to-be-identified user is located in a secondary prohibited area of the current monitoring environment is judged, and the secondary prohibited area is determined based on the target prohibited area; and obtaining an intrusion detection result of the to-be-identified user according to a judgment result. Through application of the technical scheme, the technical problem that in the prior art, personnel intrusion detection performed by adopting a face recognition technology is greatly influenced by an external environment, so that the accuracy of intrusion detection is low is solved.

Description

Technical Field

[0001] The present application relates to the field of intelligent security technology, and in particular to methods, devices, equipment, storage media, and computer program products for detecting human intrusion. Background Art

[0002] With the development of intelligent security technology, intrusion detection methods based on video surveillance have become an important means of ensuring regional security. They are widely used in security systems, smart homes, attendance management, and other fields. Human intrusion detection technology can protect the security of individuals, enterprises, and critical infrastructure by automatically monitoring and identifying unauthorized intrusions, promptly detecting abnormal behavior or potential threats, thereby enhancing security awareness and providing early warnings.

[0003] Currently, facial recognition technology is commonly used for intrusion detection. This technology continuously captures video streams from high-definition cameras installed in the surveillance area. Faces detected in the video stream are then compared with the faces of authorized personnel to identify intrusions. However, using facial recognition technology for intrusion detection places high demands on the camera's shooting distance, angle, and lighting. If the camera is too far away, the angle is poor, or the lighting is unstable, the accuracy of facial recognition will be affected, resulting in low intrusion detection accuracy. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment, storage medium and computer program product for personnel intrusion detection, aiming to solve the technical problem in the existing technology that the use of face recognition technology for personnel intrusion detection is greatly affected by the external environment, which easily leads to low accuracy of intrusion detection.

[0005] To achieve the above objectives, the present application proposes a method for detecting human intrusion, the method comprising:

[0006] Performing facial recognition on the user to be identified, and determining whether gesture recognition is required for the user to be identified based on the facial recognition result;

[0007] If necessary, performing gesture recognition on the user to be identified, and judging whether the user to be identified is an outsider according to the gesture recognition result;

[0008] If yes, determining whether the user to be identified is in the target prohibited area of ​​the current monitoring environment;

[0009] If not, determining whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, where the secondary prohibited area is determined based on the target prohibited area;

[0010] An intrusion detection result of the user to be identified is obtained according to the judgment result and the walking data of the user to be identified.

[0011] In one embodiment, before the step of performing face recognition on the user to be identified, the method further includes:

[0012] Perform data enhancement processing on the facial image corresponding to the authorized user to obtain the target facial image;

[0013] Performing feature extraction on the target face image to obtain a face feature vector;

[0014] Performing feature extraction on the point cloud data in the face image to obtain a face point cloud feature vector;

[0015] Generate a comprehensive facial feature vector based on the facial feature vector and the facial point cloud feature vector;

[0016] Training an initial face recognition model based on the comprehensive face feature vector to obtain a target face recognition model;

[0017] The step of performing face recognition on the user to be identified includes:

[0018] The target face recognition model is used to perform face recognition on the user to be identified.

[0019] In one embodiment, before the step of performing face recognition on the user to be identified, the method further includes:

[0020] Preprocessing the posture video of the authorized user to obtain a plurality of posture walking images, wherein the posture video is a posture video of the authorized user walking at multiple angles;

[0021] Extract features from walking images of different postures to obtain joint feature vectors;

[0022] Extracting features from the point cloud data in the walking images of each posture to obtain joint point cloud feature vectors;

[0023] Generate a joint comprehensive feature vector based on the joint feature vector and the joint point cloud feature vector;

[0024] Training an initial posture recognition model based on the joint comprehensive feature vector to obtain a target posture recognition model;

[0025] The step of performing gesture recognition on the user to be identified includes:

[0026] Performing posture recognition on the user to be identified using the target posture recognition model.

[0027] In one embodiment, the step of extracting features from the walking images of each posture to obtain joint feature vectors includes:

[0028] Perform data enhancement processing on the walking images of each posture to obtain several walking images of the target posture;

[0029] Extract human joint points from each target walking posture image to obtain several human joint points;

[0030] Determine the joint point feature value corresponding to each human joint point based on the position information of each human joint point;

[0031] A joint feature vector is constructed based on the joint point feature values.

[0032] In one embodiment, before the step of determining whether the user to be identified is in the secondary prohibited area of ​​the current monitoring environment, the method further includes:

[0033] Determine coordinate information of the polygonal frame corresponding to the target prohibited area;

[0034] Constructing a rectangular frame based on the quadrant maximum value and the quadrant minimum value in the coordinate information;

[0035] Extending the side length of the rectangular frame to obtain a rectangular frame with extended side lengths;

[0036] Determining whether the rectangular frame after the side length is extended exceeds a preset shooting boundary;

[0037] If not, the rectangular frame with the extended side length is determined as the secondary prohibited area of ​​the current monitoring environment.

[0038] In one embodiment, before the step of determining whether the user to be identified is in a target prohibited area of ​​the current monitoring environment, the method further includes:

[0039] Acquire the posture walking data of the authorized user from the posture video;

[0040] determining an average walking speed of the authorized user based on the posture walking data;

[0041] The step of obtaining the intrusion detection result of the user to be identified based on the judgment result and the walking data of the user to be identified includes:

[0042] If so, determining the current walking speed of the user to be identified based on the walking data of the user to be identified;

[0043] Based on the current walking speed and the average walking speed, it is determined whether the user to be identified has intrusion behavior, and an intrusion detection result of the user to be identified is obtained.

[0044] In addition, to achieve the above objectives, the present application also proposes a human intrusion detection device, the device comprising:

[0045] A face recognition module is used to perform face recognition on a user to be identified and determine whether gesture recognition is required for the user to be identified based on the face recognition result;

[0046] a gesture recognition module, configured to perform gesture recognition on the user to be identified, if necessary, and determine whether the user to be identified is an outsider based on the gesture recognition result;

[0047] an intrusion detection module, configured to determine whether the user to be identified is in a target prohibited area of ​​the current monitoring environment;

[0048] The intrusion detection module is further configured to, if not, determine whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, where the secondary prohibited area is determined based on the target prohibited area;

[0049] The detection result acquisition module is used to obtain the intrusion detection result of the user to be identified based on the judgment result and the walking data of the user to be identified.

[0050] In addition, to achieve the above-mentioned purpose, the present application also proposes a personnel intrusion detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the personnel intrusion detection method as described above.

[0051] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the steps of the human intrusion detection method described above are implemented.

[0052] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the steps of the human intrusion detection method as described above.

[0053] The present application provides a method for detecting intrusion of personnel, which discloses performing face recognition on a user to be identified, and judging whether it is necessary to perform posture recognition on the user to be identified based on the face recognition result; if necessary, performing posture recognition on the user to be identified, and judging whether the user to be identified is an outsider based on the posture recognition result; if so, judging whether the user to be identified is in a target prohibited area of ​​the current monitoring environment; if not, judging whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, the secondary prohibited area being determined based on the target prohibited area; obtaining an intrusion detection result of the user to be identified based on the judgment result; compared with the prior art method of adopting When using facial recognition technology for personnel intrusion detection, the camera's shooting distance, shooting angle and lighting requirements will affect the accuracy of facial recognition, resulting in low intrusion detection accuracy. Since the present invention can combine facial recognition and posture recognition to determine whether the user to be identified is an outsider, and when the user to be identified is an outsider and is in the secondary prohibited area of ​​the current monitoring environment but not the prohibited area, the intrusion detection result is obtained based on the walking data of the user to be identified, thereby solving the technical problem in the existing technology that the use of facial recognition technology for personnel intrusion detection is greatly affected by the external environment, which easily leads to low intrusion detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0056] Figure 1 A flowchart of the first embodiment of the human intrusion detection method provided by the present application;

[0057] Figure 2 A flowchart for determining secondary prohibited areas in the human intrusion detection method of the present application;

[0058] Figure 3 This is the overall flow chart of the personnel intrusion detection method of the present applicant;

[0059] Figure 4 A flowchart of the second embodiment of the human intrusion detection method provided by the present application;

[0060] Figure 5 This is a diagram showing the user posture video collected in the intrusion detection method of the present applicant;

[0061] Figure 6 A diagram showing the human body joints used in the applicant's intrusion detection method;

[0062] Figure 7 A flowchart of the third embodiment of the human intrusion detection method provided by the present application;

[0063] Figure 8 A diagram showing the steps of the applicant's method for detecting intrusion by personnel;

[0064] Figure 9 This is a schematic diagram of the module structure of the human intrusion detection device according to an embodiment of the present application;

[0065] Figure 10 Schematic diagram of the device structure of the hardware operating environment involved in the human intrusion detection method in the embodiment of the present application.

[0066] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0068] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0069] The main solution of the embodiment of the present application is: perform face recognition on the user to be identified, and determine whether it is necessary to perform posture recognition on the user to be identified based on the face recognition result; if necessary, perform posture recognition on the user to be identified, and determine whether the user to be identified is an outsider based on the posture recognition result; if so, determine whether the user to be identified is in the target prohibited area of ​​the current monitoring environment; if not, determine whether the user to be identified is in the secondary prohibited area of ​​the current monitoring environment, and the secondary prohibited area is determined based on the target prohibited area; obtain the intrusion detection result of the user to be identified based on the judgment result.

[0070] Since the existing technology uses facial recognition technology to detect intrusion, the requirements for the camera's shooting distance, shooting angle and lighting are relatively high. If the distance is too far, the shooting angle is not good or the lighting is unstable during shooting, the accuracy of facial recognition will be affected, which will lead to low accuracy of intrusion detection.

[0071] The present application provides a solution that can combine face recognition and posture recognition to determine whether the user to be identified is an outsider, and when the user to be identified is an outsider and is in a secondary prohibited area but not a prohibited area of ​​the current monitoring environment, obtain an intrusion detection result based on the walking data of the user to be identified, thereby solving the technical problem in the existing technology that the use of face recognition technology for personnel intrusion detection is greatly affected by the external environment, which easily leads to low accuracy of intrusion detection.

[0072] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of implementing the above functions, such as a human intrusion detection device. The following uses a human intrusion detection device as an example (hereinafter referred to as the device) to illustrate this embodiment and the following embodiments.

[0073] Based on this, the present invention provides a method for detecting human intrusion. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the human intrusion detection method of the present application.

[0074] In this embodiment, the human intrusion detection method includes steps S10 to S50:

[0075] Step S10: performing face recognition on the user to be identified, and determining whether gesture recognition is required for the user to be identified based on the face recognition result.

[0076] It is understandable that the above-mentioned user to be identified may be any external user in the current monitoring environment.

[0077] It should be noted that performing face recognition on the user to be identified can be a process of comparing the face of the user to be identified with the face of the authorized user of the current monitoring environment using a face recognition model to determine whether the user to be identified is an outsider. In this embodiment, if the face recognition model determines that the face of the user to be identified successfully matches the face of the authorized user of the current monitoring environment, the personnel intrusion detection is terminated; if the face recognition model determines that the face of the user to be identified fails to match the face of the authorized user of the current monitoring environment, the posture recognition of the user to be identified can continue. Specifically, this embodiment can detect the probability of the face of the user to be identified and the face of the authorized user through the face recognition model. If the probability of the face of the user to be identified and the face of the authorized user matches exceeds a set value (such as 60%), the user to be identified can be determined to be an authorized user of the current monitoring environment, otherwise the user to be identified is determined to be an outsider.

[0078] Step S20: If necessary, perform gesture recognition on the user to be identified, and determine whether the user to be identified is an outsider based on the gesture recognition result.

[0079] It should be noted that performing posture recognition on the user to be identified may be a process of using a posture recognition model to determine whether the user to be identified is an authorized user of the current monitoring environment according to the posture of the user to be identified during walking.

[0080] It should be understood that the above-mentioned external users may be users who are not authorized to enter the current monitoring environment.

[0081] In actual applications, when the face recognition of the user to be identified fails or is not accurate enough, gesture recognition can be used to determine whether the user to be identified is an authorized user. If the user to be identified is not an authorized user, the user to be identified can be determined as an outsider. Specifically, this embodiment can detect the probability of matching the user to be identified with the authorized user through a gesture recognition model. If the matching probability exceeds a set value (such as 60%), the user to be identified can be determined as an authorized user of the current monitoring environment, otherwise the user to be identified is determined to be an outsider. This embodiment combines face recognition and gesture recognition to enable the ability to identify people even in extreme environments, such as insufficient light or when faces cannot be detected, thereby improving the accuracy of person identification and further improving the accuracy of person intrusion detection.

[0082] Step S30: If yes, determine whether the user to be identified is in the target prohibited area of ​​the current monitoring environment.

[0083] It is understandable that the above-mentioned current monitoring environment can be any environment that requires human intrusion detection. The current monitoring environment in this embodiment can include but is not limited to a construction site environment, a laboratory environment, and a campus environment.

[0084] It should be noted that the target prohibited area can be an area in the current monitoring environment where outsiders are prohibited from entering. In this embodiment, a camera can be set up at the boundary between the prohibited area and the safe area to detect whether there is an outsider entering the prohibited area. If so, an alarm is triggered; if not, the camera continues to determine whether there is an outsider entering the secondary prohibited area of ​​the current monitoring environment.

[0085] Step S40: If not, determine whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, where the secondary prohibited area is determined based on the target prohibited area.

[0086] It should be noted that the above-mentioned secondary prohibited area can be an area obtained by expanding the prohibited area in the current monitoring environment. In actual applications, personnel intrusion detection based on intrusion detection technology usually demarcates the prohibited area based on the camera shooting angle, and an alarm and prompt will be issued when an outsider approaches the prohibited area. However, this method may only record the moment of approaching the prohibited area, and thus cannot well predict the trend of on-site events and avoid the occurrence of dangerous events. Therefore, this embodiment can determine a secondary prohibited area based on the prohibited area, and perform personnel intrusion detection in a zoned manner. By expanding the secondary prohibited area based on the original prohibited area and taking the daily speed of people as a benchmark, combined with personnel recognition technology, it is found that when a suspicious person moves quickly in the secondary prohibited area, an alarm will be issued, thereby enhancing the on-site security level and avoiding dangerous events as much as possible.

[0087] Furthermore, before step S40, the method further includes:

[0088] Step S401: Determine the coordinate information of the polygonal frame corresponding to the target prohibited area.

[0089] It should be noted that in this embodiment, an image annotation tool (such as labelme) can be used to annotate the coordinate range of the polygonal box that sets the prohibited area in the current monitoring environment, obtain the coordinate information of the polygonal box corresponding to the target prohibited area, and output the annotation file to facilitate the subsequent construction of the distance measurement algorithm. Among them, the image annotation tool in this embodiment can support a variety of annotation shapes such as polygons, rectangles, circles, line segments, points, etc., and can be used for tasks such as annotating object bounding boxes (Bounding Box), semantic segmentation (Semantic Segmentation), and instance segmentation (Instance Segmentation).

[0090] Step S402: constructing a rectangular frame based on the quadrant maximum value and the quadrant minimum value in the coordinate information.

[0091] It should be noted that the above-mentioned quadrant maximum value can be the maximum value of the coordinate information of the polygonal box corresponding to the target prohibited area in the x quadrant and the y quadrant; the above-mentioned quadrant minimum value can be the minimum value of the coordinate information of the polygonal box corresponding to the target prohibited area in the x quadrant and the y quadrant.

[0092] Step S403: Extending the side length of the rectangular frame to obtain a rectangular frame with extended side lengths.

[0093] Step S404: determining whether the rectangular frame after the length of the extended side exceeds the preset shooting boundary.

[0094] It is understandable that the above-mentioned preset shooting boundary can be the boundary of the video shot by the camera in the current monitoring environment.

[0095] Step S405: If not, the rectangular frame with the extended side length is determined as the secondary prohibited area of ​​the current monitoring environment.

[0096] In the specific implementation, refer to Figure 2 , Figure 2 This is a flow chart for determining secondary prohibited areas in the personnel intrusion detection method of this application. Figure 2 As shown, the device can first obtain the maximum value (x max ,y max ) and quadrant minimum (x min ,y min ), and draw a rectangular box based on the quadrant maximum and quadrant minimum (such as Figure 2 Then, using the rectangle as a standard, the side lengths are extended to twice the original lengths at each vertex to obtain a rectangle with extended sides, and the area where the rectangle with extended sides is located is defined as the secondary prohibited area, where the four points of the rectangle are Then, it can be determined whether the coordinates of the rectangular frame after the side length is extended exceed the boundary of the video shooting. If it exceeds, the vertices of the video shooting boundary are directly taken as the secondary prohibited area.

[0097] Step S50: obtaining an intrusion detection result of the user to be identified based on the judgment result and the walking data of the user to be identified.

[0098] It should be noted that the above-mentioned walking data can be relevant data of the user to be identified when walking in the current monitoring environment, such as walking data, residence time, etc., and this embodiment does not limit this. In this embodiment, when an outsider enters the secondary prohibited area, the walking speed of the user in the secondary prohibited area can be obtained, and when the user's current walking speed is greater than a set value (such as 1.2 times the average walking speed of ordinary workers), it is determined that the user has intrusion behavior. At this time, a loudspeaker can be used for warning, and the time of the incident can be recorded, and the on-duty management personnel can be notified at the same time. This embodiment analyzes the activities of outsiders in the secondary prohibited area, so that when the outsiders move too fast, prompts can be given in advance, thereby avoiding incidents that are not conducive to production and safety. It is safer than setting a threshold for the residence time in the prohibited area.

[0099] In the specific implementation, refer to Figure 3 , Figure 3 This is the overall flow chart of the personnel intrusion detection method of this application. Figure 3As shown, the device uses the cameras in the current monitoring environment to detect in real time whether anyone enters the camera range. If any person does enter the camera range, it first uses the face recognition model to perform facial recognition on the user entering the camera range and outputs the probability that the user's face matches the authorized user's face. If the match probability is greater than 60%, the user is determined to be an authorized user, and the intrusion detection process ends. If the match probability is less than 60%, the posture recognition model is used to perform posture recognition on the user and outputs the probability that the user's walking posture matches the authorized user's walking posture. If the probability is greater than 60%, the user is determined to be an authorized user, and the intrusion detection process ends. If the probability is less than 60%, the user is determined to be an outsider. The device then determines whether the user is in a prohibited area. If so, the process ends and an alarm is issued. If not, the device continues to determine whether the user is in a secondary prohibited area. If so, it determines whether the user's current walking speed is 1.2 times the normal walking speed. If so, the process ends and an alarm is issued. Otherwise, detection and observation continue.

[0100] The present embodiment provides a method for detecting human intrusion, which discloses performing face recognition on a user to be identified, and judging whether it is necessary to perform posture recognition on the user to be identified based on the face recognition result; if necessary, performing posture recognition on the user to be identified, and judging whether the user to be identified is an outsider based on the posture recognition result; if so, judging whether the user to be identified is in a target prohibited area of ​​the current monitoring environment; if not, judging whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, the secondary prohibited area being determined based on the target prohibited area; obtaining an intrusion detection result of the user to be identified based on the judgment result; compared with the prior art method of adopting When using facial recognition technology for personnel intrusion detection, the camera's shooting distance, shooting angle and lighting requirements will affect the accuracy of facial recognition, resulting in low intrusion detection accuracy. Since this embodiment can combine facial recognition and posture recognition to determine whether the user to be identified is an outsider, and when the user to be identified is an outsider and is in the secondary prohibited area of ​​the current monitoring environment but not the prohibited area, the intrusion detection result is obtained based on the walking data of the user to be identified, thereby solving the technical problem in the existing technology that the use of facial recognition technology for personnel intrusion detection is greatly affected by the external environment, which easily leads to low intrusion detection accuracy.

[0101] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 4 , Figure 4 This is a flow chart of the second embodiment of the human intrusion detection method provided by the present application.

[0102] In this embodiment, before step S10, the method further includes steps S011 to S015:

[0103] Step S011: performing data enhancement processing on the facial image corresponding to the authorized user to obtain a target facial image.

[0104] It should be understood that the facial image described above can be a frontal face image of an authorized user in the current monitoring environment captured by the camera. In practical applications, before building a facial recognition model, the device needs to collect facial data of authorized users in the current monitoring environment for model training and construction. Specifically, the authorized user can capture their face in front of the camera to record the facial data. The facial data obtained in this manner can then be used to build the facial recognition model.

[0105] It should be noted that the target facial image may be a facial image obtained by enhancing image data.

[0106] In actual applications, the device can use image enhancement tools (such as Albumentations) to perform data enhancement processing on the facial image corresponding to the authorized user. The image enhancement tool can support basic transformations (such as rotation, scaling, flipping, and cropping) and advanced operations (such as blurring, noise, color distortion, and elastic deformation). In this embodiment, the device can use the image enhancement tool to perform image transformations on the facial image corresponding to the authorized user, including: horizontal flipping, mirror flipping, brightness adjustment, contrast adjustment, etc., to generate more training data to adapt to shooting conditions in different scenes. At the same time, the device can also use a generative adversarial network (GAN) to generate cloudy, snowy, rainy, and night scenes for the image to enhance the generalization ability of the model. At this time, the above-mentioned target facial image can be obtained.

[0107] Step S012: extracting features from the target face image to obtain a face feature vector.

[0108] It is understood that the facial feature vectors described above may be key features extracted from the target facial image for distinguishing different faces. In this embodiment, the device may use a deep neural network (ResNet) to extract features from the target facial image, and a pre-trained deep neural network (ResNet-50) containing 50 convolutional layers to extract image features, ultimately outputting a high-dimensional feature vector with a feature dimension of 2048, i.e., the facial feature vector described above.

[0109] Step S013: performing feature extraction on the point cloud data in the face image to obtain a face point cloud feature vector.

[0110] It should be noted that the above-mentioned facial point cloud feature vector can be a feature vector obtained by processing the facial image through three-dimensional point cloud technology. The feature can be used to describe the facial contour, organ position, etc. of the face. In practical applications, in order to start the posture recognition model to recognize the user's posture when face recognition fails or the accuracy is low, this embodiment can use facial features combined with the features of the millimeter wave point cloud to construct a face recognition model. In this embodiment, the device can first obtain 4D point cloud data corresponding to the facial image from the millimeter wave radar, including spatial coordinates (x, y, z), and filter the point cloud data to remove noise and irrelevant points, retain key feature points, and thus obtain the above-mentioned facial point cloud feature vector.

[0111] Step S014: generating a comprehensive facial feature vector based on the facial feature vector and the facial point cloud feature vector.

[0112] It should be noted that the above-mentioned comprehensive facial feature vector can be a feature vector obtained after feature fusion of the facial feature vector and the facial point cloud feature vector. In practical applications, after obtaining the facial feature vector, the device can convert the point cloud data to the same coordinate system as the visual data for subsequent feature fusion. Then, the device can use a deep learning model (such as PointNet++) to extract features from the point cloud data, and apply the MLP layer point by point to upgrade the point cloud features to a 1024-dimensional space. Among them, PointNet++ can gradually abstract the local and global features of the point cloud data through a hierarchical feature learning method. It uses a pooling operation similar to that in a convolutional neural network to divide the point cloud data into multiple local areas, and extract features from each area, and then aggregate the local features to obtain a higher-level feature representation. Finally, the global features are used for tasks such as classification and segmentation. Since the point cloud features are high-dimensional vectors, in order to ensure the uniform scale of each vector, this embodiment can first perform L2 normalization on the facial point cloud feature vector. In addition, since the image features extracted by ResNet are usually close to the normal distribution, this embodiment can use Z-score normalization to ensure the stability of the vector. Finally, the device can splice the facial point cloud feature vector and the facial feature vector extracted from the image together, and perform batch normalization (BN) to adapt the model to the feature distribution and form a comprehensive facial feature vector of size 3072 dimensions (1024+2048). Among them, L2 normalization refers to dividing each element of the vector by the L2 norm of the vector, so that the L2 norm of the vector is equal to 1; Z-score normalization can be a statistical method for converting data to a mean of 0 and a standard deviation of 1. Its core is to eliminate the dimensional influence of the data through linear transformation and retain the original distribution form for easy comparison and analysis.

[0113] Step S015: training the initial face recognition model based on the comprehensive face feature vector to obtain a target face recognition model.

[0114] It should be understood that the initial face recognition model can be an untrained neural network model. In this embodiment, the comprehensive facial feature vector can be input into the fully connected layer of the initial face recognition model, and then classified through the activation function layer of the initial face recognition model. Finally, the final classification result is output, and a success is defined when the probability output by the initial face recognition model is greater than 60%. Through the above training method, the target face recognition model can be finally obtained.

[0115] The step of performing face recognition on the user to be identified includes:

[0116] Step S101: performing face recognition on the user to be identified using the target face recognition model.

[0117] It is understood that the target face recognition model can be a model used to identify whether a face in the camera image is that of an outsider. In practical applications, to improve the efficiency of intrusion detection, when performing intrusion detection, the device can directly use the trained target face recognition model to perform face recognition on the user to be identified to determine whether the user is an authorized user.

[0118] Furthermore, before step S10, the method further includes steps S021 to S025:

[0119] Step S021: pre-processing the posture video of the authorized user to obtain a plurality of posture walking images, wherein the posture video is a posture video of the authorized user walking at multiple angles.

[0120] It should be understood that the above-mentioned posture video can be a walking video of an authorized user in the current monitoring environment collected by a camera. Figure 5 , Figure 5 This is a diagram showing the user posture video collected in the intrusion detection method of this application. Figure 5 As shown, the authorized user can walk back and forth for a distance of 5 meters in the four directions of the camera at a normal walking speed to obtain the human body posture rules and facial features of the authorized user in real motion state, and obtain the posture video of the authorized user. After recording the face and posture information, object labeling can be performed. The data of the walking process obtained at this time can be used to build a posture recognition model.

[0121] It should be noted that the aforementioned walking posture images can be extracted from a walking posture video. In this embodiment, the device can evenly extract frames from the walking posture video in each turn direction over time to generate images for training, thereby obtaining a number of walking posture images. For example, if each set of turnaround images contains 20 images, and there are four sets of turnaround walks in total, then each person has a total of 80 images.

[0122] Step S022: extract features from the walking images of each posture to obtain joint feature vectors.

[0123] It can be understood that the above-mentioned joint feature vectors may be key features extracted from the walking posture images for distinguishing the walking postures of different users.

[0124] Specifically, the step S022 includes: performing data enhancement processing on each walking posture image to obtain several target posture walking images; extracting human joints from each target posture walking image to obtain several human joints; determining the joint feature values ​​corresponding to each human joint based on the position information of each human joint; and constructing a joint feature vector based on the joint feature values.

[0125] It should be noted that the target walking posture image can be obtained by enhancing the image data of the walking posture image. In practical applications, the device can use image enhancement tools (such as Albumentations) to transform the walking posture image corresponding to the authorized user, including horizontal flipping, mirror flipping, brightness adjustment, and contrast adjustment, to generate more training data to adapt to different shooting scenarios. At the same time, CycleGAN (Cycle Generative Adversarial Network) can be used to generate cloudy, snowy, rainy, and night scenes from the original image to enhance the generalization ability of the model.

[0126] In this embodiment, referring to Figure 6 , Figure 6 This is a diagram showing the human body joints in the human intrusion detection method of this application. Figure 6 As shown, this embodiment can use a deep learning-based human posture estimation framework (such as OpenPose) to extract human body joints of the user, thereby extracting 18 human body joints with relevant features of human body joints, including: left and right eye joints, left and right ear joints, nose joints, left and right shoulder joints, neck joints, left and right elbow joints, left and right wrist joints, left and right hip joints, left and right knee joints, and left and right ankle joints.

[0127] It should be noted that, after obtaining the 18 human joints in the posture walking image, this embodiment can calculate the joint distance matrix based on the position information of the 18 joints, that is, calculate the Euclidean distance between different joints, and use the Euclidean distance between the joints as the joint feature value to construct the joint feature vector. In practical applications, if X(x1, x2, ..., x n ) and Y(y1,y2,…,y n ) to represent the eigenvalues ​​of two joint points, where x and y represent joint points, and n represents the position of the joint point, which is 18. When calculating, the distance between two joint points needs to be deduplicated and the calculation of the joint point needs to be removed. Finally, 153 joint point eigenvalues ​​can be calculated to form the joint point feature vector. Among them, the calculation formula of the Euclidean distance d between joint points can be:

[0128]

[0129] Step S023: performing feature extraction on the point cloud data in the walking images of each posture to obtain joint point cloud feature vectors.

[0130] It should be noted that the joint point cloud feature vectors described above can be obtained by processing the posture walking image using 3D point cloud technology. In practical applications, the device can first obtain the 4D point cloud data corresponding to the posture walking image from the millimeter-wave radar, including spatial coordinates (x, y, z). The point cloud data is then filtered to remove noise and irrelevant points, retaining key feature points, thereby obtaining the joint point cloud feature vectors.

[0131] Step S024: Generate a joint comprehensive feature vector based on the joint feature vector and the joint point cloud feature vector.

[0132] It should be noted that the above-mentioned joint comprehensive feature vector can be a feature vector obtained after feature fusion of the joint feature vector and the joint point cloud feature vector. In practical applications, after obtaining the joint feature vector, the device can convert the point cloud data to the same coordinate system as the visual data for subsequent feature fusion. Then, the device can use a deep learning model (such as PointNet++) to extract features from the point cloud data, and apply the MLP layer point by point to promote the point cloud features to 1024-dimensional space. Since the point cloud features are high-dimensional vectors, in order to ensure the uniform scale of each vector, this embodiment can first perform L2 normalization on the joint point cloud feature vector. In addition, since the image features extracted by ResNet are usually close to a normal distribution, this embodiment can use Z-score standardization to ensure the stability of the vector. Finally, the device can splice the joint point cloud feature vector and the joint feature vector extracted from the image together, and perform batch normalization to finally obtain a joint comprehensive feature vector of size 1177 dimensions (1024+153).

[0133] Step S025: training the initial posture recognition model based on the joint comprehensive feature vector to obtain a target posture recognition model.

[0134] It should be understood that the above-mentioned initial posture recognition model can be an untrained neural network model. In this embodiment, the joint comprehensive feature vector can be input into the fully connected layer of the initial posture recognition model and classified through the activation function layer of the initial face recognition model. Finally, the final classification result is output, and it is defined as a successful judgment when the probability output by the initial posture recognition model is greater than 60%. Through the above-mentioned training method, the target posture recognition model can eventually be obtained. Among them, although the probability threshold for determining positive and negative is generally 50%, due to the need for a higher confidence level in safety scenarios, this embodiment can set the threshold here to 60%.

[0135] The step of performing gesture recognition on the user to be identified includes:

[0136] Step S201: performing gesture recognition on the user to be identified using the target gesture recognition model.

[0137] It is understood that the target posture recognition model can be used to determine whether the walking posture of a user in the camera image is that of an outsider. In practical applications, to improve the efficiency of intrusion detection, if the facial recognition model fails or lacks accuracy during intrusion detection, the device can directly use the trained target posture recognition model to continue posture recognition of the user to be identified to determine whether the user is authorized.

[0138] In this embodiment, when performing human intrusion detection, the device can first use the facial recognition model to make a judgment. If no face is found in the image (usually the back of the person is facing the camera) or if there is no facial recognition result, the posture recognition model is activated. This embodiment uses the dual recognition function to ensure that the recognition model generated based on the data of the full-angle shooting process can also perform recognition from all angles, avoiding the situation where the camera cannot capture the person's facial information due to the shooting angle of the back of the person, such as the back of the person facing the camera. This fills the blind spot of facial recognition and thus improves the detection accuracy of human intrusion detection.

[0139] In this embodiment, data enhancement processing is performed on the facial image corresponding to the authorized user to obtain a target facial image; feature extraction is performed on the target facial image to obtain a facial feature vector; feature extraction is performed on the point cloud data in the facial image to obtain a facial point cloud feature vector; a facial comprehensive feature vector is generated based on the facial feature vector and the facial point cloud feature vector; an initial facial recognition model is trained based on the facial comprehensive feature vector to obtain a target facial recognition model; and facial recognition is performed on the user to be identified through the target facial recognition model. Since this embodiment can pre-train the target facial recognition model based on the facial comprehensive feature vector generated from the facial feature vector and the facial point cloud feature vector, the target facial recognition model can be directly used to determine whether the user to be identified is an authorized user during subsequent personnel intrusion detection, thereby improving the efficiency of personnel intrusion detection.

[0140] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction and will not be described in detail later. Figure 7 , Figure 7 This is a flow chart of the third embodiment of the human intrusion detection method provided by the present application.

[0141] In this embodiment, before step S40, the method further includes steps S411 to S412:

[0142] Step S411: Acquire the authorized user's walking posture data from the posture video.

[0143] It should be noted that the above-mentioned posture walking data may be data of the authorized user walking back and forth in four directions in the posture video.

[0144] Step S412: Determine the average walking speed of the authorized user based on the posture walking data.

[0145] It should be understood that the above average walking data can be the average walking speed of all authorized users within a set time period. In this embodiment, the device can directly obtain the walking speed of the authorized user in the following manner based on the pre-collected posture video. Figure 5 The data of the four directions of return walking are collected, and the walking speed of each authorized user within 1 second is calculated, and then the walking speed of each authorized user is averaged to obtain the above average walking speed. The calculation formula of the average walking speed can be:

[0146]

[0147] Where, is the average walking speed, v i is the walking speed of the authorized user within 1s, and n is the number of authorized users.

[0148] Accordingly, step S50 includes:

[0149] Step S501: If yes, the current walking speed of the user to be identified is determined according to the walking data of the user to be identified.

[0150] It can be understood that the above-mentioned current walking speed may be the walking speed of the user to be identified in the secondary prohibited area.

[0151] Step S502: judging whether the user to be identified has any intrusion behavior based on the current walking speed and the average walking speed, and obtaining an intrusion detection result of the user to be identified.

[0152] In this embodiment, when it is determined that the user to be identified is a foreigner, the speed detection can be performed using the neck joint point. When the user to be identified enters the secondary prohibited area and the current walking speed is greater than 20% of the average walking speed (v high , which is 1.2 times the average speed of ordinary workers), a loudspeaker can be used to warn, record the time of the incident, and notify the on-duty manager at the same time.

[0153] It should be noted that this embodiment can design secondary prohibited areas based on the prohibited areas already designated in the actual monitoring environment to perform intrusion detection. The secondary prohibited areas are analyzed and prompted based on speed, while the prohibited areas are analyzed and prompted based on residence time. Specifically, if the walking speed of an outsider in the secondary prohibited area is greater than 20% of the average speed of authorized users, a loudspeaker can be used to issue a warning, the time of the incident can be recorded, and the on-duty management personnel can be notified. If an outsider approaches the prohibited area at a normal speed and any of their joints are in the danger zone, a loudspeaker warning can be issued, the time of the incident can be recorded, and the management personnel can be notified.

[0154] In the specific implementation, refer to Figure 8 , Figure 8 This is a diagram showing the steps of the human intrusion detection method of this application. Figure 8 As shown, the facial information and posture features of the user authorized to monitor the environment can be obtained and entered first. Then, a face recognition model and a posture recognition model can be constructed based on the facial information and posture features, respectively. Specifically, the device can first perform data enhancement on the facial image of the authorized user to obtain the target facial image, and perform feature extraction on the target facial image to obtain a facial feature vector. At the same time, point cloud feature extraction is performed on the facial image to obtain a facial point cloud feature vector. The facial feature vector and the facial point cloud feature vector are then feature fused, and the face separator model is trained based on the fused facial comprehensive feature vector to obtain a facial recognition model. At the same time, the device can preprocess the authorized user's posture video to obtain several posture walking images, and then perform data enhancement processing on the posture walking images to obtain the target posture walking images. The device can then extract joint points from the target posture walking images and calculate joint point feature values ​​based on the extracted joint point position information to construct joint point feature vectors. At the same time, the device can extract millimeter-wave radar point cloud features to obtain joint point cloud feature vectors, fuse the joint point feature vectors with the joint point cloud feature vectors, and then train the posture separator model based on the fused joint comprehensive feature vectors to obtain a posture recognition model. When performing human intrusion detection, the camera can be used to detect in real time whether there are people entering the current monitoring environment. If any person enters the camera range, the face recognition model and the posture recognition model can be used for inference. Specifically, the face recognition model can be used to determine whether the person is an outsider. If face recognition fails, the posture recognition model can be used to determine whether the person is an outsider. If so, the pre-built prohibited area intrusion algorithm is used to further detect whether the person has intruded. The prohibited area intrusion algorithm can be used to determine the behavior of outsiders in prohibited and sub-prohibited areas, analyzing different behaviors in different areas. When abnormal behavior is detected, an alarm is issued to prevent dangerous incidents and losses. First, monitoring equipment (such as cameras) can be deployed at the boundary between the prohibited area and the safe area. Then, the prohibited area can be expanded to calculate the sub-prohibited area. The average walking speed of the authorized user is then calculated based on the authorized user's walking data in the posture video. If an outsider enters the sub-prohibited area and the walking speed is greater than 1.2 times the average walking speed of the authorized user, a loudspeaker can be used to issue a warning. In addition, if the outsider approaches the prohibited area at a normal speed and any of their joints are in the dangerous area, a loudspeaker warning can be issued.

[0155] In this embodiment, the method of obtaining the posture walking data of the authorized user from the posture video is disclosed; determining the average walking speed of the authorized user based on the posture walking data; if yes, determining the current walking speed of the user to be identified based on the walking data of the user to be identified; judging whether the user to be identified has intrusive behavior based on the current walking speed and the average walking speed, and obtaining the intrusion detection result of the user to be identified; because this embodiment can judge whether the user to be identified has intrusive behavior based on the activities of the user to be identified in the sub-prohibited area when the user to be identified is an outsider and is in the sub-prohibited area of ​​the current monitoring environment but not the prohibited area, the safety of the environment can be ensured.

[0156] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the personnel intrusion detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0157] This application also provides a human intrusion detection device, please refer to Figure 9 , the personnel intrusion detection device includes:

[0158] The face recognition module 10 is used to perform face recognition on the user to be identified and determine whether gesture recognition is required for the user to be identified based on the face recognition result;

[0159] A gesture recognition module 20 is configured to perform gesture recognition on the user to be identified, if necessary, and determine whether the user to be identified is an outsider based on the gesture recognition result;

[0160] an intrusion detection module 30 for determining whether the user to be identified is in a target prohibited area of ​​the current monitoring environment if yes;

[0161] The intrusion detection module 30 is further configured to, if not, determine whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, wherein the secondary prohibited area is determined based on the target prohibited area;

[0162] The detection result acquisition module 40 is used to obtain the intrusion detection result of the user to be identified based on the judgment result and the walking data of the user to be identified.

[0163] The intrusion detection device provided in this application, which utilizes the intrusion detection method of the aforementioned embodiment, can resolve the technical problem in the prior art of employing facial recognition technology for intrusion detection, which is significantly affected by the external environment and easily leads to low intrusion detection accuracy. Compared with the prior art, the beneficial effects of the intrusion detection device provided in this application are the same as those of the intrusion detection method provided in the aforementioned embodiment, and the other technical features of the intrusion detection device are the same as those disclosed in the aforementioned embodiment and are not further described here.

[0164] The present application provides a human intrusion detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the human intrusion detection method in the above-mentioned embodiment one.

[0165] Reference below Figure 10 , which shows a schematic structural diagram of a human intrusion detection device suitable for implementing an embodiment of the present application. The human intrusion detection device in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 10 The human intrusion detection device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0166] like Figure 10 As shown, the human intrusion detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the human intrusion detection device. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the human intrusion detection device to communicate with other devices wirelessly or wired to exchange data. Although the figure shows a human intrusion detection device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems can be implemented or provided instead.

[0167] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0168] The intrusion detection device provided in this application utilizes the intrusion detection method described in the aforementioned embodiment to address the technical issues surrounding intrusion detection. Compared to the prior art, the beneficial effects of the intrusion detection device provided in this application are the same as those of the intrusion detection method described in the aforementioned embodiment. Other technical features of the intrusion detection device are the same as those disclosed in the aforementioned embodiment and are not further elaborated upon here.

[0169] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0170] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0171] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer program) stored thereon, and the computer-readable program instructions are used to execute the human intrusion detection method in the above embodiment.

[0172] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0173] The computer-readable storage medium may be included in the personnel intrusion detection device; or may exist independently without being assembled into the personnel intrusion detection device.

[0174] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the personnel intrusion detection device, the personnel intrusion detection device enables the personnel intrusion detection device to: perform face recognition on the user to be identified, and determine whether it is necessary to perform posture recognition on the user to be identified based on the face recognition result; if necessary, perform posture recognition on the user to be identified, and determine whether the user to be identified is an outsider based on the posture recognition result; if so, determine whether the user to be identified is in the target prohibited area of ​​the current monitoring environment; if not, determine whether the user to be identified is in the secondary prohibited area of ​​the current monitoring environment, and the secondary prohibited area is determined based on the target prohibited area; obtain the intrusion detection result of the user to be identified based on the judgment result.

[0175] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0176] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0178] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned human intrusion detection method. This computer-readable storage medium can address the technical problem in the prior art that human intrusion detection using facial recognition technology is significantly affected by the external environment, which can easily lead to low intrusion detection accuracy. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the human intrusion detection method provided in the aforementioned embodiment, and are not further elaborated here.

[0179] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned human intrusion detection method when executed by a processor.

[0180] The computer program product provided in this application can address the technical problem in the prior art of using facial recognition technology for intrusion detection, which is significantly affected by the external environment and easily leads to low intrusion detection accuracy. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the intrusion detection method provided in the above-mentioned embodiment, and will not be elaborated here.

[0181] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for detecting human intrusion, characterized in that: The method includes: Performing facial recognition on the user to be identified, and determining whether gesture recognition is required for the user to be identified based on the facial recognition result; If necessary, performing gesture recognition on the user to be identified, and judging whether the user to be identified is an outsider according to the gesture recognition result; If yes, determining whether the user to be identified is in the target prohibited area of ​​the current monitoring environment; If not, determining whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, where the secondary prohibited area is determined based on the target prohibited area; An intrusion detection result of the user to be identified is obtained according to the judgment result and the walking data of the user to be identified.

2. The method according to claim 1, wherein Before the step of performing face recognition on the user to be identified, the method further includes: Perform data enhancement processing on the facial image corresponding to the authorized user to obtain the target facial image; Performing feature extraction on the target face image to obtain a face feature vector; Performing feature extraction on the point cloud data in the face image to obtain a face point cloud feature vector; Generate a comprehensive facial feature vector based on the facial feature vector and the facial point cloud feature vector; Training an initial face recognition model based on the comprehensive face feature vector to obtain a target face recognition model; The step of performing face recognition on the user to be identified includes: The target face recognition model is used to perform face recognition on the user to be identified.

3. The method according to claim 2, wherein Before the step of performing face recognition on the user to be identified, the method further includes: Preprocessing the posture video of the authorized user to obtain a plurality of posture walking images, wherein the posture video is a posture video of the authorized user walking at multiple angles; Extract features from walking images of different postures to obtain joint feature vectors; Extracting features from the point cloud data in the walking images of each posture to obtain joint point cloud feature vectors; Generate a joint comprehensive feature vector based on the joint feature vector and the joint point cloud feature vector; Training an initial posture recognition model based on the joint comprehensive feature vector to obtain a target posture recognition model; The step of performing gesture recognition on the user to be identified includes: Performing posture recognition on the user to be identified using the target posture recognition model.

4. The method according to claim 3, wherein The step of extracting features from the walking images of each posture to obtain joint feature vectors includes: Perform data enhancement processing on the walking images of each posture to obtain several walking images of the target posture; Extract human joint points from each target walking posture image to obtain several human joint points; Determine the joint point feature value corresponding to each human joint point based on the position information of each human joint point; A joint feature vector is constructed based on the joint point feature values.

5. The method according to any one of claims 1 to 4, characterized in that Before the step of determining whether the user to be identified is in the secondary prohibited area of ​​the current monitoring environment, the method further includes: Determine coordinate information of the polygonal frame corresponding to the target prohibited area; Constructing a rectangular frame based on the quadrant maximum value and the quadrant minimum value in the coordinate information; Extending the side length of the rectangular frame to obtain a rectangular frame with extended side lengths; Determining whether the rectangular frame after the side length is extended exceeds a preset shooting boundary; If not, the rectangular frame with the extended side length is determined as the secondary prohibited area of ​​the current monitoring environment.

6. The method according to claim 3, wherein Before the step of determining whether the user to be identified is in the target prohibited area of ​​the current monitoring environment, the method further includes: Acquire the posture walking data of the authorized user from the posture video; determining an average walking speed of the authorized user based on the posture walking data; The step of obtaining the intrusion detection result of the user to be identified based on the judgment result and the walking data of the user to be identified includes: If so, determining the current walking speed of the user to be identified based on the walking data of the user to be identified; Based on the current walking speed and the average walking speed, it is determined whether the user to be identified has intrusion behavior, and an intrusion detection result of the user to be identified is obtained.

7. A human intrusion detection device, characterized in that: The device comprises: A face recognition module is used to perform face recognition on a user to be identified and determine whether gesture recognition is required for the user to be identified based on the face recognition result; a gesture recognition module, configured to perform gesture recognition on the user to be identified, if necessary, and determine whether the user to be identified is an outsider based on the gesture recognition result; an intrusion detection module, configured to determine whether the user to be identified is in a target prohibited area of ​​the current monitoring environment; The intrusion detection module is further configured to, if not, determine whether the user to be identified is in a secondary prohibited area of ​​the current monitoring environment, where the secondary prohibited area is determined based on the target prohibited area; The detection result acquisition module is used to obtain the intrusion detection result of the user to be identified based on the judgment result and the walking data of the user to be identified.

8. A human intrusion detection device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the human intrusion detection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the human intrusion detection method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the human intrusion detection method according to any one of claims 1 to 6 are implemented.

Images