Indoor safety monitoring system and method based on image recognition

Abstract

The invention relates to the technical field of image recognition analysis, in particular to an indoor safety monitoring system and method based on image recognition, and the system comprises a plurality of modules and a lower setting unit, an image collection module selects a high-frame-rate and high-resolution camera, and is connected with a rear-end processing device through a USB, Ethernet or Wi-Fi communication interface; according to the scheme, after an owner receives an alarm and checks monitoring content, the owner can manually cancel the alarm through visual judgment of a door condition, after the alarm is cancelled, the system automatically enters the dangerous action capture module, and the module focuses on danger assessment of personnel actions and temporarily ignores other processes such as personnel identity recognition within a certain time after being started, so that the system is more convenient to use. During the period, if a new person enters the monitoring area, the system only carries out dangerous action capture, such setting can filter a large amount of alarm information generated due to frequent person contact to a certain extent, reduce interference to owners, and ensure that potential dangerous actions are kept alert at the same time.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition and analysis, and in particular to an indoor security monitoring system and method based on image recognition. Background Art

[0002] An indoor security monitoring system and method based on image recognition is an intelligent solution designed for indoor security protection needs. In modern life, indoor security issues have attracted much attention, and people need effective monitoring methods to ensure the safety of their living environment;

[0003] Although traditional analysis systems use area limitation and intrusion analysis alarms, they cannot prevent people from passing by or staying without malicious intent. Frequent alarms will inevitably affect the life and work of the owner. If the alarm is turned off directly, it may cause property losses.

[0004] To this end, the present invention proposes an indoor security monitoring system and method based on image recognition. By collecting images through a high-frame rate and high-resolution camera, combined with modules such as time and position analysis, face recognition, and dangerous motion capture, accurate identification and behavior analysis of personnel can be achieved, and the identity of the personnel and whether there are dangerous actions can be effectively judged. At the same time, the system gives the owner the power of manual intervention under specific circumstances. For example, the owner can cancel the alarm and enable the dangerous motion capture module based on actual visual judgment, and the module only captures dangerous motions within the set time, avoiding unnecessary alarms in special circumstances, greatly improving the accuracy, flexibility and practicality of the system, and providing more reliable and intelligent protection for indoor safety. Summary of the invention

[0005] Technical problem to be solved: Although the traditional analysis system uses area limitation and personnel intrusion analysis alarm, it cannot avoid situations such as people passing by and staying without malicious intent.

[0006] In view of the deficiencies in the prior art, the present invention provides an indoor security monitoring system and method based on image recognition, thereby solving the technical problems mentioned in the background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] An indoor security monitoring system based on image recognition, the system comprises the following modules and subordinate units:

[0009] The image acquisition module uses a high-frame rate, high-resolution camera, which is connected to the back-end processing equipment through a communication interface such as USB, Ethernet or Wi-Fi. It continuously collects video image data of the door area and converts it into digital images for transmission to subsequent modules;

[0010] The time-position analysis module includes a position recognition unit and a time analysis unit. The position recognition unit is used to define the monitoring area, identify and track human targets entering the area, and the time analysis unit counts the time of the targets entering the area. When the residence time exceeds a set threshold, a trigger signal and real-time image data are sent to the facial recognition module.

[0011] The facial recognition module includes a feature extraction unit and a comparison and matching unit. The feature extraction unit uses a convolutional neural network architecture to extract facial feature vectors. The comparison and matching unit compares the vectors with feature vectors in a pre-established face database to determine the identity of the person.

[0012] The alarm notification module includes an alarm triggering unit and an information sending unit. The alarm triggering unit determines whether to trigger an alarm based on the identity recognition result of the facial recognition module and the judgment result of the dangerous action capture module. The information sending unit sends the abnormal information to the mobile phones of family members through the network communication protocol.

[0013] After receiving manual instructions, the dangerous action capture module uses human posture estimation technology to identify joint points based on the OpenPose model, judges dangerous actions according to preset rules, and sends signals to the alarm notification module.

[0014] In a possible implementation, in the position identification unit, an operator manually defines a monitoring area through a system software interface, and the system uses an edge detection algorithm, a target detection algorithm, and a target tracking algorithm to identify and track a human target.

[0015] In one possible implementation, after the alarm notification module of the system has reviewed the monitoring content, it can cancel the alarm if it visually determines that there is no danger, and enter the dangerous motion capture module at the same time. When the dangerous motion capture module is enabled, only dangerous motion capture will be performed within a certain period of time, and if other people enter the monitoring area, only dangerous motion capture will also be performed.

[0016] In a possible implementation, a monitoring method of an indoor security monitoring system based on image recognition is performed, and the method includes the following steps:

[0017] S1, image acquisition, using a high frame rate, high resolution camera to collect video image data of the door area and convert it into digital image transmission;

[0018] S2, time-position analysis, defining the monitoring area, identifying and tracking human targets entering the area, timing the entry of targets, and triggering the facial recognition process if it exceeds 10 seconds;

[0019] S3, facial recognition, extracting facial feature vectors, comparing them with feature vectors in the database, and determining the identity of the person;

[0020] S4, alarm notification, decide whether to trigger the alarm based on the results of identity recognition and dangerous action judgment, and send abnormal information to the mobile phones of family members;

[0021] S5, dangerous action capture, after receiving manual instructions, identify joint points, judge dangerous actions according to preset rules and send signals.

[0022] Beneficial effects compared with the prior art:

[0023] 1. In this solution, after receiving the alarm and checking the monitoring content, the owner can manually cancel the alarm by visually judging the situation at the door. After canceling the alarm, the system automatically enters the dangerous action capture module. Within a certain period of time after activation (such as the set 30 minutes), this module will focus on the dangerous assessment of human actions and temporarily ignore other processes such as human identity recognition. During this period, if a new person enters the monitoring area, the system will only perform dangerous action capture. This setting can filter out a large amount of alarm information generated by frequent personnel exchanges (such as weddings at neighbors' homes, etc.), reduce interference to the owner, and ensure vigilance against potential dangerous actions;

[0024] 2. In this solution, the image acquisition module uses a high frame rate and high resolution camera to provide clear and accurate original image data for subsequent analysis, ensuring that the system accurately identifies the characteristics and behaviors of the target object. The position recognition unit in the time-position analysis module manually defines the monitoring area and combines edge detection, target detection and tracking algorithms to effectively screen out targets related to indoor safety and avoid invalid monitoring. The time analysis unit sets the retention time threshold to further identify people who stay for a long time, improve the system security and accuracy, and avoid excessive response to people who stay for a short time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings.

[0026] Figure 1 This is a system framework diagram of an indoor security monitoring system based on image recognition;

[0027] Figure 2 The figure is a flowchart of the steps of an indoor security monitoring method based on image recognition. DETAILED DESCRIPTION

[0028] Preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the present invention can also be implemented in various forms, so the present invention is not limited to the embodiments described below.

[0029] The technical solution in the embodiment of the present application is to solve the problems of the above-mentioned background technology, and the overall idea is as follows:

[0030] Embodiment 1:

[0031] Please refer to Figure 1 As shown, this embodiment introduces an indoor security monitoring system based on image recognition, including the following modules and units:

[0032] 1. Image acquisition module

[0033] Provide the most original image data for the entire system. These data are the basis for all subsequent analysis and judgment. High-quality image acquisition can ensure that the system can accurately identify the characteristics and behaviors of the target object;

[0034] The image acquisition module uses a high-frame-rate, high-resolution camera. For example, the frame rate is set to 30 frames per second and the resolution is 1920×1080 pixels. The camera is connected to the back-end processing device (such as a local server or edge computing device) through a communication interface such as USB, Ethernet or Wi-Fi. The camera continuously collects video image data of the door area at the set frame rate, converts the optical signal into an electrical signal, and then transmits it to the subsequent module in the form of a digital image after analog-to-digital conversion;

[0035] 2. Time and location analysis module, which is used to determine the time a person stays in a designated area and judge whether further identification is required. It consists of a location identification unit and a time analysis unit;

[0036] 2.1. The location identification unit clearly defines the spatial scope that needs to be monitored. Only people who enter the area will have their stay time further analyzed by the system, thereby effectively screening out targets that may be related to indoor security and avoiding ineffective monitoring of irrelevant personnel;

[0037] In the early stage of system deployment, operators manually define the designated monitoring area on the image captured by the camera through the system software interface using the mouse or touch operation. For example, a rectangular area with a width of 1.5 meters and a height of 2 meters is defined as the monitoring range with the door frame as a reference. The system uses the edge detection algorithm in image processing to identify the outline of objects in the image, and uses the target detection algorithm to identify the human target. When the human outline is detected entering the defined rectangular area, the system uses the target tracking algorithm to continuously track the human target, providing accurate target information for subsequent time analysis.

[0038] The above edge detection algorithm specifically takes the Sobel edge detection algorithm as an example. The Sobel algorithm calculates the horizontal and vertical gradients respectively by applying convolution kernels in the x and y directions. The gradient amplitude reflects the intensity of the change in pixel value, and the gradient direction indicates the direction of the change. By setting a threshold, points with larger gradient amplitudes are selected as edge points, thereby outlining the contour of the object. First, define the Sobel operator in the x direction (G x ) and y direction (G y )’s convolution kernel: For each pixel (x, y) in the image, its pixel value is I(x, y), and after convolution with the Sobel operator, the approximate gradient in the x and y directions is obtained: Then calculate the gradient magnitude M(x,y) and direction θ(x,y), Usually, the gradient amplitude is thresholded and a threshold T is set. When M(x,y)>T, the point is considered to be an edge point.

[0039] For example, for the door frame and human body at the door, the pixel values ​​between the door frame and the background, and between the human body and the background in the image vary significantly. After being processed by the Sobel algorithm, the edges of the door frame and the human body will have a larger gradient amplitude. Through threshold screening, these edge points are retained, thereby highlighting the outlines of the door frame and the human body, helping the system to accurately identify human targets in the future. For example, in an image of a doorway, after the edge of the human body is processed by the Sobel algorithm, the gradient amplitude is higher at the human body outline. Based on this, the system can preliminarily outline the human body's shape, providing a basis for subsequent target detection and tracking.

[0040] 2.2. The time analysis unit sets the retention time threshold to screen out people who stay at the door for a long time. These people may pose potential security risks and need further identity recognition, thereby improving the security and accuracy of the system and avoiding over-response to normal visitors who stay for a short time;

[0041] From the moment the target is identified and enters the set area, the system starts a high-precision timer (such as a timing mechanism based on a computer system clock) to start timing. While the target is in the area, the target's position information is monitored in real time through the target tracking algorithm. If the target leaves the set area, the system stops timing and clears the relevant timing information of the target. If the target stays in the area for a cumulative total of 10 seconds or more, the system generates a trigger signal, which will be transmitted to the facial recognition module to trigger the facial recognition process;

[0042] The above target tracking algorithm takes the Kalman filter tracking algorithm as an example. Assume that the target state vector X contains the position (x coordinate, y coordinate) and the velocity (v x、v y ) information, that is, X = [x, y, v x ,v y ] T . State transfer matrix F k It can be expressed as: Where Δt is the time interval, and the observation vector Z may only contain the location information of the target, that is, Z = [x obs ,y obs ] T , the observation matrix H k for: At the initial moment, set the initial state estimate of the target and the initial covariance P 0|0 As time goes by, each time a new frame of image data is acquired (i.e., a new observation value Z k ), first perform the prediction step and get the prediction state and the predicted covariance P k|k-1 , and then calculate the Kalman gain K by combining the observations through the update step k , and then get the optimal state estimate In this way, the position and speed estimation of the target are continuously corrected to achieve accurate tracking of human targets;

[0043] For example, at a certain moment, the predicted target position is (100, 150), while the observed target position is (105, 155). Through the update step of the Kalman filter, the predicted value and the observed value are comprehensively considered to obtain a more accurate target position estimate, and the position of the person is accurately marked with a rectangular frame on the image screen, and a unique identification number (such as ID001) is assigned for subsequent tracking and analysis;

[0044] Assume that visitor ID001 enters the monitoring area at 10:00:00 am, the system starts timing, and at 10:00:12 am, the visitor is still in the area, satisfying the condition of more than 10 seconds, and the system sends a trigger signal to the facial recognition module, along with the real-time image data of the visitor;

[0045] 3. Facial recognition module, used to identify the identity of a person and determine whether he is a foreigner. It consists of a feature extraction unit and a comparison and matching unit;

[0046] 3.1. Feature extraction unit, which converts facial images into digital feature vectors. These feature vectors are key data for subsequent identity comparison and can accurately reflect the unique features of the face, enabling the system to determine the identity of a person by comparing the feature vectors;

[0047] This unit adopts the convolutional neural network (CNN) architecture in deep learning. When receiving the trigger signal sent by the position and time analysis module and the image of the relevant person, the image is input into the CNN model. The CNN model automatically extracts the key features of the face through the operation of multiple convolutional layers, pooling layers and fully connected layers. The convolutional layer uses the convolution kernel to slide on the image to extract features of different scales and directions. The pooling layer downsamples the feature map to reduce the amount of data while retaining the key features. The fully connected layer integrates the extracted features and finally outputs a feature vector of fixed dimension, which can uniquely represent the feature information of the face.

[0048] As mentioned above, the convolutional neural network automatically extracts the features of the data through the combination of different layers. The convolution layer uses the convolution kernel to slide on the input feature map to perform convolution operations. Each convolution kernel can capture specific local features. Different combinations of convolution kernels can extract features of different scales and directions. The pooling layer downsamples the feature map. By taking the maximum value (such as maximum pooling) or the average value (such as average pooling) of the local area, the main features are retained while reducing the amount of data, reducing the amount of calculation and model complexity. The fully connected layer integrates the features extracted by the previous layer and maps the features to the category space of the sample for the final classification or regression task. In the feature extraction unit of the face recognition module, CNN converts the facial image into a feature vector that can uniquely represent the facial features through the operations of these layers;

[0049] Convolutional layer: Assume the input feature map is X, and its size is W in ×H in ×C in (width, height, number of channels), the size of the convolution kernel K is w×h×C in ×C out (convolution kernel width, height, number of input channels, number of output channels), the step size is s, the padding is p, and the size of the output feature map Y is W out ×H out ×C out ,in: For each element Y(i,j,k) in the output feature map Y (i is the row index of the output feature map, j is the column index, and k is the output channel index), the calculation method is:

[0050] Pooling layer (taking maximum pooling as an example): Assume that the input feature map is X and the size is W in ×H in ×C, the pooling window size is w×h, and the step size is s. The size of the output feature map is W out ×H out ×C, where: For each element Y(i,j,k) in the output feature map Y, the calculation method is:

[0051] Fully connected layer: Assume the input vector is X and the dimension is d in , the weight matrix is ​​W, and the dimension is d out ×d in , the bias vector is b and the dimension is d out , then the dimension of the output vector y is d out , the calculation method is: y = Wx + b;

[0052] For example, in the feature extraction unit, when a facial image (assuming it is an RGB image, C in =3, size 100×100) to the CNN model, first through a convolutional layer, the convolution kernel size w=3, h=3, C out = 16, step size s = 1, padding p = 1, then the output feature map size H out =100, C out = 16, the convolution layer extracts local features of the face through convolution operations, such as the edges and textures of the eyes and nose. Then it passes through a maximum pooling layer with a pooling window size of w = 2, h = 3, and a step size of s = 2. The output feature map size becomes H out =50, C out =16, the amount of data is reduced. After multiple convolutional layers and pooling layers are alternating, a low-dimensional feature representation is obtained. These features are then integrated through the fully connected layer, and finally a 4096-dimensional feature vector is output. This vector contains key feature information such as the proportion of facial features and contour shape, which is used for subsequent identity comparison;

[0053] 3.2. The comparison and matching unit accurately determines the identity of the person based on the comparison results of the facial feature vectors and determines whether it is an intruder, providing an important basis for subsequent alarm decisions;

[0054] The system pre-establishes a face database, which stores the facial feature vectors of all authorized family members and their corresponding identity identifiers (such as name, relationship with the head of the household, etc.). When the feature extraction unit outputs the facial feature vector of the person to be identified, the comparison and matching unit compares the vector with all feature vectors in the database. The comparison process uses a similarity measurement algorithm, such as a cosine similarity algorithm, to obtain a set of similarity scores by calculating the cosine similarity value between the feature vector to be identified and each feature vector in the database. The system sets a similarity threshold, such as 0.8. If the highest similarity score is greater than or equal to the threshold, the person is determined to be an authorized family member in the database. If the highest similarity score is less than the threshold, the person is determined to be an outsider.

[0055] The above similarity measurement algorithm takes the cosine similarity algorithm as an example. For two n-dimensional vectors and The calculation formula for cosine similarity is: Cosine similarity is used to measure the degree of similarity between two vectors in direction. Its value ranges from -1 to 1. The closer the value is to 1, the more similar the directions of the two vectors are, that is, the more similar the features represented by the two vectors are; the closer the value is to 0, the greater the difference in the directions of the two vectors is, and the less similar the features are; a value of -1 means that the directions of the two vectors are completely opposite. For example, in the comparison matching unit, assuming that the facial feature vector of visitor ID001 is The feature vector of family member A in the database is Calculate their cosine similarity: If the calculated cosine similarity is 0.92, which is greater than the set similarity threshold of 0.8, the system determines that the visitor is family member A. If the highest similarity calculated with all feature vectors in the database is only 0.75, which is less than the threshold of 0.8, the visitor is determined to be an outsider. This indicates that the facial feature vector of visitor ID001 is similar to the feature vector of family member A in direction. Compared with other member vectors in the database, if the highest similarity is lower than the threshold, it means that its features are quite different from those of authorized members and it may be an outsider.

[0056] 4. Alarm notification module, which determines and sends alarms based on abnormal information, and includes an alarm triggering unit and an information sending unit;

[0057] 4.1. The alarm trigger unit is used to comprehensively analyze the abnormal information transmitted from different modules and decide whether to trigger the alarm. It is the decision-making center for the entire system to issue an alarm;

[0058] This unit receives the identity recognition result from the facial recognition module and the dangerous action judgment result from the dangerous action capture module (to be introduced later). If the facial recognition determines that an outsider has intruded (i.e., the similarity is lower than the threshold), or the identity of the person cannot be recognized and the retention time exceeds 10 seconds, or the dangerous action capture module detects a dangerous action, the alarm trigger unit immediately generates an alarm signal;

[0059] For example, when the facial recognition module determines that the person at the door is a stranger, the alarm trigger unit immediately generates a high-level alarm signal after receiving the result, and starts the subsequent alarm sending process;

[0060] 4.2. Information sending unit, used to promptly notify family members of abnormal situations detected by the system, so that they can understand the safety status of the door at the first time and view relevant image information through mobile phones for further judgment;

[0061] When the alarm trigger unit generates an alarm signal, the information sending unit first prepares the alarm information to be sent, including the alarm type (such as an outsider breaking in, detecting a dangerous action, etc.), the time and location of the event (the location information of the camera at the door), and related image data (such as a real-time screenshot of the door when the alarm is triggered). Then, this information is sent to the mobile phone of the family member through the network communication protocol. The system supports multiple network communication protocols, such as MQTT (Message Queue Telemetry Transport Protocol). The monitoring application installed on the mobile phone of the family member subscribes to the topic as an MQTT client, so as to receive the alarm information in real time.

[0062] Assuming that the system detects an outsider breaking in, the information sending unit will send the text message "outsider breaking in, time is 10:30:00 on January 1, 2025, location is the doorstep" and the current real-time screen screenshot of the doorstep to the family member's mobile phone application through the MQTT protocol. After the family member's mobile phone receives the notification, a prompt box pops up on the screen to display relevant information. Click to view and you can immediately call up the real-time monitoring screen;

[0063] After checking the monitoring content, the owner can cancel the alarm if he / she visually judges that there is no danger, and enter the dangerous motion capture module at the same time. When the dangerous motion capture module is enabled, only dangerous motion capture will be performed within a certain period of time. The specific setting can be set according to the needs. For example, within 30 minutes, if another person enters the monitoring area during this period of time, only dangerous motion capture will be performed. This setting can avoid the system from constantly issuing alarms in special circumstances. For example, when there is a wedding at a neighbor's house, there are often people in the monitoring area. In this case, this setting can avoid the problem of continuous alarms.

[0064] 5. Dangerous action capture module, which is used to identify and judge potentially dangerous human actions in special circumstances, further improve the system's safety monitoring function, and provide more comprehensive safety protection for family members;

[0065] When family members use the mobile monitoring application to determine that there are strangers lingering at the door but there are no obvious signs of danger, they can manually send instructions to start the dangerous action capture module. This module first uses human posture estimation technology and the OpenPose model based on deep learning. The OpenPose model processes the input video images and can identify the position coordinates of multiple joints of the human body (such as head, shoulders, elbows, wrists, hips, knees, ankles, etc.). Then, according to the real-time position and movement trajectory of these joints, combined with the preset dangerous action rules, analysis and judgment are performed. For example, the preset rule is that when the hand joints quickly approach the door lock position and rotate, or the arms are raised for a long time and the arms are waving quickly, it is judged as a dangerous action. The system calculates the distance between the joints, the angle change, the movement speed and other parameters, and compares them with the threshold in the preset rules to determine whether there is a dangerous action. If a dangerous action that meets the rules is detected, a dangerous action signal is sent to the alarm notification module;

[0066] The core of the above OpenPose model is to detect human joints based on convolutional neural networks. During the training process, a loss function is used to optimize the model parameters to minimize the difference between the predicted joint position and the actual joint position. Commonly used loss functions include the mean square error loss function (MSE): Where N is the number of samples, J is the number of joint points, is the predicted position of the jth joint point in the ith sample, is the real position of the jth joint in the ith sample. In actual operation, the model predicts the position of the joint through a series of operations such as convolution, pooling and deconvolution. For example, the image features are extracted through the convolution layer, and the feature map is restored to a size related to the original image size through the deconvolution layer. The confidence of the corresponding joint point is predicted at each position on this feature map. The OpenPose model can process the input video image and accurately identify the position coordinates of multiple joints of the human body (such as head, shoulder, elbow, wrist, hip, knee, ankle, etc.). It is based on the architecture of deep learning and is trained through a large amount of image data with relevant node annotations to learn the mapping relationship between the human body posture and the position of the joints in the image. In the dangerous action capture module, the real-time position and motion trajectory of these joints are used, combined with the preset dangerous action rules, and the distance between the joints, angle change, and motion speed and other parameters are calculated, and compared with the threshold in the preset rules to determine whether there is a dangerous action.

[0067] In the dangerous action capture module, when a stranger at the door holds a stick-like object for a long time and raises his arms and swings them quickly. The OpenPose model processes the input video image, first extracts the features of the human body in the image through the convolution layer, and then predicts the position coordinates of each joint point (such as the shoulder, elbow, wrist and other arm joint points) through operations such as deconvolution. Over time, the model tracks the motion trajectory of these joint points, calculates the speed of the arm swing (for example, by dividing the change in the position of the joint points in adjacent frames by the time interval) and the angle change (for example, by calculating the angle change between the lines between different joint points). Assuming that the preset rules stipulate that the arm swing speed exceeds a certain threshold and the angle change is within a specific range, it is a dangerous action. The model compares the calculated parameters with the threshold in the preset rules. If it meets the rules, it is judged that the behavior meets the dangerous action standard and sends a dangerous action signal to the alarm notification module. For example, the arm swing speed is calculated to be 3 radians per second, which exceeds the preset threshold of 2 radians per second, and the angle change is between 90° and 180°, which meets the preset dangerous action rules, thereby triggering a dangerous action signal.

[0068] Embodiment 2:

[0069] Based on the above-mentioned embodiment 1, an indoor security monitoring system based on image recognition, this embodiment provides an indoor security monitoring method based on image recognition, and the method steps are as follows:

[0070] S1, image acquisition, select a camera with high frame rate (30 frames per second) and high resolution (1920×1080 pixels), connect the camera to the back-end processing equipment via Wi-Fi, and the camera continuously collects video image data of the door area at the set frame rate, converts the optical signal into an electrical signal, and then transmits it to the subsequent module in the form of a digital image after analog-to-digital conversion;

[0071] S2. Time and position analysis

[0072] S2-1, position identification, through the system software interface, manually delineate the designated monitoring area on the image captured by the camera. For example, with the door frame as a reference, delineate a rectangular area of ​​1.5 meters wide and 2 meters high as the monitoring range, identify the outline of objects in the image through the edge detection algorithm, and identify the human target through the target detection algorithm. When the human outline is detected to enter the delimited rectangular area, the system uses the target tracking algorithm to continuously track the human target. When the human target is detected to enter the monitoring area, the person is marked with a rectangular frame on the image and a unique identification number (such as ID001) is assigned for subsequent tracking and analysis;

[0073] S2-2, time analysis, from the moment the target is identified and enters the set area, the system starts a high-precision timer (such as a timing mechanism based on a computer system clock) to start timing. While the target is in the area, the target's location information is monitored in real time through the target tracking algorithm. If the target leaves the set area, the system stops timing and clears the relevant timing information of the target. If the target stays in the area for a cumulative total of 10 seconds or more, the system generates a trigger signal, which will be transmitted to the facial recognition module, accompanied by the visitor's real-time image data, to trigger the facial recognition process;

[0074] S3, facial recognition

[0075] S3-1, feature extraction. When the facial recognition module receives the trigger signal and the image of the relevant person sent by the position and time analysis module, the image is input into the convolutional neural network (CNN) architecture model based on deep learning. Through the operation of multiple convolutional layers, pooling layers and fully connected layers, the key features of the face are automatically extracted. The convolutional layer uses the convolution kernel to slide on the image to extract features of different scales and directions. The pooling layer downsamples the feature map to reduce the amount of data while retaining the key features. The fully connected layer integrates the extracted features and finally outputs a fixed-dimensional feature vector, which can uniquely represent the feature information of the face. For example, a 4096-dimensional feature vector is output, which contains key features such as the proportion of facial features and contour shape.

[0076] S3-2, comparison and matching. The system pre-establishes a face database to store the facial feature vectors of all authorized family members and their corresponding identity identifiers (such as name, relationship with the head of household, etc.). The comparison and matching unit compares the facial feature vector of the person to be identified output by the feature extraction unit with all feature vectors in the database. The comparison process adopts a similarity measurement algorithm to obtain a set of similarity scores by calculating the cosine similarity value between the feature vector to be identified and each feature vector in the database. The system sets a similarity threshold, such as 0.8. If the highest similarity score is greater than or equal to the threshold, the person is determined to be an authorized family member in the database. If the highest similarity score is less than the threshold, the person is determined to be an outsider.

[0077] S4. Alarm notification

[0078] S4-1, alarm triggering, receiving the identity recognition result from the facial recognition module and the dangerous action judgment result from the dangerous action capture module (to be introduced later), if the facial recognition determines that an outsider has broken in (i.e., the similarity is lower than the threshold), or the identity of the person cannot be recognized and the retention time exceeds 10 seconds, or the dangerous action capture module detects a dangerous action, the alarm triggering unit immediately generates an alarm signal. For example, when the facial recognition module determines that the person at the door is an outsider, the alarm triggering unit immediately generates a high-level alarm signal after receiving the result, and starts the subsequent alarm sending process;

[0079] S4-2, information sending. When the alarm triggering unit generates an alarm signal, the information sending unit first prepares the alarm information to be sent, including the alarm type (such as an outsider breaking in, detecting a dangerous action, etc.), the time and location of the event (location information of the camera at the door), and related image data (such as a real-time screenshot of the door when the alarm is triggered). This information is sent to the mobile phone of the family member through a network communication protocol (such as MQTT-message queue telemetry transmission protocol); the monitoring application installed on the mobile phone of the family member subscribes to the topic as an MQTT client, thereby receiving the alarm information in real time. For example, when the system detects an outsider breaking in, the information sending unit sends the text information of "outsider breaking in, time is 10:30:00 on January 1, 2025, location is the door of the house" and the real-time screenshot of the current door to the mobile phone application of the family member through the MQTT protocol. After the mobile phone of the family member receives the notification, a prompt box pops up on the screen to display the relevant information. Click to view and you can immediately retrieve the real-time monitoring screen;

[0080] S5. Dangerous action capture. When a family member determines through the mobile phone monitoring application that there is a stranger lingering at the door but there is no obvious sign of danger, he or she can manually send a command to start the dangerous action capture module. This module uses human posture estimation technology and processes the input video image based on the OpenPose model of deep learning to identify the position coordinates of multiple joints of the human body (such as head, shoulders, elbows, wrists, hips, knees, ankles, etc.). According to the real-time position and movement trajectory of these joints, combined with the preset dangerous action rules, analysis and judgment are performed. For example, the preset rule is that when the hand joint quickly approaches the door lock position and rotates, or the arm is raised for a long time and the arm is waving quickly, it is judged as a dangerous action. The system calculates the distance between the joints, the angle change, the movement speed and other parameters, and compares them with the threshold in the preset rules to determine whether there is a dangerous action. If a dangerous action that meets the rules is detected, a dangerous action signal is sent to the alarm notification module, and the alarm notification module performs corresponding processing according to the above alarm triggering and information sending process.

[0081] Finally, it should be noted that: Obviously, the above embodiments are only examples for clearly explaining the present invention, and are not intended to limit the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from this are still within the scope of protection of the present invention.

Claims

1. An indoor security monitoring system based on image recognition, characterized in that: The system includes the following modules and subordinate units: The image acquisition module uses a high-frame rate, high-resolution camera, which is connected to the back-end processing equipment through a communication interface such as USB, Ethernet or Wi-Fi. It continuously collects video image data of the door area and converts it into digital images for transmission to subsequent modules; The time-position analysis module includes a position recognition unit and a time analysis unit. The position recognition unit is used to define the monitoring area, identify and track human targets entering the area, and the time analysis unit counts the time of the targets entering the area. When the residence time exceeds a set threshold, a trigger signal and real-time image data are sent to the facial recognition module. The facial recognition module includes a feature extraction unit and a comparison and matching unit. The feature extraction unit uses a convolutional neural network architecture to extract a facial feature vector. The comparison and matching unit compares the vector with a feature vector in a pre-established face database to determine the identity of the person. The alarm notification module includes an alarm triggering unit and an information sending unit. The alarm triggering unit determines whether to trigger an alarm based on the identity recognition result of the facial recognition module and the judgment result of the dangerous action capture module. The information sending unit sends the abnormal information to the mobile phones of family members through the network communication protocol. After receiving manual instructions, the dangerous action capture module uses human posture estimation technology to identify joint points based on the OpenPose model, judges dangerous actions according to preset rules, and sends signals to the alarm notification module.

2. The indoor security monitoring system based on image recognition as claimed in claim 1, characterized in that: In the position identification unit, the operator manually defines the monitoring area through the system software interface, and the system uses edge detection algorithm, target detection algorithm and target tracking algorithm to identify and track human targets.

3. The indoor security monitoring system based on image recognition as claimed in claim 1, characterized in that: After the alarm notification module of the system has accessed the monitoring content, it can cancel the alarm if it visually determines that there is no danger, and enter the dangerous motion capture module at the same time. When the dangerous motion capture module is enabled, only dangerous motion capture will be performed within a certain period of time, and if other people enter the monitoring area, only dangerous motion capture will be performed.

4. A monitoring method for an indoor security monitoring system based on image recognition as described in any one of claims 1 to 3, characterized in that: The method comprises the following steps: S1, image acquisition, using a high frame rate, high resolution camera to collect video image data of the door area and convert it into digital image transmission; S2, time-position analysis, defining the monitoring area, identifying and tracking human targets entering the area, timing the entry of targets, and triggering the facial recognition process if it exceeds 10 seconds; S3, facial recognition, extracting facial feature vectors, comparing them with feature vectors in the database, and determining the identity of the person; S4, alarm notification, decide whether to trigger the alarm based on the results of identity recognition and dangerous action judgment, and send abnormal information to the mobile phones of family members; S5, dangerous action capture, after receiving manual instructions, identify joint points, judge dangerous actions according to preset rules and send signals.

Images