A non-inductive safety monitoring method and system based on radio frequency identification technology

Abstract

The application relates to a non-inductive safety monitoring method and system based on a radio frequency identification technology, and relates to the technical field of data processing. The method mainly comprises the following steps: collecting original radio frequency data and original video data of a target area in a current time period; performing feature extraction on the original radio frequency data and the original video data to obtain a radio frequency feature vector and a visual feature vector; performing feature vector fusion on the radio frequency feature vector and the visual feature vector to obtain a fusion feature vector; inputting the fusion feature vector of the current time period and the fusion feature vectors of N time periods before the current time period into a behavior classification model to obtain a behavior prediction result; if the behavior prediction result is a limb movement, determining a limb recognition result corresponding to the current time period according to a limb recognition model; and if the behavior prediction result is a gesture movement, determining a gesture recognition result corresponding to the current time period according to a gesture recognition model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a non-intrusive security monitoring method and system based on radio frequency identification (RFID) technology. Background Technology

[0002] As the country gradually enters an aging society and various industries undergo intelligent transformation, the demand for personnel monitoring is experiencing explosive growth, with requirements for "non-intrusive, high-precision, and low-intrusion" monitoring. Traditional technologies have significant shortcomings. In the field of smart elderly care, the elderly are resistant to wearable devices, and there are risks of privacy leaks. Positioning tags on ships are inconvenient to wear and require regular maintenance. Smart home devices mostly rely on buttons or voice interaction, lacking a non-intrusive recognition and response mechanism.

[0003] To meet the urgent needs of the nation and society, researching a technology capable of monitoring, locating, and recognizing people's behavior without their awareness is of great significance. From a technical perspective, traditional camera-based location and recognition systems suffer from privacy risks and poor environmental adaptability, while sensor fusion based on radio technology can solve these problems and improve the robustness of monitoring. Summary of the Invention

[0004] The present invention aims to provide a non-intrusive security monitoring method and system based on radio frequency identification technology to overcome the shortcomings of the existing technology. The technical problem to be solved by the present invention is achieved through the following technical solution.

[0005] This invention provides a non-intrusive security monitoring method based on radio frequency identification (RFID) technology, the method comprising: Collect raw radio frequency data and raw video data of the target area during the current time period; Feature extraction is performed on the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors, respectively. The radio frequency feature vector and the visual feature vector are fused to obtain a fused feature vector. The behavior prediction result is obtained by inputting the fused feature vector of the current time period and the fused feature vector of the N time periods before the current time period into the behavior classification model. If the behavior prediction result is limb movement, then the limb recognition result corresponding to the current time period is determined according to the limb recognition model; If the behavior prediction result is a gesture movement, then the gesture recognition result corresponding to the current time period is determined according to the gesture recognition model.

[0006] In an optional embodiment, the step of extracting features from the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors respectively includes: Determine the micro-Doppler spectrum corresponding to the original radio frequency data, and determine the radio frequency feature vector based on the micro-Doppler spectrum; Video frames are extracted from the raw radio frequency data, and the key point coordinate sequence corresponding to each video frame is determined; the video feature vector is determined based on the key point coordinate sequence.

[0007] In an optional embodiment, determining the radio frequency feature vector based on the micro-Doppler spectrum includes: Time-domain features, frequency-domain features, and time-frequency-domain features are extracted from the micro-Doppler spectrum; and the micro-Doppler spectrum is input into the spectrum feature extraction model to obtain radio frequency features. The radio frequency feature vector is obtained by combining the time-domain features, the frequency-domain features, the time-frequency-domain features, and the radio frequency features.

[0008] In an optional embodiment, determining the video feature vector based on the keypoint coordinate sequence includes: For the keypoint coordinate sequence corresponding to the video frame, calculate the relative vector of each keypoint with respect to the reference keypoint; and calculate the displacement vector of the same keypoint between adjacent video frames. Statistical characteristics are calculated from the relative vector and the displacement vector, wherein the statistical characteristics include at least the mean, standard deviation, maximum value, and minimum value; The overall motion characteristics are obtained by averaging the magnitudes of the displacement vectors of all key points. The video feature vector is obtained by combining the relative vector, the displacement vector, the statistical features, and the overall motion features.

[0009] In an optional embodiment, the step of fusing the radio frequency feature vector and the visual feature vector to obtain a fused feature vector includes: The visual baseline confidence score is calculated based on the illumination quality score, occlusion score, and motion blur score corresponding to the original video data; the radio frequency baseline confidence score is calculated based on the signal-to-noise ratio score, multipath interference score, and target separation score corresponding to the original radio frequency data. Determine the radio frequency feature quality score and the visual feature quality score corresponding to the radio frequency feature vector and the visual feature vector, respectively. Visual weights are calculated based on the visual baseline confidence score and the visual feature quality score; radio frequency weights are calculated based on the radio frequency baseline confidence score and the radio frequency feature quality score. The fused feature vector is obtained by weighting and fusing the radio frequency feature vector and the visual feature vector using the visual weight and the radio frequency weight.

[0010] In an optional embodiment, determining the limb recognition result corresponding to the current time period based on the limb recognition model includes: The fused feature vector is input into the limb recognition model to obtain the initial limb recognition result corresponding to the current time period; The limb recognition result for the current time period is determined based on the initial limb recognition results for the current time period and the initial limb recognition results for the time periods preceding the current time period.

[0011] In an optional embodiment, determining the limb recognition result corresponding to the current time period based on the initial limb recognition result of the current time period and the initial limb recognition results of the time periods preceding the current time period includes: Calculate the state transition score, motion continuity score, physical constraint score, and pattern matching score based on the initial limb recognition results of the current time period and the initial limb recognition results of the time periods before the current time period; The reasonableness score of the initial limb recognition result for the current time period is calculated based on the state transition score, the motion continuity score, the physical constraint score, and the pattern matching score. The limb recognition result corresponding to the current time period is determined based on the reasonableness score.

[0012] In an optional embodiment, determining the gesture recognition result corresponding to the current time period based on the gesture recognition model includes: The fused feature vector is input into the gesture recognition model to obtain the initial gesture recognition result corresponding to the current time period; The gesture recognition result for the current time period is determined based on the initial gesture recognition result for the current time period and the initial gesture recognition result for the time periods preceding the current time period.

[0013] In an optional embodiment, determining the gesture recognition result corresponding to the current time period based on the initial gesture recognition result of the current time period and the initial gesture recognition results of the time periods preceding the current time period includes: Determine whether the initial gesture recognition results for the current time period are consistent with the initial gesture recognition results for time periods before the current time period; If they match, the initial gesture recognition result for the current time period will be used as the gesture recognition result for the current time period. If they are inconsistent, the initial gesture recognition result with higher confidence will be used as the gesture recognition result for the current time period.

[0014] This invention provides a non-intrusive security monitoring system based on radio frequency identification (RFID) technology, the system comprising: The acquisition module is used to acquire raw radio frequency data and raw video data of the target area during the current time period; The extraction module is used to extract features from the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors, respectively. The fusion module is used to fuse the radio frequency feature vector and the visual feature vector to obtain a fused feature vector; The prediction module is used to input the fused feature vector of the current time period and the fused feature vector of N time periods before the current time period into the behavior classification model to obtain the behavior prediction result. The determination module is used to determine the limb recognition result corresponding to the current time period based on the limb recognition model if the behavior prediction result is limb movement. The determining module is further configured to determine the gesture recognition result corresponding to the current time period based on the gesture recognition model if the behavior prediction result is a gesture movement.

[0015] The embodiments of the present invention have the following advantages: This invention provides a non-intrusive security monitoring method and system based on radio frequency identification (RFID) technology. First, raw radio frequency (RF) data and raw video data of a target area are collected for the current time period. Then, feature extraction is performed on the raw RF data and the raw video data to obtain RF feature vectors and visual feature vectors, respectively. The RF feature vectors and visual feature vectors are fused to obtain a fused feature vector. The fused feature vector for the current time period and the fused feature vectors for N time periods prior to the current time period are input into a behavior classification model to obtain a behavior prediction result. If the behavior prediction result is limb movement, the limb recognition result corresponding to the current time period is determined according to a limb recognition model; if the behavior prediction result is gesture movement, the gesture recognition result corresponding to the current time period is determined according to a gesture recognition model. Therefore, this application achieves non-intrusive security monitoring of a target area, enabling personnel monitoring and behavior recognition without their awareness, thereby improving the efficiency of security monitoring. Attached Figure Description

[0016] Figure 1 This is a flowchart of a non-intrusive security monitoring method based on radio frequency identification technology provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a non-contact security monitoring system based on radio frequency identification technology provided in an embodiment of the present invention. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] Please see Figure 1 This invention provides a non-intrusive security monitoring method based on radio frequency identification (RFID) technology, which specifically includes steps S101-S106: S101, collect raw radio frequency data and raw video data of the target area during the current time period.

[0019] The target area is the area that needs to be monitored. The time period can be divided according to actual needs, such as 3 minutes, 5 minutes, or 10 minutes per period. This embodiment does not make a specific limitation on this. The raw video data is a continuous video stream captured by a camera. The raw video data is the signal characteristics (such as signal strength, phase, etc.) collected by radio frequency sensors (such as Wi-Fi CSI, radar).

[0020] S102, feature extraction is performed on the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors, respectively.

[0021] In one optional embodiment provided in this application, the step of extracting features from the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors respectively includes: S1021, determine the micro-Doppler spectrum corresponding to the original radio frequency data, and determine the radio frequency feature vector based on the micro-Doppler spectrum.

[0022] When a person is partially obscured by an obstacle in the current area, vision may be lost for a few frames due to the obstruction. However, the original radio frequency (RF) data can still fully perceive the human body's outline and movement, ensuring that recognition continues. The micro-Doppler features of the original RF data can continuously provide the overall movement speed and rhythm, ensuring that misjudgments are not made due to brief visual loss.

[0023] In this embodiment, determining the radio frequency feature vector based on the micro-Doppler spectrum includes: extracting time-domain features, frequency-domain features, and time-frequency-domain features from the micro-Doppler spectrum; inputting the micro-Doppler spectrum into a spectrum feature extraction model to obtain radio frequency features; and combining the time-domain features, the frequency-domain features, the time-frequency-domain features, and the radio frequency features to obtain the radio frequency feature vector.

[0024] Specifically, in this embodiment, the original radio frequency (RF) data is sequentially subjected to range-dimensional FFT and Doppler-dimensional FFT to obtain a range-Doppler map. For each tracked target, its micro-Doppler features are extracted. A micro-Doppler spectrum map can be obtained by performing STFT on the target's velocity changes over continuous time. Then, features are extracted from the micro-Doppler spectrum map: time-domain features (mean, variance, and entropy of signal amplitude / phase); frequency-domain features (centroid of the spectrum, bandwidth, and roll-off point); and time-frequency-domain features (HOG and LBP texture features of the spectrum map). The time-domain features, frequency-domain features, time-frequency-domain features, and RF features are then combined to obtain the RF feature vector. The spectrum feature extraction model is a 2D CNN or 3D CNN model; the RF features can be obtained by directly inputting the micro-Doppler spectrum map into the spectrum feature extraction model.

[0025] S1022, extract video frames from the original radio frequency data and determine the key point coordinate sequence corresponding to each video frame; determine the video feature vector based on the key point coordinate sequence.

[0026] In this embodiment, determining the video feature vector based on the keypoint coordinate sequence includes: calculating the relative vector of each keypoint relative to a reference keypoint for the keypoint coordinate sequence corresponding to the video frame; calculating the displacement vector of the same keypoint between adjacent video frames; calculating statistical features from the relative vector and the displacement vector, the statistical features including at least the mean, standard deviation, maximum value, and minimum value; averaging the magnitudes of all keypoint displacement vectors to obtain the overall motion feature; and combining the relative vector, the displacement vector, the statistical feature, and the overall motion feature to obtain the video feature vector.

[0027] Specifically, this embodiment uses object detection models (such as YOLO, SSD, Faster R-CNN) to perform human detection on each video frame, obtaining bounding boxes for all people in the image. Then, a multi-object tracking algorithm is used to track the human bodies in consecutive frames, assigning a unique ID to each object and generating a trajectory for each person. For each tracked person, a human pose estimation model is used to extract 2D or 3D human keypoints (including 17 keypoints, such as the nose, left and right shoulders, left and right elbows, etc.). Thus, for each video frame, a keypoint coordinate sequence is obtained ((x,y) for 2D and (x,y,z) for 3D).

[0028] Then, for each video frame, the relative vector of each keypoint is calculated relative to a reference keypoint (e.g., the pelvis or torso center) in that video frame, thus capturing body posture. The displacement vector (i.e., velocity vector) of the same keypoint between video frames is calculated, thus capturing the keypoint's motion. From the relative and motion vectors, the following statistical characteristics are calculated (for each keypoint or keypoint pair): Mean: The average value of the vector in each dimension, representing average position or average motion; Standard Deviation: The standard deviation of the vector in each dimension, representing the magnitude of variation; Maximum and Minimum Values: The maximum and minimum values ​​of the vector in each dimension, representing extreme positions or motions; Correlation Coefficient: The correlation coefficient of motion between different keypoints is calculated, representing the coordination of motion between keypoints.

[0029] S103, the radio frequency feature vector and the visual feature vector are fused to obtain a fused feature vector.

[0030] It's important to note that millimeter-wave radar is extremely sensitive to minute movements, directly providing the micro-trajectory and speed of hand movements. Even if the hand is in a pocket or behind the back (completely obstructing vision), the radar can still detect its movement pattern. Vision and radio frequency (RF) sensors perform a secondary fusion and completion of the hand trajectory at the feature level. When a person is half-obstructed by an obstacle, the visual model may fail, but RF can fully perceive the human outline and movement, ensuring continued recognition.

[0031] In this embodiment, the numerical ranges of the radio frequency feature vector and the visual feature vector need to be unified to the same scale, such as scaling all feature values ​​to the range of 0-1 or standardizing them to a mean of 0 and a variance of 1. Then, different weights are assigned according to the importance and reliability of the features. For example, the visual feature vector has a higher weight when the lighting is good, while the radio frequency feature vector has a higher weight when there is occlusion.

[0032] In an optional embodiment provided in this application, the step of fusing the radio frequency feature vector and the visual feature vector to obtain a fused feature vector includes: S1031, calculate the visual baseline confidence score based on the illumination quality score, occlusion score, and motion blur score corresponding to the original video data; calculate the radio frequency baseline confidence score based on the signal-to-noise ratio score, multipath interference score, and target separation score corresponding to the original radio frequency data.

[0033] The illumination quality score (LQ) is calculated from 0 to 1 using image histogram analysis; for example, uniform illumination results in LQ = 0.9, while excessively dark / bright illumination results in LQ = 0.3. The occlusion score (OQ) is calculated based on the visibility of human keypoints; for example, no occlusion results in OQ = 0.95, partial occlusion in OQ = 0.6, and severe occlusion in OQ = 0.2. The motion blur score (MQ) is calculated through image gradient analysis; for example, clear MQ = 0.9, slightly blurred MQ = 0.7, and severely blurred MQ = 0.3. After obtaining the illumination quality score, occlusion score, and motion blur score corresponding to the original video data, a weighted average of these scores can be calculated to obtain the visual baseline confidence score. For example, the visual baseline confidence score... .

[0034] In this embodiment, the signal-to-noise ratio (SNR_Q) score can be obtained by calculating the signal-to-noise power ratio. For example, if SNR > 20 dB, then SNR_Q = 0.95; if SNR = 10-20 dB, then SNR_Q = 0.7; if SNR < 10 dB, then SNR_Q = 0.3. The multipath interference score (MP_Q) can be obtained through channel impulse response analysis. If single-path interference is dominant, then MP_Q = 0.9; if multipath interference is significant, then MP_Q = 0.5. The target separation score (TS_Q) can be obtained through range-Doppler analysis. If the target separation is clear, then TS_Q = 0.9; if the targets overlap, then TS_Q = 0.4. Radio frequency basic confidence level. .

[0035] S1032, determine the radio frequency feature quality score and visual feature quality score corresponding to the radio frequency feature vector and the visual feature vector, respectively.

[0036] In this embodiment, the radio frequency feature quality score is determined by analyzing the continuity of micro-Doppler features in the radio frequency feature vector; the visual feature quality score can be obtained by calculating the cosine similarity of visual feature vectors between consecutive frames.

[0037] S1033, calculate the visual weight based on the visual baseline confidence score and the visual feature quality score, and calculate the radio frequency weight based on the radio frequency baseline confidence score and the radio frequency feature quality score.

[0038] The formula for calculating visual weight is as follows: The formula for calculating radio frequency weight is: .

[0039] For visual weight, For visual baseline confidence, The visual feature quality score. This is the visual preference coefficient. As environmental factors, These are the normalization coefficients; For radio frequency weights, For the radio frequency base confidence level, Radio frequency characteristic quality fraction, For radio frequency preference coefficient, As environmental factors, The normalization coefficient is used to limit the range of various parameters to a reasonable range.

[0040] S1034, the fused feature vector is obtained by weighting and fusing the radio frequency feature vector and the visual feature vector using the visual weight and the radio frequency weight.

[0041] S104: Input the fused feature vector of the current time period and the fused feature vectors of the N time periods before the current time period into the behavior classification model to obtain the behavior prediction result.

[0042] In this embodiment, the behavior classification model is a time prediction model, and the prediction results include body movements, hand gestures, and their corresponding confidence levels.

[0043] S105, if the behavior prediction result is limb movement, then determine the limb recognition result corresponding to the current time period based on the limb recognition model.

[0044] In the body recognition results, the body movement categories are regarded as states, such as: standing, walking, running, sitting down, standing up, falling down, jumping, etc.

[0045] In one optional embodiment provided in this application, determining the limb recognition result corresponding to the current time period based on the limb recognition model includes: inputting the fused feature vector into the limb recognition model to obtain the initial limb recognition result corresponding to the current time period; and determining the limb recognition result corresponding to the current time period based on the initial limb recognition result of the current time period and the initial limb recognition result of the time periods before the current time period.

[0046] Specifically, determining the limb recognition result corresponding to the current time period based on the initial limb recognition result of the current time period and the initial limb recognition results of the time periods before the current time period includes: calculating a state transition score, a motion continuity score, a physical constraint score, and a pattern matching score based on the initial limb recognition result of the current time period and the initial limb recognition results of the time periods before the current time period; calculating a reasonableness score for the initial limb recognition result of the current time period based on the state transition score, the motion continuity score, the physical constraint score, and the pattern matching score; and determining the limb recognition result corresponding to the current time period based on the reasonableness score.

[0047] In this embodiment, state transition scores are determined based on a state transition rule base. For example, standing → walking: state transition score = 0.9; walking → running: state transition score = 0.8; standing → sitting: state transition score = 0.7; sitting → standing: state transition score = 0.8; walking → standing: state transition score = 0.9; lying down → running: state transition score = 0.1; sitting → running: state transition score = 0.2.

[0048] The motion continuity score determines whether a transitional state can last longer or shorter than a specific time; for example, a sitting state cannot exceed 2 seconds. Minimum durations are: sitting down: at least 0.5 seconds; standing up: at least 0.3 seconds; falling down: at least 0.2 seconds, maximum 3 seconds. The physical constraint score calculates the physically reasonable range of energy changes, analyzes the momentum changes in the motion sequence, and detects abrupt changes that violate the laws of physics; it also includes constraints on joint angles.

[0049] Pattern matching scores can be determined using an action pattern library, which may include: Falling pattern: Standing → Leaning forward → Losing balance → Falling to the ground Standing pattern: Lie down → Support yourself with your arms → Bend your knees → Stand up Sitting pattern: Stand → Lower your body → Buttocks touch the supporting surface → Sit down stably This embodiment obtains the corresponding pattern matching score by matching the model with the model in the action pattern library. After obtaining the state transition score, motion continuity score, physical constraint score, and pattern matching score, the rationality score is calculated according to the following formula: Rationality score R = State transition score × 0.4 + Motion continuity score × 0.3 + Physical constraint score × 0.2 + Pattern matching score × 0.1.

[0050] S106, if the behavior prediction result is a gesture movement, then determine the gesture recognition result corresponding to the current time period according to the gesture recognition model.

[0051] The gesture recognition results can include static gestures (clenched fist, open fist, thumbs up, OK gesture, etc.), dynamic gestures (waving, swiping, drawing circles, zooming, etc.), and continuous gestures (gesture sequence combinations).

[0052] In one optional embodiment provided in this application, determining the gesture recognition result corresponding to the current time period based on the gesture recognition model includes: S1061, The fused feature vector is input into the gesture recognition model to obtain the initial gesture recognition result corresponding to the current time period.

[0053] S1062, determine the gesture recognition result corresponding to the current time period based on the initial gesture recognition result of the current time period and the initial gesture recognition result of the time periods before the current time period.

[0054] Specifically, determining the gesture recognition result corresponding to the current time period based on the initial gesture recognition result of the current time period and the initial gesture recognition result of the time periods before the current time period includes: determining whether the initial gesture recognition result of the current time period and the initial gesture recognition result of the time periods before the current time period are consistent; if they are consistent, then the initial gesture recognition result of the current time period is taken as the gesture recognition result corresponding to the current time period; if they are inconsistent, then the initial gesture recognition result with higher confidence is taken as the gesture recognition result corresponding to the current time period.

[0055] This embodiment provides a non-intrusive security monitoring method based on radio frequency identification (RFID) technology. First, raw radio frequency (RF) data and raw video data of a target area are collected for the current time period. Then, feature extraction is performed on the raw RF data and the raw video data to obtain RF feature vectors and visual feature vectors, respectively. The RF feature vectors and visual feature vectors are fused to obtain a fused feature vector. The fused feature vector for the current time period and the fused feature vectors for N time periods prior to the current time period are input into a behavior classification model to obtain a behavior prediction result. If the behavior prediction result is limb movement, the limb recognition result corresponding to the current time period is determined according to a limb recognition model; if the behavior prediction result is gesture movement, the gesture recognition result corresponding to the current time period is determined according to a gesture recognition model. Therefore, this application achieves non-intrusive security monitoring of a target area, enabling monitoring and behavior recognition of personnel without their awareness, thereby improving the efficiency of security monitoring.

[0056] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0057] In one embodiment, a non-intrusive security monitoring system based on radio frequency identification (RFID) technology is provided. For example... Figure 2 As shown, the functional modules of this non-contact security monitoring system based on radio frequency identification technology are described in detail below: Acquisition module 21 is used to acquire raw radio frequency data and raw video data of the target area during the current time period; Extraction module 22 is used to extract features from the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors, respectively; The fusion module 23 is used to fuse the radio frequency feature vector and the visual feature vector to obtain a fused feature vector; Prediction module 24 is used to input the fused feature vector of the current time period and the fused feature vector of N time periods before the current time period into the behavior classification model to obtain the behavior prediction result; The determination module 25 is used to determine the limb recognition result corresponding to the current time period based on the limb recognition model if the behavior prediction result is limb movement. The determining module 25 is further configured to determine the gesture recognition result corresponding to the current time period based on the gesture recognition model if the behavior prediction result is a gesture movement.

[0058] In an optional embodiment, the extraction module 22 is specifically used for: Determine the micro-Doppler spectrum corresponding to the original radio frequency data, and determine the radio frequency feature vector based on the micro-Doppler spectrum; Video frames are extracted from the raw radio frequency data, and the key point coordinate sequence corresponding to each video frame is determined; the video feature vector is determined based on the key point coordinate sequence.

[0059] In an optional embodiment, the extraction module 22 is specifically used for: Time-domain features, frequency-domain features, and time-frequency-domain features are extracted from the micro-Doppler spectrum; and the micro-Doppler spectrum is input into the spectrum feature extraction model to obtain radio frequency features. The radio frequency feature vector is obtained by combining the time-domain features, the frequency-domain features, the time-frequency-domain features, and the radio frequency features.

[0060] In an optional embodiment, the extraction module 22 is specifically used for: For the keypoint coordinate sequence corresponding to the video frame, calculate the relative vector of each keypoint with respect to the reference keypoint; and calculate the displacement vector of the same keypoint between adjacent video frames. Statistical characteristics are calculated from the relative vector and the displacement vector, wherein the statistical characteristics include at least the mean, standard deviation, maximum value, and minimum value; The overall motion characteristics are obtained by averaging the magnitudes of the displacement vectors of all key points. The video feature vector is obtained by combining the relative vector, the displacement vector, the statistical features, and the overall motion features.

[0061] In an optional embodiment, the fusion module 23 is specifically used for: The visual baseline confidence score is calculated based on the illumination quality score, occlusion score, and motion blur score corresponding to the original video data; the radio frequency baseline confidence score is calculated based on the signal-to-noise ratio score, multipath interference score, and target separation score corresponding to the original radio frequency data. Determine the radio frequency feature quality score and the visual feature quality score corresponding to the radio frequency feature vector and the visual feature vector, respectively. Visual weights are calculated based on the visual baseline confidence score and the visual feature quality score; radio frequency weights are calculated based on the radio frequency baseline confidence score and the radio frequency feature quality score. The fused feature vector is obtained by weighting and fusing the radio frequency feature vector and the visual feature vector using the visual weight and the radio frequency weight.

[0062] In an optional embodiment, the determining module 25 is specifically used for: The fused feature vector is input into the limb recognition model to obtain the initial limb recognition result corresponding to the current time period; The limb recognition result for the current time period is determined based on the initial limb recognition results for the current time period and the initial limb recognition results for the time periods preceding the current time period.

[0063] In an optional embodiment, the determining module 25 is specifically used for: Calculate the state transition score, motion continuity score, physical constraint score, and pattern matching score based on the initial limb recognition results of the current time period and the initial limb recognition results of the time periods before the current time period; The reasonableness score of the initial limb recognition result for the current time period is calculated based on the state transition score, the motion continuity score, the physical constraint score, and the pattern matching score. The limb recognition result corresponding to the current time period is determined based on the reasonableness score.

[0064] In an optional embodiment, the determining module 25 is specifically used for: The fused feature vector is input into the gesture recognition model to obtain the initial gesture recognition result corresponding to the current time period; The gesture recognition result for the current time period is determined based on the initial gesture recognition result for the current time period and the initial gesture recognition result for the time periods preceding the current time period.

[0065] In an optional embodiment, the determining module 25 is specifically used for: Determine whether the initial gesture recognition results for the current time period are consistent with the initial gesture recognition results for time periods before the current time period; If they match, the initial gesture recognition result for the current time period will be used as the gesture recognition result for the current time period. If they are inconsistent, the initial gesture recognition result with higher confidence will be used as the gesture recognition result for the current time period.

[0066] It should be noted that the above detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0067] Specific limitations regarding the contactless security monitoring system based on RFID technology can be found in the limitations of the contactless security monitoring method based on RFID technology mentioned above, and will not be repeated here. Each module in the aforementioned device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0068] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A non-intrusive security monitoring method based on radio frequency identification (RFID) technology, characterized in that, The method includes: Collect raw radio frequency data and raw video data of the target area during the current time period; Feature extraction is performed on the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors, respectively. The radio frequency feature vector and the visual feature vector are fused to obtain a fused feature vector; this includes: calculating the visual baseline confidence score based on the illumination quality score, occlusion score, and motion blur score corresponding to the original video data; and calculating the radio frequency baseline confidence score based on the signal-to-noise ratio score, multipath interference score, and target separation score corresponding to the original radio frequency data. Determine the radio frequency feature quality score and the visual feature quality score corresponding to the radio frequency feature vector and the visual feature vector, respectively. Visual weights are calculated based on the visual baseline confidence score and the visual feature quality score; radio frequency weights are calculated based on the radio frequency baseline confidence score and the radio frequency feature quality score. The fused feature vector is obtained by weighting and fusing the radio frequency feature vector and the visual feature vector using the visual weight and the radio frequency weight; The behavior prediction result is obtained by inputting the fused feature vector of the current time period and the fused feature vector of the N time periods before the current time period into the behavior classification model. If the behavior prediction result is limb movement, then the limb recognition result corresponding to the current time period is determined according to the limb recognition model; If the behavior prediction result is a gesture movement, then the gesture recognition result corresponding to the current time period is determined according to the gesture recognition model.

2. The method according to claim 1, characterized in that, The step of extracting features from the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors includes: Determine the micro-Doppler spectrum corresponding to the original radio frequency data, and determine the radio frequency feature vector based on the micro-Doppler spectrum; Video frames are extracted from the raw radio frequency data, and the key point coordinate sequence corresponding to each video frame is determined; a visual feature vector is determined based on the key point coordinate sequence.

3. The method according to claim 2, characterized in that, The step of determining the radio frequency feature vector based on the micro-Doppler spectrum includes: Time-domain features, frequency-domain features, and time-frequency-domain features are extracted from the micro-Doppler spectrum; and the micro-Doppler spectrum is input into the spectrum feature extraction model to obtain radio frequency features. The radio frequency feature vector is obtained by combining the time-domain features, the frequency-domain features, the time-frequency-domain features, and the radio frequency features.

4. The method according to claim 2, characterized in that, The step of determining the visual feature vector based on the key point coordinate sequence includes: For the keypoint coordinate sequence corresponding to the video frame, calculate the relative vector of each keypoint with respect to the reference keypoint; and calculate the displacement vector of the same keypoint between adjacent video frames. Statistical characteristics are calculated from the relative vector and the displacement vector, wherein the statistical characteristics include at least the mean, standard deviation, maximum value, and minimum value; The overall motion characteristics are obtained by averaging the magnitudes of the displacement vectors of all key points. The visual feature vector is obtained by combining the relative vector, the displacement vector, the statistical features, and the overall motion features.

5. The method according to any one of claims 1-4, characterized in that, The step of determining the limb recognition result corresponding to the current time period based on the limb recognition model includes: The fused feature vector is input into the limb recognition model to obtain the initial limb recognition result corresponding to the current time period; The limb recognition result for the current time period is determined based on the initial limb recognition results for the current time period and the initial limb recognition results for the time periods preceding the current time period.

6. The method according to any one of claims 1-4, characterized in that, The step of determining the gesture recognition result corresponding to the current time period based on the gesture recognition model includes: The fused feature vector is input into the gesture recognition model to obtain the initial gesture recognition result corresponding to the current time period; The gesture recognition result for the current time period is determined based on the initial gesture recognition result for the current time period and the initial gesture recognition result for the time periods preceding the current time period.

7. The method according to claim 6, characterized in that, The step of determining the gesture recognition result corresponding to the current time period based on the initial gesture recognition result of the current time period and the initial gesture recognition results of the time periods before the current time period includes: Determine whether the initial gesture recognition results for the current time period are consistent with the initial gesture recognition results for time periods before the current time period; If they match, the initial gesture recognition result for the current time period will be used as the gesture recognition result for the current time period. If they are inconsistent, the initial gesture recognition result with higher confidence will be used as the gesture recognition result for the current time period.

8. A non-intrusive security monitoring system based on radio frequency identification (RFID) technology, characterized in that, The system includes: The acquisition module is used to acquire raw radio frequency data and raw video data of the target area during the current time period; The extraction module is used to extract features from the original radio frequency data and the original video data to obtain radio frequency feature vectors and visual feature vectors, respectively. The fusion module is used to fuse the radio frequency feature vector and the visual feature vector to obtain a fused feature vector; calculate the visual basic confidence score based on the illumination quality score, occlusion score, and motion blur score corresponding to the original video data; and calculate the radio frequency basic confidence score based on the signal-to-noise ratio score, multipath interference score, and target separation degree corresponding to the original radio frequency data. Determine the radio frequency feature quality score and the visual feature quality score corresponding to the radio frequency feature vector and the visual feature vector, respectively. Visual weights are calculated based on the visual baseline confidence score and the visual feature quality score; radio frequency weights are calculated based on the radio frequency baseline confidence score and the radio frequency feature quality score. The fused feature vector is obtained by weighting and fusing the radio frequency feature vector and the visual feature vector using the visual weight and the radio frequency weight; The prediction module is used to input the fused feature vector of the current time period and the fused feature vector of N time periods before the current time period into the behavior classification model to obtain the behavior prediction result. The determination module is used to determine the limb recognition result corresponding to the current time period based on the limb recognition model if the behavior prediction result is limb movement. The determining module is further configured to determine the gesture recognition result corresponding to the current time period based on the gesture recognition model if the behavior prediction result is a gesture movement.

Images