Abnormal person studying and judging and transaction sensing method
The integration of flexible sensors and deep learning for abnormal personnel identification and movement perception addresses inefficiencies in traditional security monitoring, providing precise and timely responses to complex behaviors.
Patent Information
- Application Number
- CN202510203540.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-24
- Publication Date
- 2025-07-15
AI Technical Summary
Traditional monitoring systems are inefficient, have limited coverage and lagging reactions when dealing with the complex and changeable abnormal behavior of abnormal personnel, and cannot meet the real-time and accuracy requirements of modern public security management. They also have insufficient data mining, shallow correlation analysis, single risk assessment, and lack of dynamic adjustment mechanisms.
A flexible electronic pressure sensor array is used to combine gait recognition and deep learning technology, and through pressure perception and visual gait recognition, a multi-dimensional and dynamic adjustment abnormal personnel analysis and abnormal dynamic perception methods are constructed, and intelligent data transmission and multi-source data analysis are achieved using the Internet of Things, and multi-dimensional analysis is conducted with deep learning algorithms.
It significantly improves the accuracy and early warning efficiency of abnormal personnel identification, shortens the identification time, ensures rapid response and effective intervention, reduces control costs, and provides timely and accurate security services.
Smart Images
Figure CN120318897A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a method for judging abnormal personnel and sensing abnormal movements, which is particularly applicable to the identification of abnormal personnel in key places of residence and the prediction of their abnormal movements, and belongs to the technical field of public security. Background Art
[0002] Traditional means may have problems such as low efficiency, limited coverage, and lagging response when dealing with the complex and changeable abnormal behaviors of abnormal personnel, and cannot meet the efficient and accurate requirements of modern society for public security prevention and control.
[0003] Traditional monitoring relies on manual analysis of video recordings, which is inefficient in complex environments and is affected by factors such as visual dead angles, bad weather, and insufficient light, making it difficult to meet the requirements of modern public security management for real-time performance and accuracy. Although existing security monitoring systems have begun to apply data analysis and artificial intelligence technologies, there are still significant limitations in actual operation: insufficient data mining, shallow correlation analysis, single risk assessment, and lack of dynamic adjustment mechanisms. Summary of the Invention
[0004] The purpose of the present invention is to propose a method for judging abnormal personnel and sensing abnormal movements in view of the defects and deficiencies of the above-mentioned existing technologies. This method integrates flexible electronics, gait recognition, Internet of Things (IoT), and deep learning for judging abnormal personnel and sensing abnormal movements, aiming to build an intelligent security protection network. Through a pressure-sensing IoT device based on a flexible sensor array, the footprint and stride characteristics of abnormal personnel can be accurately identified. Combining visual gait recognition technology, even in the case of poor video quality or suspect disguise, the walking posture of abnormal personnel can be effectively recorded and analyzed, and then the abnormal personnel can be judged by combination. The IoT technology realizes intelligent data transmission, automatically identifies the behavior patterns of abnormal personnel, significantly improves the monitoring effect, reduces the management and control costs, and provides timely and accurate security services. The present invention uses deep learning algorithms and big data technologies to conduct multi-dimensional judgment and all-round detection of the abnormal movements of abnormal personnel. The system makes full use of multi-source data, such as personnel attribute information, spatio-temporal trajectories, and associated attributes, to reveal the deep-level associations between behaviors and comprehensively understand their abnormal movement patterns. A multi-dimensional and dynamically adjustable abnormal discrimination model is constructed to ensure the accuracy and timeliness of the evaluation results. Through automated detection, the system significantly shortens the recognition time, improves the early warning efficiency, and ensures rapid response and effective intervention.
[0005] The present invention proposes a method for judging abnormal personnel and sensing abnormal movements. This method constructs a flexible pressure sensor array to sense and analyze the ground pressure footprints generated by personnel, judge whether they are normal or abnormal personnel, and at the same time uses camera devices at key points as complementary sensing means, and uses posture recognition to supplement the judgment of abnormal personnel to avoid common situations such as video blur and face occlusion. Method flow:
[0006] Step 1: To realize the judgment of abnormal personnel, first, design an array of piezoresistive material thin films composed of small square electrodes to construct a flexible pressure sensor array, which has good flexibility and stability and can accurately sense the pressure distribution of human footprints. Each sensor unit is connected to a charge amplifier through horizontal and vertical wires, and the resistance change is converted into an electrical signal through a voltage-dividing resistor. The single-chip microcomputer A / D interface receives and generates several resistance values of the array, and the calibration range is the image gray scale range. Then the data is transmitted to the upper computer through a multi-channel analog multiplexer array and an FPC interface, and feature extraction and convolution operations are performed using an identification model based on a convolutional neural network. Finally, the model is trained through an open-source computing framework to achieve the efficient identification and real-time warning of abnormal personnel. The system realizes real-time data transmission and visual display through the UDP protocol and Socket communication. By laying this device on the ground of the venue, the permanent personnel and abnormal personnel in the venue can be judged to a certain extent.
[0007] Step 2: At the same time, use the camera devices installed at key points as complementary sensing means, and use pose recognition to supplement the judgment of abnormal personnel to avoid common situations such as video blurring and facial occlusion. First, design a target detection and tracking algorithm model to realize the real-time identification and tracking of people in the video. Steps 3-6 are its specific identification steps.
[0008] Step 3: Perform pose recognition on the target, mark and draw lines on the human body key points, track the changes in the human body posture, and store the posture information of the permanent personnel in the venue.
[0009] Step 4: Extract the position of each key point relative to the center of gravity to obtain the relative coordinates of the key points in each frame of the image, and avoid directly using the key points as gait data.
[0010] Step 5: Process the original data, subtract the coordinates of the center of gravity from the coordinates of the key points respectively to obtain the relative positions of each key point and the center of gravity in each frame of the image (the distances on the X and Y axes). Over time, the difference between the key point coordinates and the center of gravity coordinates represents the movement trajectory of the key points. Save every few frames as a unit, and each frame corresponds to a txt document to record the movement trajectory of the key points and construct a gait dataset of permanent personnel.
[0011] Step 6: Process the new video in the same way, intercept one frame every few frames, extract the movement trajectory of the key points, and compare it with the local dataset to complete the gait recognition of abnormal personnel.
[0012] Step 7: Design a lightweight web visualization platform, create an instance app, define the communication protocol and basic routing / return HTML pages to form a basic user interface. Send the pictures of the identified abnormal personnel and their footprint pictures to the Web application through Socket communication to achieve real-time warning and visual analysis of abnormal personnel.
[0013] Step 8: Import information such as the face, travel, and accompanying persons of abnormal persons, construct a diversified deep learning model, and comprehensively analyze and predict the abnormal information of abnormal persons. Steps 9-14 are specific steps.
[0014] Step 9: Construct a training dataset for the abnormal feature detection model. The attribute features (such as clothing, personal belongings, travel mode, etc.) and their abnormal labels (0 or 1) of each abnormal person are recorded in an Excel table. Obtain the spatio-temporal trajectory and information of accompanying persons of abnormal persons, and construct a training dataset for the associated feature abnormal detection model.
[0015] Step 10: Obtain the pedestrian image and its coordinates through object detection, crop to get a full-body image of a single person, perform face recognition and compare with the database to confirm the identity, and at the same time identify the clothing, personal belongings attributes, and travel mode.
[0016] Step 11: Construct a basic feature abnormal detection model, which is a classification model based on deep learning.
[0017] Step 12: Construct an associated feature abnormal detection model, with the input including spatio-temporal trajectory and information of accompanying persons, and the output being whether there is an abnormality (0 or 1).
[0018] Step 13: Construct a movement discrimination model, with the input including the basic feature abnormal detection result and the associated feature abnormal detection result, and the output being the movement category of the person (0 or 1). Set the number of iterations, with the performance parameter being the loss function value, and use whether the threshold is reached to control the termination condition of model training.
[0019] Step 14: Build a web application on the platform, integrate the basic feature abnormal detection model, the associated feature abnormal detection model, and the movement discrimination model into the web application. When the system detects an abnormal person, notify the user by SMS and display the relevant information of the abnormal person on the web interface. Use the basic information of the user and the above feature data to train the model.
[0020] Furthermore, the deep learning classification model constructed in Step 11 of the present invention includes the following steps:
[0021] Step 11-1: Set the neural network model parameters. The input layer contains 14 neurons, the hidden layer contains multiple neurons, and the output layer contains 3 neurons.
[0022] Step 11-2: Set the number of iterations and performance parameters, and use whether the threshold is reached to control the termination condition of model training. Use the constructed dataset to perform supervised training on the model.
[0023] The unsupervised machine learning model constructed in Step 12 of the present invention includes the following steps:
[0024] Step 12-1: Detect abnormal points in the spatio-temporal trajectory using unsupervised anomaly detection algorithms, and identify abnormal behaviors of fellow travelers using machine learning classification algorithms.
[0025] Step 12-2: Set the number of iterations and performance parameters, and use whether to reach the threshold to control the termination condition of model training. Train the model using the constructed dataset.
[0026] Beneficial effects:
[0027] 1. The present invention combines a flexible pressure sensor array and gait recognition technology to improve the reliability and accuracy of abnormal person judgment in cases.
[0028] 2. The present invention extracts structured and unstructured data from multiple dimensions, constructs a person abnormal feature dataset using multi-source person information data resources, improves data quality, greatly shortens the model development cycle, and improves training efficiency and accuracy.
[0029] 3. The present invention uses a deep learning-based object detection algorithm to construct a high-precision abnormal detection model for basic features of abnormal persons, with a low misjudgment rate, and improves the accurate recognition of key abnormal movement judgment features of persons.
[0030] 4. The present invention combines supervised and unsupervised algorithms to improve the feature extraction efficiency and accuracy for different abnormal movement features of abnormal persons, and effectively detects abnormal situations in spatio-temporal trajectories and fellow travelers. Description of the drawings
[0031] Figure 1 It is a flowchart of the method of the present invention. Detailed implementation manners
[0032] To more clearly illustrate the technical solution of the present invention, the following will be described in conjunction with specific embodiments and drawings. Obviously, the embodiments described below are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other examples can also be obtained based on these embodiments.
[0033] As Figure 1 shown, an abnormal person judgment and abnormal movement perception method proposed by the present invention includes the following steps:
[0034] Step 1: The present invention proposes a flexible pressure sensor based on a piezoresistive material thin-film array composed of 60×60 square small electrodes, which has good flexibility and stability and can accurately sense the footprint pressure distribution. Each sensor unit is connected to a charge amplifier through horizontal and vertical wires, and the resistance change is converted into an electrical signal through a voltage-dividing resistor. The single-chip microcomputer A / D interface receives and generates 3600 resistance values, and the calibration range is from 0 to 255. The data is transmitted to the host computer through a multiplexed analog switch array and an FPC interface, and a convolutional neural network (CNN) is used for feature extraction and convolution operations. Finally, the model is trained through the PyTorch framework, and real-time data transmission and visual display are realized through the UDP protocol and Socket communication.
[0035] This step realizes the efficient identification and real-time warning of abnormal personnel based on pressure-sensing feature recognition.
[0036] Step 2: As a complementary technology, design the judgment of abnormal personnel based on posture recognition.
[0037] Use the Ultralytics YOLOv8 model for object detection and tracking, and combine the BoT-SORT and ByteTrack algorithms to realize the real-time identification and tracking of people in the video.
[0038] Step 3: Use the OpenCV library and the built-in functions of YOLOv8 for posture recognition, mark and draw lines on the human body key points, and track the changes in the human body posture.
[0039] Step 4: Extract the position of each key point relative to the center of gravity to obtain the relative coordinates of the key points in each frame of the image, avoiding directly using the key points as gait data.
[0040] Step 5: Process the original data, subtract the coordinates of the center of gravity from the coordinates of the key points respectively to obtain the relative positions of each key point and the center of gravity in each frame of the image (the distances on the X and Y axes). As time changes, the difference between the key point coordinates and the center of gravity coordinates represents the movement trajectory of the key points. Save every three frames as a unit, and each frame corresponds to a txt document to record the movement trajectory of the key points, and construct a gait dataset of resident (normal) personnel.
[0041] Step 6: Process the new video in the same way, intercept one frame every three frames, extract the movement trajectory of the key points, compare it with the local dataset, complete the judgment of abnormal personnel based on gait recognition, and complement the judgment of abnormal personnel based on pressure-sensing recognition.
[0042] Step 7: Import the Flask and SocketIO modules, create a Flask instance app and a SocketIO instance socketio, define the basic route / to return the HTML page, and form the basic user interface. When starting the application, call socketio.run(app, debug=True) to enable the debug mode for easy development and rapid iteration.
[0043] Send the recognized pictures of abnormal personnel and their pressure footprint data to the Web application through Socket communication to achieve real-time early warning and visual display.
[0044] Step 8: Import information such as the faces, travel, and accompanying personnel of abnormal personnel for further analysis and prediction of the movements of abnormal personnel.
[0045] Step 9: Build a training dataset for the abnormal feature detection model. The attribute features (such as clothing, personal belongings, travel mode, etc.) and their abnormal labels (0 or 1) of each abnormal personnel are recorded in an Excel table. Obtain the spatio-temporal trajectories and accompanying personnel information of abnormal personnel to build a training dataset for the associated feature abnormal detection model.
[0046] Step 10: Perform object detection through YOLOv8 to obtain the personnel images and their coordinates, use OpenCV to crop to get the full-body images of single persons, use Deepface technology for face recognition and compare with the database to confirm the identity, and at the same time use PaddleDetection to identify the clothing and personal belongings attributes, and use YOLOv8 again to identify the travel mode.
[0047] Step 11: Build a basic feature extraction model, which contains a fully connected deep neural network.
[0048] Step 12: Build an associated feature abnormal detection model, with the input including spatio-temporal trajectories and accompanying personnel information, and the output being whether there is an abnormality (0 or 1).
[0049] Step 13: Build a movement discrimination model based on the random forest algorithm, with the input including the results of the basic feature abnormal detection and the associated feature abnormal detection, and the output being the movement category of the personnel (0 or 1). Set the number of iterations Epoch, and the performance parameter Performance as the loss function value, and use whether to reach the threshold to control the termination condition of model training.
[0050] Step 14: Use the Flask framework to build a Web application, integrate the basic feature anomaly detection model, the associated feature anomaly detection model, and the abnormality discrimination model into the Web application. When the system detects abnormal features of abnormal personnel, it notifies the user through SMS and displays the relevant information of the abnormal personnel on the Web interface. Use a large amount of relevant abnormal personnel information and the above feature data to train the model.
[0051] The model constructed in step 11 of the present invention comprises the following steps:
[0052] Step 11-1: Set the neural network model parameters. The input layer contains 14 neurons, the hidden layer contains multiple neurons, and the output layer contains 3 neurons.
[0053] Step 11-2: Set the number of iterations Epoch and the performance parameter Performance, and use whether the threshold is reached to control the termination condition of model training. Use the constructed dataset to conduct supervised training on the model.
[0054] The model constructed in step 12 of the present invention comprises the following steps:
[0055] Step 12-1: Use the DBSCAN algorithm to detect abnormal points in the spatiotemporal trajectory and use the isolation forest algorithm to identify abnormalities among fellow travelers.
[0056] Step 12-2: Set the number of iterations Epoch and the performance parameter Performance, and use whether the threshold is reached to control the termination condition of the model training. Use the constructed dataset to train the model.
[0057] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. An abnormal personnel judgment and movement perception method, characterized in that, The method includes the following steps: Step 1: To achieve the judgment of abnormal personnel, first design a piezoresistive material thin-film array composed of small square electrodes to construct a flexible pressure sensor array, which has good flexibility and stability and can accurately sense the pressure distribution of human footprints. Each sensor unit is connected to a charge amplifier through horizontal and vertical wires. The resistance change is converted into an electrical signal through a voltage-dividing resistor. The single-chip microcomputer's A / D interface receives and generates several resistance values of the array. The calibration range is the image grayscale range. Then, the data is transmitted to the host computer through a multiplexed analog switch array and an FPC interface. Use an identification model based on a convolutional neural network for feature extraction and convolution operations. Finally, train the model through an open-source computing framework to achieve the efficient identification and real-time warning of abnormal personnel. The system realizes real-time data transmission and visual display through the UDP protocol and Socket communication. By laying this device on the ground of the venue, it can judge the regular personnel and abnormal personnel in the venue to a certain extent; Step 2: At the same time, use the camera devices installed at key points as complementary sensing means, and use pose recognition to supplement the judgment of abnormal personnel to avoid common situations such as video blur and face occlusion. First, design a target detection and tracking algorithm model to achieve real-time recognition and tracking of people in the video; Step 3: Perform pose recognition on the target, mark and draw lines on the human body key points, track the changes in the human body posture, and store the posture information of the regular personnel in the venue; Step 4: Extract the position of each key point relative to the center of gravity to obtain the relative coordinates of the key points in each frame of the image, avoiding directly using the key points as gait data; Step 5: Process the original data. Subtract the coordinates of the center of gravity from the coordinates of the key points respectively to obtain the relative positions of each key point and the center of gravity in each frame of the image (the distances on the X and Y axes). As time changes, the difference between the key point coordinates and the center of gravity coordinates represents the movement trajectory of the key points. Save every few frames as a unit, and each frame corresponds to a txt document to record the movement trajectory of the key points, and construct a regular personnel gait data set; Step 6: Process the new video in the same way. Intercept one frame every few frames, extract the movement trajectory of the key points, and compare it with the local data set to complete the gait recognition of abnormal personnel; Step 7: Design a lightweight web visualization platform, create an instance app, define the communication protocol and basic routing / return the HTML page to form a basic user interface, and send the recognized abnormal personnel pictures and their footprint pictures to the Web application through Socket communication to achieve real-time warning and visual analysis of abnormal personnel; Step 8: Import the information such as the face, travel, and accompanying personnel of the abnormal personnel to construct a diversified deep learning model, and comprehensively analyze and predict the abnormal information of the abnormal personnel. Steps 9-14 are specific steps; Step 9: Construct a training dataset for the abnormal feature detection model. The attribute features (such as clothing, personal belongings, travel mode, etc.) and their abnormal labels (0 or 1) of each abnormal person are recorded in an Excel table. Obtain the spatio-temporal trajectories and information of accompanying persons of the abnormal persons to construct a training dataset for the associated feature abnormal detection model; Step 10: Obtain pedestrian images and their coordinates through object detection, crop to get full-body images of single persons, perform face recognition and compare with the database to confirm the identity, and at the same time identify the clothing, personal belongings attributes and travel mode; Step 11: Construct a basic feature abnormal detection model, which is a classification model based on deep learning; Step 12: Construct an associated feature abnormal detection model, with the input including spatio-temporal trajectories and information of accompanying persons, and the output being whether there is an abnormality (0 or 1); Step 13: Construct a movement discrimination model, with the input including the basic feature abnormal detection result and the associated feature abnormal detection result, and the output being the movement category of the person (0 or 1). Set the number of iterations, and the performance parameter is the loss function value. Use whether the threshold is reached to control the termination condition of model training; Step 14: Build a web application on the platform, integrate the basic feature abnormal detection model, the associated feature abnormal detection model and the movement discrimination model into the web application. When the system detects an abnormal person, notify the user by text message and display the relevant information of the abnormal person on the web interface. Use the basic information of the person and the above feature data to train the model.
2. The abnormal person judgment and movement perception method according to claim 1, characterized in that The deep learning classification model constructed in Step 11 includes the following steps: Step 11-1: Set the neural network model parameters. The input layer contains 14 neurons, the hidden layer contains multiple neurons, and the output layer contains 3 neurons; Step 11-2: Set the number of iterations and performance parameters, use whether the threshold is reached to control the termination condition of model training, and perform supervised training on the model using the constructed dataset.
3. The abnormal person judgment and movement perception method according to claim 1, characterized in that The unsupervised machine learning model constructed in Step 12 includes the following steps: Step 12-1: Use an unsupervised anomaly detection algorithm to detect abnormal points in the spatio-temporal trajectory, and use a machine learning classification algorithm to identify the abnormal behaviors of accompanying persons, Step 12-2: Set the number of iterations and performance parameters, use whether the threshold is reached to control the termination condition of model training, and train the model using the constructed dataset.