A human behavior recognition and data acquisition system based on artificial intelligence

Abstract

The present invention discloses a human behavior recognition and data acquisition system based on artificial intelligence, which belongs to the field of artificial intelligence. The system collects multimodal data in real time through high-definition cameras, microphone arrays and sensor networks, and performs preprocessing and abnormal behavior detection through edge computing devices. After the data is encrypted using the AES algorithm, it is transmitted to the cloud server, and the deep learning fusion algorithm module is used to analyze the behavior and learn personalized patterns. The system can adapt to environmental changes and automatically adjust parameters to optimize the recognition algorithm. In addition, the system also integrates a user interaction interface for real-time feedback and behavior training guidance, as well as system optimization based on user feedback. The design of the present invention focuses on data security and privacy protection, and adopts advanced encryption technology and anonymization processing to ensure user data security. Through automated system optimization and updating, the system can adapt to changing application scenarios.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a human behavior recognition and data acquisition system based on artificial intelligence. Background Art

[0002] With the rapid development of artificial intelligence technology, human behavior recognition has become an active research direction in the field of pattern recognition and machine learning. Human behavior recognition technology has broad application prospects in many fields, including but not limited to security monitoring, health monitoring, human-computer interaction and sports training. Traditional human behavior recognition systems usually rely on a single data source, such as video or sensor data, and perform behavior analysis through simple feature extraction and classification algorithms. However, these systems have problems such as low accuracy, poor real-time performance and weak environmental adaptability.

[0003] In terms of data collection, existing technologies often lack efficient data preprocessing and encrypted transmission mechanisms, making data vulnerable to attacks during collection and transmission and insufficient privacy protection. In addition, the combined application of cloud and edge computing has not yet been fully realized, making data processing efficiency and real-time performance unable to meet the growing demand.

[0004] In order to improve the accuracy and robustness of behavior recognition, researchers have begun to explore multimodal data fusion technology, which combines multiple information sources such as video, audio, and sensor data for analysis. Although multimodal learning has significant advantages in theory, it still faces challenges in real-time data processing, algorithm complexity, and model generalization ability.

[0005] In addition, personalized behavior pattern learning is a key factor in improving the performance of the recognition system. Current systems often ignore individual differences and lack personalized learning and prediction of user behavior habits. At the same time, the lack of a user feedback mechanism also limits the ability to optimize and update the system.

[0006] In response to the above problems, the present invention proposes an artificial intelligence-based human behavior recognition and data acquisition system, which aims to achieve efficient, accurate, safe and user-friendly human behavior recognition and data acquisition through innovative technologies such as multimodal data fusion, real-time data processing, personalized behavior pattern learning, environmental adaptive adjustment and system optimization and update. Summary of the invention

[0007] The main purpose of the present invention is to provide a human behavior recognition and data acquisition system based on artificial intelligence, which can effectively solve the problems mentioned in the background technology.

[0008] To achieve the above object, the technical solution adopted by the present invention is:

[0009] A human behavior recognition and data collection system based on artificial intelligence, the system architecture consists of:

[0010] Data acquisition and preprocessing module: using video image collector, audio collector and sensor data collector to collect visual data, sound data and body sensor data respectively;

[0011] Real-time data processing and anomaly detection unit: Utilize the real-time data processing engine to pre-process data and quickly identify abnormal behaviors through anomaly detection algorithms;

[0012] Behavior recognition and learning engine: Using deep learning fusion algorithm module: integrating multimodal data, performing behavior recognition, and using personalized learning algorithm to learn personalized behavior patterns from the user behavior database; personalized behavior patterns include: resident behavior habits, worker operation habits, and traffic patterns;

[0013] Environmental perception and adaptive adjustment system: Use environmental perception sensors to monitor environmental conditions, and automatically adjust recognition algorithm parameters according to environmental changes;

[0014] User interaction and feedback system: Design an interactive interface to provide users with an operation interface and feedback display, provide personalized behavior training based on user behavior data through behavior training algorithms, and generate and provide real-time feedback to users through real-time feedback generators;

[0015] Data security and privacy protection mechanisms: including data encryption modules to encrypt data in transit, and anonymization tools to remove or replace personal identification information;

[0016] System optimization and maintenance module: Set up system status monitors to monitor system performance, and use data synchronization and transmission mechanisms to ensure data synchronization between the edge and the cloud;

[0017] The workflow of the system includes the following steps:

[0018] S1. Data Collection

[0019] User Interface (UI): As the front end of the system, it provides a platform for users to interact with the system, and provides an operation interface and feedback display;

[0020] Multimodal data collection system: Start and run, using high-frame-rate cameras, high-quality microphones, and wearable devices to collect user behavior video, voice, and body motion sensor data in real time;

[0021] S2. Edge computing processing

[0022] Real-time data processing engine (located in edge computing device): receives collected data and performs preliminary data processing;

[0023] S3. Abnormal behavior detection

[0024] Anomaly detection algorithms: Apply machine learning algorithms to real-time processed data to quickly identify abnormal behavior;

[0025] Alarm system: If abnormal behavior is detected, the system will trigger an alarm and notify relevant personnel through preset communication channels (such as email, SMS);

[0026] S4. Data encryption and transmission

[0027] Data encryption module: Encrypts data before it is transmitted from the edge device to the cloud to ensure the security of data transmission;

[0028] S5. Cloud data processing and analysis

[0029] Cloud data processing and analysis: Cloud servers receive encrypted data and perform large-scale data storage and complex computing tasks;

[0030] S6. Environmental Adaptation and Parameter Adjustment

[0031] Adaptive environmental perception and adjustment system: monitors changes in external environmental conditions and automatically adjusts recognition algorithm parameters to adapt to these changes;

[0032] S7. Deep learning and personalized behavior analysis

[0033] Deep learning fusion algorithm module: fuses multimodal data and uses deep learning technology for behavior recognition;

[0034] Personalized behavior pattern learning module: uses individual historical behavior data in the user behavior database to establish and optimize personalized behavior patterns through machine learning algorithms;

[0035] S8. Behavior pattern recognition and user feedback

[0036] Behavior pattern recognition module: predicts and identifies behaviors based on personalized models;

[0037] Interactive behavior training and feedback module: provides an interactive interface, provides behavior training and improvement guidance based on the recognition results, and provides feedback to users through a real-time feedback generator;

[0038] S9, System Optimization and Update

[0039] System optimization and update module: Continuously optimize system performance based on user feedback and system status monitoring data.

[0040] Preferably, the specific processing steps of S2 are:

[0041] ①Data reception

[0042] Use the publish-subscribe model of the real-time operating system (RTOS) to receive video, audio, and sensor data streams from data collectors in real time;

[0043] ②Data preprocessing

[0044] Noise reduction: Use a median filter to remove noise from audio data. This is a simple and effective nonlinear filter that is suitable for removing impulse noise. The expression is as follows:

[0045] x filtered [n]=median{x[nk],x[n],x[n+k]}

[0046] Among them, x[n] represents the original signal, x filtered [n] represents the filtered signal, k is the kernel width of the filter;

[0047] Compression: Video data is compressed using the H.264 encoding standard, which is a widely supported video compression format suitable for reducing data size while maintaining acceptable image quality;

[0048] ③Feature extraction

[0049] Video: Use the Sobel operator for edge detection. This is an algorithm used to quickly calculate image gradients in image processing and is often used for edge detection. The expression is as follows:

[0050]

[0051]

[0052]

[0053] Where I is the image intensity, G x and G y Corresponding to the gradients in the x and y directions, respectively,

[0054] Audio: Calculates the Mel Spectral Coefficient (MFCC) of the sound signal, which is a widely used acoustic feature for speech and audio recognition. The expression is as follows:

[0055]

[0056] Among them, MFCC k is the kth Mel spectrum coefficient, c m is the coefficient of the inverse Mel filter, s m is the energy in the Mel frequency domain;

[0057] ④Data formatting

[0058] Implementation: Convert the extracted features into vectorized format to adapt to the subsequent machine learning model input;

[0059] ⑤ Real-time processing optimization

[0060] Batch processing: Due to the limited computing resources of edge devices, data is processed in batches to balance real-time performance and computing efficiency;

[0061] ⑥Hardware acceleration

[0062] GPU acceleration: If the edge device is equipped with a GPU, use CUDA for data processing to accelerate parallel computing tasks;

[0063] ⑦Synchronization and transmission

[0064] Implementation: Use timestamps and network protocols (such as TCP or UDP) to synchronize pre-processed data to the cloud or other processing units.

[0065] Through the above steps, the edge computing processing engine can efficiently process multimodal data in real time, providing the system with continuous and real-time data processing capabilities while reducing dependence on cloud computing resources.

[0066] Preferably, the specific processing steps of S3 are:

[0067] ①Data preprocessing

[0068] The data points xt processed in real time are standardized or normalized to meet the input requirements of the machine learning model;

[0069] ②Training Isolation Forest Model

[0070] Using the normal behavior dataset D normal To train the isolation forest model IF;

[0071] Isolation Forest constructs multiple trees T i To identify outliers in the data, each tree recursively divides the data into two parts by randomly selecting features and split values;

[0072] ③Anomaly score calculation

[0073] For a real-time data point x t , use the trained isolation forest model IF to calculate its anomaly score s(x t );

[0074] The anomaly score is calculated based on the path length of the tree, and the formula is:

[0075]

[0076] Where N is the number of isolation trees, T(x t , T i ) is the data point x t The path length in the i-th tree, T max is the maximum depth of a fully grown tree;

[0077] ④Threshold setting

[0078] Set an abnormal score threshold θ to distinguish normal behavior from abnormal behavior; the threshold is determined by analyzing the score distribution of the normal behavior data set;

[0079] ⑤Anomaly detection

[0080] For a real-time data point x t , if its anomaly score s(x t ) is less than the threshold θ, then it is considered that x t is abnormal:

[0081] Ifs(x t )<θ,thenx t isabnormal

[0082] ⑥Alarm system triggered

[0083] Once abnormal behavior is detected, the system will trigger an alert:

[0084] Send a notification email through the email service API. The email content includes a description of the abnormal behavior, a timestamp, and the next steps.

[0085] ⑦Alarm system integration

[0086] Integrate mail services (such as SMTP) into the system to facilitate sending alerts.

[0087] Through the above process, we can build a system that can monitor behavioral data in real time, quickly identify abnormal behavior and trigger alarms. This system can be applied to various scenarios, such as industrial safety monitoring, health monitoring, etc.

[0088] Preferably, the data encryption module in S4 includes: algorithm selection, key management, data encryption, secure transmission, and cloud decryption, wherein: algorithm selection adopts the AES algorithm as the standard for data encryption; key management: implement a key generation mechanism to create and maintain the AES keys used in the encryption process, and enhance the security of the keys through key encryption key (KEK) technology; data encryption: on the edge device, encrypt the data blocks to be transmitted through the AES algorithm; secure transmission: transmit the encrypted data using the HTTPS or TLS security protocol; cloud decryption: the cloud service uses the same AES key as the edge device to decrypt the encrypted data. Through such an organizational method, the data encryption module provides a comprehensive security solution for data transmission from the edge device to the cloud, ensuring the confidentiality and integrity of the data.

[0089] Preferably, the cloud data processing and analysis in S5 includes: data reception, decryption, storage, analysis and intelligent processing, wherein: data reception: the cloud server receives encrypted data transmitted from the edge device through a secure communication channel; decryption: decrypt the encrypted data using a key agreed upon with the edge device and convert the data into a readable format; storage: store the decrypted data in a cloud database or data warehouse; analysis: use data mining, machine learning, statistical analysis and other technologies to conduct in-depth analysis of the stored data; intelligent processing: based on the analysis results, perform advanced processing tasks such as intelligent decision support, predictive modeling and automated task scheduling; result feedback: feed back the results of the analysis and processing to the user interface or notification system. As the core of the entire system, the cloud data processing and analysis module not only provides secure data storage and efficient processing capabilities, but also can provide users with in-depth business insights and decision support through intelligent analysis.

[0090] Preferably, the specific processing steps of S6 are:

[0091] ①Environmental monitoring

[0092] Deploy environmental sensors, such as light and sound sensors, to collect environmental data in real time;

[0093] ②Data preprocessing

[0094] Filter and normalize environmental data;

[0095] ③Environmental feature extraction

[0096] Extract features from the preprocessed data to form an environmental feature vector E = [e1, e2, ..., e n ], where e i is the i-th environmental feature;

[0097] ④Parameter adjustment strategy design

[0098] Design algorithm parameter adjustment strategies based on environmental characteristics, such as adjusting the brightness threshold of image processing when the lighting changes;

[0099] ⑤Machine learning model training

[0100] Use historical environment data and algorithm parameter pairs to train the machine learning model M:E→P and learn the relationship between features and parameters;

[0101] ⑥Parameter optimization

[0102] Using the trained model M, the algorithm parameters P = [p1, p2, ..., p m ];

[0103] ⑦Real-time feedback and adjustment

[0104] Based on the real-time results of behavior recognition, the optimization algorithm is used to fine-tune the parameters to minimize the recognition error rate;

[0105] ⑧Automated parameter adjustment

[0106] Automate the entire parameter adjustment process to ensure that the system can respond to environmental changes without human intervention.

[0107] Through the above steps and mathematical expressions, the adaptive environmental perception and adjustment system can automatically monitor environmental changes and dynamically adjust algorithm parameters through machine learning models to adapt to different environmental conditions, thereby improving the accuracy and robustness of the behavior recognition algorithm.

[0108] Preferably, the specific processing steps of S7 are:

[0109] In the deep learning fusion algorithm module:

[0110] ①Data fusion and preprocessing

[0111] Synchronize and fuse video, audio, and sensor data into a unified data representation; pre-process the fused data;

[0112] ②Feature extraction

[0113] Use convolutional neural network (CNN) to extract features F from video data V v

[0114] F v =CNN(V)

[0115] Use a recurrent neural network (RNN) to extract features F from audio data A and sensor data S a and F s

[0116] F a =RNN(A)

[0117] F s =RNN(S)

[0118] ③Multimodal feature fusion

[0119] The features F of different modes v , F a , F s Fusion into a single feature representation F:

[0120] F=Fusion(F v , F a , F s )

[0121] In the Personalized Behavior Pattern Learning Module:

[0122] ④Personalized model training

[0123] Using user behavior database D personal The individual historical behavior data in the training personalized model M:

[0124] M=Train(F,D personal )

[0125] Select a suitable machine learning algorithm (SVM or random forest) for training;

[0126] ⑤Model evaluation and optimization

[0127] Use the validation set to evaluate the performance of model M and apply model optimization techniques to improve inference speed;

[0128] ⑥Deployment and application

[0129] Deploy the trained model to the cloud or edge devices to support real-time behavior recognition and personalized analysis.

[0130] In the entire process, the deep learning fusion algorithm module is responsible for extracting features from multimodal data, while the personalized behavior pattern learning module uses these features to train and optimize the personalized behavior recognition model. In this way, the system can recognize and adapt to the behavior patterns of different users and provide more personalized services and experiences.

[0131] Preferably, the specific processing steps of S8 are:

[0132] ①Behavior pattern recognition module

[0133] Model training: Use historical behavior data to train personalized models;

[0134] Behavior prediction: Use trained models to predict real-time or near-real-time behavior data and identify user behavior patterns;

[0135] Mathematical expression: For a given behavior data point x t , the model’s prediction is expressed as

[0136]

[0137] Where f is the model function and θ is the model parameter;

[0138] ②Interactive behavior training and feedback module

[0139] User Interface: Design an interactive user interface that allows the user to interact with the system, provide input or receive feedback;

[0140] Behavior training algorithm: Based on the recognition results, the algorithm is designed to provide personalized behavior training suggestions or guidance;

[0141] Real-time feedback generator: implements a module that generates real-time feedback based on the model's prediction results and the user's behavioral training feedback;

[0142] ③Feedback integration and optimization

[0143] Feedback loop: Establish a feedback loop where users provide feedback through the interface and the system optimizes the behavior recognition model based on the feedback;

[0144] Performance evaluation: Evaluate model performance and optimize based on these metrics;

[0145] ④Real-time performance optimization

[0146] Batch processing: Due to the limited computing resources of edge devices, data is processed in batches to balance real-time performance and computing efficiency;

[0147] Hardware acceleration: If the edge device is equipped with a GPU, use CUDA for data processing to accelerate parallel computing tasks;

[0148] ⑤Deployment and application

[0149] Model deployment: deploy the trained model to edge devices or cloud servers;

[0150] System integration: Integrate the behavior pattern recognition module and the interactive behavior training and feedback module into a larger user behavior analysis system.

[0151] Through the above steps, we can build a system that can perform behavioral pattern recognition and provide user feedback. The system can predict and identify behaviors based on personalized models, provide behavioral training and improvement guidance through an interactive interface, and optimize based on user feedback.

[0152] Preferably, the implementation of the system optimization and update module involves a closed-loop automated process, which aims to continuously improve system performance and user satisfaction. First, key data is collected through user feedback channels and system monitoring tools, including user experience feedback, system performance indicators such as processing delays, error rates, and resource usage. Then, the collected data is analyzed in depth using data analysis techniques to identify performance bottlenecks and user needs. Based on these analysis results, the system will automatically formulate and apply optimization strategies, which may include adjusting algorithm parameters, improving code efficiency, increasing resource allocation, or reconfiguring the system architecture. To ensure the effectiveness of the optimization, the automated testing framework will verify the impact of each change to avoid introducing new problems. After that, the optimized system is deployed to the production environment through a secure update mechanism, while continuing to monitor system performance to ensure the optimization effect and collect user feedback on the updated system. Ultimately, the system will form a continuous feedback loop, constantly self-adjusting and optimizing based on new user feedback and performance data. In this process, security and stability are key considerations to ensure that the system remains reliable and secure while being optimized.

[0153] Compared with the prior art, the present invention has the following beneficial effects:

[0154] 1. In the present invention, the video image collector, audio collector, sensor data collector, and deep learning fusion algorithm module form a multimodal behavior recognition technology. The system collects data through these collectors, and then uses deep learning algorithms (such as CNN, RNN, Transformer, etc.) to fuse and analyze the data to identify different behaviors.

[0155] 2. In the present invention, the real-time data processing engine, anomaly detection algorithm (such as isolation forest, SVM, etc.), and alarm system constitute a real-time behavior anomaly detection system. The system can process behavior data in real time, quickly identify abnormal behavior, and promptly notify relevant personnel or trigger a security response through the alarm system.

[0156] 3. In the present invention, the personalized learning algorithm, user behavior database, and behavior pattern recognition module constitute a personalized behavior pattern learning system. The system uses a machine learning algorithm to analyze an individual's historical behavior data, establish a personalized behavior pattern, and continuously optimize it to improve prediction accuracy.

[0157] 4. In the present invention, the system components of environmental perception sensors, parameter automatic adjustment algorithms, and system status monitors constitute an adaptive environmental perception and adjustment system, which monitors external conditions through environmental perception sensors, automatically adjusts recognition algorithm parameters to adapt to environmental changes, and maintains system stability and reliability.

[0158] 5. In the present invention, the system components of the interactive interface, the behavior training algorithm, and the real-time feedback generator are combined to form an interactive behavior training and feedback system, which provides an interactive interface to communicate with the user, uses the training algorithm to provide improvement guidance based on the recognition results, and provides real-time feedback to the user through the feedback generator.

[0159] The above together constitute an efficient, intelligent, secure and user-friendly human behavior recognition and data collection system, which solves the problems of accuracy, real-time, personalization and privacy protection in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0160] Figure 1 is a system architecture diagram of the present invention;

[0161] Figure 2 It is a schematic diagram of the system flow of the present invention;

[0162] Figure 3 It is a schematic diagram of the edge computing process flow of the present invention;

[0163] Figure 4 It is a schematic diagram of the abnormal behavior detection process of the present invention;

[0164] Figure 5 It is a schematic diagram of the environmental adaptation and parameter adjustment process of the present invention;

[0165] Figure 6 A schematic diagram of the deep learning and personalized behavior analysis process of the present invention;

[0166] Figure 7 It is a flow chart of behavior pattern recognition and user feedback of the present invention. DETAILED DESCRIPTION

[0167] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0168] like Figure 1-Figure 2 As shown, a human behavior recognition and data collection system based on artificial intelligence, the system architecture consists of:

[0169] Data acquisition and preprocessing module: using video image collector, audio collector and sensor data collector to collect visual data, sound data and body sensor data respectively;

[0170] Real-time data processing and anomaly detection unit: Utilize the real-time data processing engine to pre-process data and quickly identify abnormal behaviors through anomaly detection algorithms;

[0171] Behavior recognition and learning engine: Using deep learning fusion algorithm module: integrating multimodal data, performing behavior recognition, and using personalized learning algorithm to learn personalized behavior patterns from the user behavior database; personalized behavior patterns include: resident behavior habits, worker operation habits, and traffic patterns;

[0172] Environmental perception and adaptive adjustment system: Use environmental perception sensors to monitor environmental conditions, and automatically adjust recognition algorithm parameters according to environmental changes;

[0173] User interaction and feedback system: Design an interactive interface to provide users with an operation interface and feedback display, provide personalized behavior training based on user behavior data through behavior training algorithms, and generate and provide real-time feedback to users through real-time feedback generators;

[0174] Data security and privacy protection mechanisms: including data encryption modules to encrypt data in transit, and anonymization tools to remove or replace personal identification information;

[0175] System optimization and maintenance module: Set up system status monitors to monitor system performance, and use data synchronization and transmission mechanisms to ensure data synchronization between the edge and the cloud;

[0176] The workflow of the system includes the following steps:

[0177] S1. Data Collection

[0178] User Interface (UI): As the front end of the system, it provides a platform for users to interact with the system, and provides an operation interface and feedback display;

[0179] Multimodal data collection system: Start and run, using high-frame-rate cameras, high-quality microphones, and wearable devices to collect user behavior video, voice, and body motion sensor data in real time;

[0180] S2. Edge computing processing

[0181] Real-time data processing engine (located in edge computing device): receives collected data and performs preliminary data processing;

[0182] S3. Abnormal behavior detection

[0183] Anomaly detection algorithms: Apply machine learning algorithms to real-time processed data to quickly identify abnormal behavior;

[0184] Alarm system: If abnormal behavior is detected, the system will trigger an alarm and notify relevant personnel through preset communication channels (such as email, SMS);

[0185] S4. Data encryption and transmission

[0186] Data encryption module: Encrypts data before it is transmitted from the edge device to the cloud to ensure the security of data transmission;

[0187] S5. Cloud data processing and analysis

[0188] Cloud data processing and analysis: Cloud servers receive encrypted data and perform large-scale data storage and complex computing tasks;

[0189] S6. Environmental Adaptation and Parameter Adjustment

[0190] Adaptive environmental perception and adjustment system: monitors changes in external environmental conditions and automatically adjusts recognition algorithm parameters to adapt to these changes;

[0191] S7. Deep learning and personalized behavior analysis

[0192] Deep learning fusion algorithm module: fuses multimodal data and uses deep learning technology for behavior recognition;

[0193] Personalized behavior pattern learning module: uses individual historical behavior data in the user behavior database to establish and optimize personalized behavior patterns through machine learning algorithms;

[0194] S8. Behavior pattern recognition and user feedback

[0195] Behavior pattern recognition module: predicts and identifies behaviors based on personalized models;

[0196] Interactive behavior training and feedback module: provides an interactive interface, provides behavior training and improvement guidance based on the recognition results, and provides feedback to users through a real-time feedback generator;

[0197] S9, System Optimization and Update

[0198] System optimization and update module: Continuously optimize system performance based on user feedback and system status monitoring data.

[0199] The following is an actual working example embodiment 1 of the human behavior recognition and data acquisition system based on artificial intelligence

[0200] In the field of smart home, reference Figure 2-Figure 7 The process shown in the figure, the implementation method of the system is as follows:

[0201] 1. Deploy hardware equipment

[0202] Install the necessary hardware devices in the smart home environment, including:

[0203] High-definition cameras to capture images of occupants' activities.

[0204] Microphone array for collecting ambient sound.

[0205] Wearable devices or indoor sensors, such as motion sensors and temperature sensors, are used to monitor occupant behavior and environmental conditions.

[0206] 2. Data Collection

[0207] The system collects the following data in real time:

[0208] Video data: Indoor activities are captured by cameras at 30fps.

[0209] Audio data: Collects room sounds, such as conversations or unusual noises, through a microphone array.

[0210] Sensor data: Monitor the occupants’ heart rate, steps, etc. through wearable devices, and monitor the indoor temperature and humidity through environmental sensors.

[0211] 3. Real-time data processing

[0212] Edge computing devices pre-process data:

[0213] Use a median filter to denoise the audio data.

[0214] Compress video data to reduce transmission bandwidth requirements.

[0215] Filter and normalize sensor data.

[0216] 4. Abnormal behavior detection

[0217] Use the Isolation Forest algorithm to analyze real-time data streams and quickly identify abnormal behaviors, such as:

[0218] If a resident suddenly falls, the system will trigger a fall alarm.

[0219] 5. Data encryption and transmission

[0220] The processed data is encrypted using the AES algorithm and sent to the cloud server via a secure network protocol.

[0221] 6. Cloud Data Analysis

[0222] The cloud server uses deep learning models to analyze the data:

[0223] Fusion of video and audio data to identify daily behavior patterns of residents, such as cooking, cleaning, etc.

[0224] 7. Personalized behavior pattern learning

[0225] The system uses machine learning algorithms to optimize the behavior recognition model based on the behavior habits of residents, such as:

[0226] If residents typically turn off the lights and go to bed at 10 p.m., the system will learn this pattern and automatically adjust the sensitivity of indoor security monitoring after the lights are turned off.

[0227] 8. User interaction and feedback

[0228] Provides a user interface that allows residents to:

[0229] View daily activity reports, including behavior recognition results and health recommendations.

[0230] Provide feedback, such as confirming a false positive anomalous behavior alert.

[0231] 9. Environmental Adaptation

[0232] The system automatically adjusts parameters according to environmental changes:

[0233] When the indoor light becomes dim, the sensitivity of video recognition is automatically increased.

[0234] 10. System optimization and update

[0235] Regularly and automatically update the system:

[0236] Optimize behavior recognition algorithms based on user feedback and the latest machine learning models.

[0237] 11. Security alarm system integration

[0238] When unusual behavior is detected:

[0239] The system will send alerts to occupants or designated contacts via text message, email or smart home app.

[0240] For example, suppose the resident is an elderly person, and the system monitors his / her movements of getting up at night and going to the kitchen through a camera. Since elderly people may be at risk of falling when getting up at night, the system marks this behavior as a behavior that needs attention. By analyzing the frequency and pattern of this behavior, the system can predict and alert family members or caregivers to ensure the safety of the elderly. Through the above implementation, the system not only improves the safety of smart homes, but also continuously optimizes its performance through personalized learning and user feedback to better adapt to the needs of residents.

[0241] Example 2

[0242] In the field of industrial safety production, reference Figure 2-Figure 7The process shown in the figure, the implementation method of the system is as follows:

[0243] 1. Hardware deployment

[0244] Install necessary hardware equipment at the industrial site, including:

[0245] High-definition cameras to monitor employee behavior and machine operation within the work area.

[0246] Microphone array, used to capture device operating sounds and possible abnormal noises.

[0247] A sensor network monitors working environment parameters such as temperature, humidity, and concentration of harmful gases.

[0248] 2. Data Collection

[0249] The system collects the following data in real time:

[0250] Video data: Capture workers’ operating behaviors and equipment status through cameras.

[0251] Audio data: Collects device operation sounds through the microphone array.

[0252] Sensor data: Monitoring of safety-related parameters in industrial environments via sensors.

[0253] 3. Real-time data processing

[0254] Edge computing devices pre-process data:

[0255] Use filtering techniques to reduce noise in audio data.

[0256] Compress video data to adapt to network bandwidth limitations.

[0257] Clean the sensor data to ensure the accuracy of the data.

[0258] 4. Abnormal behavior detection

[0259] Use machine learning algorithms to analyze real-time data streams to quickly identify potential security risks, such as:

[0260] If the system detects that a worker is not wearing safety equipment as required, it will trigger a safety warning.

[0261] 5. Data encryption and transmission

[0262] The processed data is encrypted using an encryption algorithm and sent to the cloud server via a secure network protocol.

[0263] 6. Cloud Data Analysis

[0264] The cloud server uses deep learning models to analyze the data:

[0265] Fusion of video and audio data to identify workers’ non-standard operations and abnormal equipment behavior.

[0266] 7. Personalized behavior pattern learning

[0267] The system uses machine learning algorithms to optimize behavior recognition models based on workers’ operating habits. For example, if the system identifies that a worker often does not check safety devices before operating machinery, it will alert managers to intervene.

[0268] 8. User interaction and feedback

[0269] Provides a user interface that allows security managers to:

[0270] Monitor the safety status of the work area in real time.

[0271] Receive and act upon system-generated security alerts.

[0272] 9. Environmental Adaptation

[0273] The system automatically adjusts parameters according to environmental changes:

[0274] When the concentration of harmful gases is detected to be elevated, the system will increase the monitoring sensitivity of the relevant areas.

[0275] 10. System optimization and update

[0276] Regularly and automatically update the system:

[0277] Update behavior recognition algorithms according to the latest industrial safety standards and operating procedures.

[0278] 11. Security alarm system integration

[0279] When a potential security risk is detected:

[0280] The system will send instant alerts to security managers via SMS, email or a dedicated app.

[0281] For example, suppose that in a chemical plant, the system detects a worker approaching a hazardous chemical storage area without taking proper protective measures through a camera. The system quickly identifies this behavior through real-time data processing and abnormal behavior detection modules, and triggers an alarm. Upon receiving the alarm, the safety manager immediately takes action to instruct the worker to wear protective equipment correctly and reminds him to pay attention to safety regulations. Through this incident, the system learns about the safety risks in this specific area and strengthens the monitoring and alarm sensitivity of this area in the future. Through the above implementation methods, the system not only improves the safety of industrial production, but also continuously optimizes its performance through personalized learning and real-time feedback to better adapt to the safety needs of the industrial environment.

[0282] Example 3

[0283] In the field of traffic management and monitoring, reference Figure 2-Figure 7 The process shown in the figure, the implementation method of the system is as follows:

[0284] 1. Hardware deployment

[0285] Install necessary hardware equipment at key traffic nodes, including:

[0286] High-definition cameras to monitor traffic flow and pedestrian behavior.

[0287] Microphone arrays to capture traffic noise and emergency signals.

[0288] Sensor networks monitor traffic environment parameters such as traffic flow, speed, and weather conditions.

[0289] 2. Data Collection

[0290] The system collects the following data in real time:

[0291] Video data: The dynamics of vehicles and pedestrians at traffic intersections are captured by cameras.

[0292] Audio data: Traffic noise is collected via a microphone array.

[0293] Sensor data: Traffic flow and vehicle speed are monitored through sensors.

[0294] 3. Real-time data processing

[0295] Edge computing devices pre-process data:

[0296] Use filtering techniques to reduce noise in audio data.

[0297] Compress video data to adapt to network bandwidth limitations.

[0298] Clean the sensor data to ensure the accuracy of the data.

[0299] 4. Abnormal behavior detection

[0300] Use machine learning algorithms to analyze real-time data streams to quickly identify traffic violations or incidents, such as:

[0301] If a pedestrian is detected running a red light or a vehicle is speeding, the system will trigger a warning.

[0302] 5. Data encryption and transmission

[0303] The processed data is encrypted using an encryption algorithm and sent to the cloud server via a secure network protocol.

[0304] 6. Cloud Data Analysis

[0305] The cloud server uses deep learning models to analyze the data:

[0306] Fusion of video and audio data to identify traffic violations and potential accidents.

[0307] 7. Personalized behavior pattern learning

[0308] The system uses machine learning algorithms to optimize behavior recognition models based on traffic patterns, such as:

[0309] If the system identifies that violations frequently occur at a certain intersection during a specific time period, the monitoring and alarm mechanism during that period will be strengthened.

[0310] 8. User interaction and feedback

[0311] Provides a user interface that allows the traffic management center to:

[0312] Monitor the status of key traffic nodes in real time.

[0313] Receive system-generated alerts of violations and direct field staff to intervene.

[0314] 9. Environmental Adaptation

[0315] The system automatically adjusts parameters according to environmental changes:

[0316] In severe weather conditions, the system will increase its sensitivity in monitoring road conditions.

[0317] 10. System optimization and update

[0318] Regularly and automatically update the system:

[0319] Update behavior recognition algorithms based on the latest traffic regulations and traffic flow patterns.

[0320] 11. Traffic Alert System Integration

[0321] When a traffic violation or potential accident is detected:

[0322] The system will send instant alerts to relevant personnel through the dispatch system of the control center.

[0323] Working Example

[0324] Suppose that at a major traffic intersection in a city, the system detects a car crossing the stop line at a red light through a camera. The system quickly identifies this violation through real-time data processing and abnormal behavior detection modules, and triggers a traffic violation alarm. Upon receiving the alarm, the traffic management center immediately warns the vehicle through the traffic broadcast system and records the violation for subsequent punishment. At the same time, the system learns this violation pattern and will strengthen the monitoring and alarm mechanism of the intersection in the future. Through the above implementation method, the system not only improves the efficiency and safety of traffic management, but also continuously optimizes its performance through personalized learning and real-time feedback to better meet the needs of traffic management.

[0325] In addition to the specific applications listed in the above embodiments, the system can also be applied to the following scenarios:

[0326] Security Monitoring:

[0327] Intrusion Detection: When the system detects unusual behavior or unauthorized entry, an alarm can be immediately sent to notify family members or relevant security agencies.

[0328] Health and Medical:

[0329] Rehabilitation therapy: During rehabilitation therapy, the system can provide real-time feedback and guidance based on the patient's movement performance to help them improve their rehabilitation movements.

[0330] Athlete training: In sports training, the system can analyze parameters such as the athlete's movement posture, speed and strength, and provide them with personalized training suggestions.

[0331] Retail and Commercial:

[0332] Customer behavior analysis: The system can analyze the behavior patterns of customers in the store, such as browsing products, purchasing behavior, etc., to provide merchants with a basis for marketing strategies.

[0333] Public Safety:

[0334] Crowd monitoring: In public places such as stations, airports, large-scale events, etc., the system can monitor the behavior of the crowd in real time to ensure public safety.

[0335] Emergency response: When the system detects an emergency, such as fire or earthquake, it can immediately activate the emergency plan and coordinate resources from all parties for rescue.

[0336] Education and Research:

[0337] Student behavior analysis: In schools, the system can analyze students’ classroom performance, activity participation, etc., providing teachers with a basis for teaching improvements.

[0338] Academic research: By collecting and analyzing human behavior data, valuable materials can be provided for research in psychology, sociology and other disciplines.

[0339] Traffic Management:

[0340] Driver behavior analysis: By analyzing the driver's driving behavior, such as speed, steering, braking, etc., their driving skills and safety can be evaluated, providing data support for insurance companies or traffic management departments.

[0341] The application principles of these fields are consistent with the above embodiments, and these scene changes all fall within the scope of the invention to be protected. The scope of protection of the invention is defined by the attached claims and their equivalents.

Claims

1. A human behavior recognition and data collection system based on artificial intelligence, Features: The system architecture consists of: Data acquisition and preprocessing module: using video image collector, audio collector and sensor data collector to collect visual data, sound data and body sensor data respectively; Real-time data processing and anomaly detection unit: Use the real-time data processing engine to pre-process data and quickly identify abnormal behaviors through anomaly detection algorithms; Behavior recognition and learning engine: Using deep learning fusion algorithm module: integrating multimodal data, performing behavior recognition, and using personalized learning algorithm to learn personalized behavior patterns from the user behavior database; Individualized behavior patterns include: residents’ behavior habits, workers’ operating habits, and traffic patterns; Environmental perception and adaptive adjustment system: Use environmental perception sensors to monitor environmental conditions, and automatically adjust recognition algorithm parameters according to environmental changes; User interaction and feedback system: Design an interactive interface to provide users with an operation interface and feedback display, provide personalized behavior training based on user behavior data through behavior training algorithms, and generate and provide real-time feedback to users through real-time feedback generators; Data security and privacy protection mechanisms: including data encryption modules to encrypt data in transit, and anonymization tools to remove or replace personal identification information; System optimization and maintenance module: Set up system status monitors to monitor system performance, and use data synchronization and transmission mechanisms to ensure data synchronization between the edge and the cloud; The workflow of the system includes the following steps: S1. Data Collection User interface: As the front end of the system, it provides a platform for users to interact with the system, and provides an operation interface and feedback display; Multimodal data collection system: Up and running, using high-frame-rate cameras, high-quality microphones, and wearable devices to collect user behavior video, voice, and body motion sensor data in real time; S2. Edge computing processing Real-time data processing engine: receives collected data and performs preliminary data processing; S3. Abnormal behavior detection Anomaly detection algorithms: Apply machine learning algorithms to real-time processed data to quickly identify abnormal behavior; Alarm system: If abnormal behavior is detected, the system will trigger an alarm and notify relevant personnel through pre-set communication channels; S4. Data encryption and transmission Data encryption module: Encrypts data before it is transmitted from the edge device to the cloud to ensure the security of data transmission; S5. Cloud data processing and analysis Cloud data processing and analysis: Cloud servers receive encrypted data and perform large-scale data storage and complex computing tasks; S6. Environmental Adaptation and Parameter Adjustment Adaptive environmental perception and adjustment system: monitors changes in external environmental conditions and automatically adjusts recognition algorithm parameters to adapt to these changes; the specific processing steps are: ①Environmental monitoring Deploy environmental sensors, including light and sound sensors, to collect environmental data E in real time; ②Data preprocessing Filter and normalize environmental data; ③Environmental feature extraction Extract features from the preprocessed data to form the environmental feature vector E = [e 1 ,e 2 ,…,e n ], where e i is the i-th environmental feature; ④Parameter adjustment strategy design Design algorithm parameter adjustment strategies based on environmental characteristics and adjust the brightness threshold of image processing when the lighting changes; ⑤Machine learning model training Use historical environment data and algorithm parameter pairs to train the machine learning model M:E→P and learn the relationship between features and parameters; ⑥Parameter optimization Using the trained model M, the algorithm parameters P are optimized according to the current environment characteristics E. 1 ,p 2 ,…,p m ]; ⑦Real-time feedback and adjustment Based on the real-time results of behavior recognition, the optimization algorithm is used to fine-tune the parameters to minimize the recognition error rate; ⑧Automated parameter adjustment Automate the entire parameter adjustment process; S7. Deep learning and personalized behavior analysis Deep learning fusion algorithm module: fuses multimodal data and uses deep learning technology for behavior recognition; Personalized behavior pattern learning module: uses individual historical behavior data in the user behavior database to establish and optimize personalized behavior patterns through machine learning algorithms; S8. Behavior pattern recognition and user feedback Behavior pattern recognition module: predicts and identifies behaviors based on personalized models; Interactive behavior training and feedback module: provides an interactive interface, provides behavior training and improvement guidance based on the recognition results, and provides feedback to users through a real-time feedback generator; S9, System Optimization and Update System optimization and update module: Continuously optimize system performance based on user feedback and system status monitoring data.

2. According to the human behavior recognition and data acquisition system based on artificial intelligence as described in claim 1, Features: The specific processing steps of S2 are: ①Data reception Use the publish-subscribe model of the real-time operating system to receive video, audio, and sensor data streams from data collectors in real time; ②Data preprocessing Noise reduction: Use a median filter to remove noise from audio data. This is a simple and effective nonlinear filter that is suitable for removing impulse noise. The expression is as follows: x filtered [n]=median{x[n-k],x[n],x[n+k]} Among them, x[n] represents the original signal, x filtered [n] represents the filtered signal, k is the kernel width of the filter; Compression: Video data is compressed using the H.264 encoding standard; ③Feature extraction Video: Use the Sobel operator for edge detection. This is an algorithm used to quickly calculate image gradients in image processing and is often used for edge detection. The expression is as follows: Where I is the image intensity, G x and G y Corresponding to the gradients in the x and y directions, respectively, Audio: Calculates the Mel-spectrogram coefficients of the sound signal, which is a widely used acoustic feature for speech and audio recognition. The expression is as follows: Among them, MFCC k is the kth Mel spectrum coefficient, c m is the coefficient of the inverse Mel filter, s m is the energy in the Mel frequency domain; ④Data formatting Implementation: Convert the extracted features into vectorized format to adapt to the subsequent machine learning model input; ⑤ Real-time processing optimization Batch processing: Due to the limited computing resources of edge devices, data is processed in batches to balance real-time performance and computing efficiency; ⑥Hardware acceleration GPU acceleration: If the edge device is equipped with a GPU, use CUDA for data processing to accelerate parallel computing tasks; ⑦Synchronization and transmission Implementation: Use timestamps and network protocols to synchronize pre-processed data to the cloud or other processing units.

3. According to the human behavior recognition and data acquisition system based on artificial intelligence as described in claim 1, Features: The specific processing steps of S3 are: ①Data preprocessing Data point x after real-time processing t Standardize or normalize the data to meet the input requirements of machine learning models; ②Training Isolation Forest Model Using the normal behavior dataset D normal To train the isolation forest model IF; Isolation Forest constructs multiple trees T i To identify outliers in the data, each tree recursively divides the data into two parts by randomly selecting features and split values; ③Anomaly score calculation For a real-time data point x t , use the trained isolation forest model IF to calculate its anomaly score s(x t ); The anomaly score is calculated based on the path length of the tree, and the formula is: Where N is the number of isolation trees, T(x t , T i ) is the data point x t The path length in the i-th tree, T max is the maximum depth of a fully grown tree; ④Threshold setting Set an abnormal score threshold θ to distinguish normal behavior from abnormal behavior; the threshold is determined by analyzing the score distribution of the normal behavior data set; ⑤Anomaly detection For a real-time data point x t , if its anomaly score s(x t ) is less than the threshold θ, then x t is abnormal: Ifs(x t )<θ,thenx t isabnormal ⑥Alarm system triggered Once abnormal behavior is detected, the system will trigger an alert: Send a notification email through the email service API. The email content includes a description of the abnormal behavior, a timestamp, and the next steps. (⑦ Alarm system integration Integrate email services into the system to facilitate sending alerts.

4. According to claim 1, a human behavior recognition and data acquisition system based on artificial intelligence, Features: The data encryption module in S4 includes: algorithm selection, key management, data encryption, secure transmission, and cloud decryption, wherein: algorithm selection adopts the AES algorithm as the standard for data encryption; key management: implement a key generation mechanism to create and maintain the AES keys used in the encryption process, and enhance the security of the keys through key encryption key technology; data encryption: on the edge device, encrypt the data blocks to be transmitted using the AES algorithm; secure transmission: use the HTTPS or TLS security protocol to transmit the encrypted data; cloud decryption: the cloud service uses the same AES key as the edge device to decrypt the encrypted data.

5. The human behavior recognition and data acquisition system based on artificial intelligence according to claim 1, Features: The cloud data processing and analysis in S5 includes: data reception, decryption, storage, analysis and intelligent processing, wherein: data reception: the cloud server receives encrypted data transmitted from the edge device through a secure communication channel; decryption: decrypt the encrypted data using a key agreed upon with the edge device and convert the data into a readable format; storage: store the decrypted data in a cloud database or data warehouse; analysis: use data mining, machine learning, statistical analysis and other technologies to conduct in-depth analysis of the stored data; intelligent processing: based on the analysis results, perform advanced processing tasks such as intelligent decision support, predictive modeling and automated task scheduling; result feedback: feed back the results of the analysis and processing to the user interface or notification system.

6. The human behavior recognition and data acquisition system based on artificial intelligence according to claim 1, Features: The specific processing steps of S7 are: In the deep learning fusion algorithm module: ①Data fusion and preprocessing Synchronize and fuse video, audio, and sensor data into a unified data representation; pre-process the fused data; ②Feature extraction Use convolutional neural network to extract features F from video data V v F v =CNN(V) Use a recurrent neural network to extract features F from audio data A and sensor data S a and F s F a =RNN(A) F s =RNN(S) ③Multimodal feature fusion The features F of different modes v , F a , F s Fusion into a single feature representation F: F=Fusion(F v ,F a ,F s ) In the Personalized Behavior Pattern Learning Module: ④Personalized model training Using user behavior database D personal The individual historical behavior data in the training personalized model M: M=Train(F,D personal ) Select appropriate machine learning algorithms for training; ⑤Model evaluation and optimization Use the validation set to evaluate the performance of model M and apply model optimization techniques to improve inference speed; ⑥Deployment and application Deploy the trained model to the cloud or edge devices to support real-time behavior recognition and personalized analysis.

7. The human behavior recognition and data acquisition system based on artificial intelligence according to claim 1, Features: The specific processing steps of S8 are: ①Behavior pattern recognition module Model training: Use historical behavior data to train personalized models; Behavior prediction: Use trained models to predict real-time or near-real-time behavior data and identify user behavior patterns; Mathematical expression: For a given behavior data point x t , the model’s prediction is expressed as Where f is the model function and θ is the model parameter; ②Interactive behavior training and feedback module User Interface: Design an interactive user interface that allows the user to interact with the system, provide input or receive feedback; Behavior training algorithm: Based on the recognition results, the algorithm is designed to provide personalized behavior training suggestions or guidance; Real-time feedback generator: implements a module that generates real-time feedback based on the model's prediction results and the user's behavioral training feedback; ③Feedback integration and optimization Feedback loop: Establish a feedback loop where users provide feedback through the interface and the system optimizes the behavior recognition model based on the feedback; Performance evaluation: Evaluate model performance and optimize based on these metrics; ④Real-time performance optimization Batch processing: Due to the limited computing resources of edge devices, data is processed in batches to balance real-time performance and computing efficiency; Hardware acceleration: If the edge device is equipped with a GPU, use CUDA for data processing to accelerate parallel computing tasks; ⑤Deployment and application Model deployment: deploy the trained model to edge devices or cloud servers; System integration: Integrate the behavior pattern recognition module and the interactive behavior training and feedback module into a larger user behavior analysis system.

8. The human behavior recognition and data acquisition system based on artificial intelligence according to claim 1, Features: In the S9, key data is first collected through user feedback channels and system monitoring tools, and then the collected data is deeply analyzed using data analysis technology to identify performance bottlenecks and user needs. Based on these analysis results, the system will automatically formulate and apply optimization strategies, and the automated testing framework will verify the impact of each change. After that, the optimized system will be deployed to the production environment through a secure update mechanism, while continuing to monitor system performance and collect user feedback on the updated system; ultimately, the system will form a continuous feedback loop, constantly adjusting and optimizing itself based on new user feedback and performance data.

Images