Home security integrated system and method based on artificial intelligence

Abstract

The invention relates to the technical field of security alarm, in particular to a home security integrated system and method based on artificial intelligence, and the system comprises a multi-source data collection module which collects environment data through an infrared sensor, a thermal imaging camera and a standard video camera, carries out the data fusion, and generates an environment data fusion result. According to the invention, the acquisition module fuses infrared, thermal imaging and video cameras, so that the monitoring coverage is expanded, the capability of capturing environmental details is enhanced, and the detection rate of abnormal behaviors is improved. Standardized monitoring data is generated through data fusion and format unified processing, high-quality input is provided for feature extraction and behavior analysis, data streams are optimized, and information loss is reduced. A pre-trained deep learning model is applied to an intelligent analysis module, recognition of complex behavior modes is enhanced, and the accuracy and timeliness of anomaly detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of security alarm, and particularly to an artificial intelligence-based home security integration system and method. Background Art

[0002] The security alarm technology field encompasses a series of systems and solutions aimed at protecting individuals, families, and enterprises from unauthorized access, theft, fire, and other security threats. This field utilizes various sensors, monitoring devices, control systems, and communication technologies to monitor and respond to potential risks and intrusion events in real time. A home security integration system applies modern security technologies to the home environment to provide comprehensive protection and convenience.

[0003] However, in the prior art, it mostly relies on a single data source, which limits the adaptability and accuracy in complex environments. For example, relying solely on video surveillance is prone to false alarms or missed alarms when the light is insufficient or the line of sight is blocked. In addition, it often starts to respond only after an event occurs, lacking an effective prediction mechanism to warn users in advance, delaying the deployment of countermeasures, and potentially resulting in property losses or security risks. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of the present invention is to solve the drawbacks existing in the prior art, and to propose an artificial intelligence-based home security integration system and method.

[0005] To achieve the above purpose, the present invention adopts the following technical solutions: An artificial intelligence-based home security integration system includes:

[0006] A multi-source data acquisition module that collects environmental data through an infrared sensor, a thermal imaging camera, and a standard video camera, performs data fusion to generate an environmental data fusion result; and according to the environmental data fusion result, performs data cleaning and format unification processing to obtain standardized monitoring data;

[0007] An intelligent analysis and recognition module that uses the standardized monitoring data to perform feature extraction to obtain a feature extraction result; and according to the feature extraction result, uses a pre-trained deep learning model to perform behavior recognition and anomaly detection to generate a behavior recognition result;

[0008] A risk assessment and early warning module that, according to the behavior recognition result, performs risk level assessment to obtain a risk assessment result; and matches the risk assessment result with an event database to generate a comprehensive risk assessment result;

[0009] A response and notification module that receives the comprehensive risk assessment result and determines whether to activate an alarm and notify the user.

[0010] Preferably, the steps for obtaining the environmental data fusion result are:

[0011] Collect infrared data, thermal images, and visible light video streams through an infrared sensor, a thermal imaging camera, and a standard video camera respectively to form an original environmental data set;

[0012] Based on the original environmental data set, perform spatio-temporal alignment on the data from the sensors and eliminate data redundancy to obtain an environmental data fusion result.

[0013] Preferably, the steps for obtaining the standardized monitoring data are as follows:

[0014] Based on the environmental data fusion result, identify and delete noisy or corrupted data to form a preliminary cleaned data set;

[0015] Based on the preliminary cleaned data set, convert all data into a unified encoding format and synchronize all timestamps to generate standardized monitoring data.

[0016] Preferably, the steps for obtaining the feature extraction result are as follows:

[0017] Based on the standardized monitoring data, calculate the information gain value. The calculation formula is:

[0018]

[0019] where T represents the total data set, F represents the feature set, |T f | is the number of data sets under the feature f, |T| is the total number of data, H(T) is the entropy of the data set T, and H(T f ) is the entropy of the data set under the feature f. IG(T,F) represents the information gain value of the total data set T relative to the feature set F;

[0020] Based on the information gain value, screen the feature combination with the highest information gain value to obtain the feature extraction result.

[0021] Preferably, the steps for obtaining the behavior recognition result are as follows:

[0022] Input the feature extraction result into a pre-trained deep learning model, and complete data propagation in sequence according to the deep learning model structure, including input layer data mapping, intermediate layer feature transformation, and output layer classification calculation, and perform the activation function operation of each layer of neurons to form a preliminary behavior recognition result;

[0023] Based on the preliminary behavior recognition result, perform anomaly detection on the recognized behavior categories, analyze the matching degree between the current behavior category and the normal behavior pattern, and obtain the behavior recognition result.

[0024] Preferably, the steps for obtaining the risk assessment result are as follows:

[0025] Based on the behavior recognition result, extract the behavior type, duration, occurrence times, change range of the movement path, and object interaction situation, and construct behavior feature data;

[0026] Based on the behavior feature data, calculate the risk level score, and the expression is:

[0027]

[0028] where, R is the risk level score, A j is the duration of the j-th type of behavior, B is the average duration of normal behaviors, C j is the occurrence times of the j-th type of behavior, D j is the change range of the movement path of the j-th type of behavior, E j is the number of object interactions of the j-th type of behavior, ZF j is the change rate of the j-th type of behavior, m is the total number of behavior categories;

[0029] Based on the risk level score, sort all abnormal behaviors, classify them according to the risk level score, and label the risk levels of the abnormal behaviors to obtain the risk assessment result.

[0030] Preferably, the steps for obtaining the comprehensive risk assessment result are as follows:

[0031] Based on the risk assessment result, query the historical events in the event database that match the risk features, and extract the occurrence time, geographical location, affected target, affected area, and event duration period of the matching historical events to obtain the matching event data;

[0032] Based on the matching event data, calculate the risk matching score, and the expression is:

[0033]

[0034] where, Q is the risk matching score, L k is the k-th type of risk assessment value, M k is the risk assessment value of the k-th type of historical event, N is the average affected area of all matching events, P is the average duration period of all matching events, S is the total number of matching feature categories between the current risk event and the historical events, TE is the occurrence time of the current event, U is the occurrence time of the matching historical event, and r is the total number of matching event categories;

[0035] Based on the risk matching score, screen the historical event with the highest matching score, and conduct inductive analysis on the risk assessment result and the historical event with the highest matching score to obtain the comprehensive risk assessment result.

[0036] Preferably, the specific steps of receiving the comprehensive risk assessment result, judging whether to activate the alarm and notifying the user are as follows:

[0037] Receive the comprehensive risk assessment result, extract the risk level, the size of the affected area, the expected duration, and the number of affected targets, and generate a risk decision input;

[0038] Based on the risk decision input, calculate the alarm activation index, and the expression is:

[0039]

[0040] where I represents the alarm activation index, and f i represents the risk level of the i-th risk event, and l i represents the expected duration of the i-th risk event, LT is the total sum of the durations of all risk events, and a i represents the size of the area affected by the i-th risk event, and A total represents the total area that can be affected, and n is the total number of events in the assessment;

[0041] Based on the alarm activation index, judge whether it exceeds the set activation threshold. If it exceeds the activation threshold, execute the alarm activation program and notify the user. If it does not exceed, maintain the monitoring state.

[0042] The present invention provides a home security integration method, including the following steps:

[0043] Collect the indoor and outdoor temperature, infrared data, and video images through an infrared sensor, a thermal imaging camera, and a standard video camera, and generate an environmental data fusion result;

[0044] Perform outlier detection, null value processing, and data format standardization on the environmental data fusion result, and generate standardized monitoring data;

[0045] Based on the standardized monitoring data, perform feature point marking, extract the dynamic features in the environment, and use a pre-trained deep learning network to analyze the behavior pattern, identify abnormal behaviors, and generate a behavior recognition result;

[0046] Based on the behavior recognition result, classify the risk levels of each behavior pattern, identify risk behaviors, and perform risk analysis by matching with historical event data to generate a comprehensive risk assessment result;

[0047] Receive the comprehensive risk assessment result, judge whether to trigger an alarm according to the risk level, and select SMS or application push to inform the user of the current security status, and generate a response and notification result.

[0048] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0049] In the present invention, the acquisition module integrates infrared, thermal imaging, and video cameras, which not only expands the monitoring coverage but also enhances the ability to capture environmental details and improves the detection rate of abnormal behaviors. The fusion and unified processing of data formats generate standardized monitoring data, providing high-quality input for feature extraction and behavior analysis, optimizing the data stream and reducing information loss. The pre-trained deep learning model is applied in the intelligent analysis module to strengthen the recognition of complex behavior patterns and improve the accuracy and timeliness of abnormal detection. The risk assessment module combines historical data matching to make the assessment results more detailed and provides specific security suggestions for users. The instant response mechanism of the notification module ensures that users receive alerts promptly when high-risk behaviors are detected, improving the efficiency of handling emergencies. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a system flowchart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0052] Please refer to Figure 1 , the present invention provides a technical solution: An artificial intelligence-based home security integration system includes:

[0053] A multi-source data acquisition module that collects environmental data through infrared sensors, thermal imaging cameras, and standard video cameras, performs data fusion, and generates an environmental data fusion result; according to the environmental data fusion result, performs data cleaning and unified processing of formats to obtain standardized monitoring data;

[0054] An intelligent analysis and recognition module that uses the standardized monitoring data to perform feature extraction and obtain a feature extraction result; according to the feature extraction result, uses a pre-trained deep learning model to perform behavior recognition and abnormal detection, and generates a behavior recognition result;

[0055] A risk assessment and early warning module that, according to the behavior recognition result, performs risk level assessment to obtain a risk assessment result; matches the risk assessment result with the event database to generate a comprehensive risk assessment result;

[0056] A response and notification module that receives the comprehensive risk assessment result and determines whether to activate an alarm and notify the user.

[0057] The steps for obtaining the environmental data fusion result are as follows:

[0058] Infrared sensors, thermal imaging cameras, and standard video cameras are used to collect infrared data, thermal images, and visible light video streams, respectively, to form the original environmental data set;

[0059] Based on the original environmental data set, the data from the sensors are aligned in time and space, and data redundancy is eliminated to obtain the environmental data fusion result.

[0060] Specifically, based on the infrared sensor, thermal imaging camera and standard video camera to obtain the corresponding detection signals, first, according to the range instructions provided by the equipment manufacturer, when the infrared sensor and thermal imaging camera are deployed indoors, the temperature measurement threshold range is set to -20℃ to 150℃ respectively. This range is obtained by the average temperature boundary given after repeated experiments in the equipment calibration manual. At the same time, the working brightness level of the standard video camera is referenced to the ambient light range between 0 lux and 500 lux. When the actual brightness value is higher than 500 lux, it is distinguished according to the pre-recorded visible light saturation value, and the excess part is marked as strong light interference. Then, a sampling period of 2 to 5 seconds is set at each camera position. The sampling period is composed of multiple comparisons of the speed of moving objects in the room. The experimental results of the temperature and sensor acquisition stability are used to determine that all collected data will be registered with the device label and timestamp at the same time. If the temperature information exceeds 150°C, it will be determined that the measurement is abnormal and will be discarded. Readings significantly below -20°C will also be considered as equipment failures and will be handled in the same way. When setting the frame rate of the visible light video stream, the final frame rate is determined to be 20 frames per second by pre-calculating the impact of the camera on image resolution under 10 frames per second and 30 frames per second sampling conditions. If the frame loss rate is found to be higher than 3% during the test, the exposure time is shortened and the shutter speed is readjusted. After completing the above acquisition, the retained infrared data, thermal images, and visible light video streams are integrated into corresponding preliminary data sets, ultimately forming the original environmental data set.

[0061] Based on the original environmental dataset, first read the timestamps of each piece of data and compare the time differences between adjacent data. If the time difference exceeds 0.03 seconds, insert linear interpolation in the corresponding infrared data or thermal image to align the time axis of the visible light video frames. This 0.03 - second threshold is obtained by comparing multiple groups of synchronously collected samples and statistically analyzing the cumulative offset of visible light frames. Then, when comparing the spatial coordinates, consider the installation plane coordinates of the thermal imaging camera and the infrared sensor as a unified benchmark. If the pixel matching deviation is greater than 5 pixels, mark it as coordinate misalignment in the original dataset and correct the position deviation according to the edge information of the two - end images. This 5 - pixel upper limit is selected after repeated experiments based on the camera internal parameters and focal length measurement results. Compare the temperature values that may repeatedly appear within the same frame of the visible light video stream. If the temperature difference at the same position is less than 0.2 °C, merge them into one valid record. This 0.2 °C is the redundancy elimination standard selected after statistical analysis of the temperature measurement accuracy of the thermal imaging camera and the resolution of the infrared sensor. Subsequently, delete the redundant duplicate records and re - sort all sensor labels and the corresponding time axis during the integration process. Mark the unqualified or missing data in the processed set for subsequent tracking, and finally obtain the environmental data fusion result.

[0062] The steps for obtaining standardized monitoring data are as follows:

[0063] Based on the environmental data fusion result, identify and delete noisy or damaged data to form a preliminarily cleaned dataset;

[0064] Based on the preliminarily cleaned dataset, convert all data to a unified encoding format, synchronize all timestamps, and generate standardized monitoring data.

[0065] Specifically, based on the environmental data fusion result, first read the temperature measurement value, thermal pixel gray value, and video pixel comparison information corresponding to each record therein. When checking the temperature measurement value, compare it with the pre-established range of -20°C to 150°C. If it is found that the temperature is higher than 150°C or lower than -20°C, mark this record as noise. This temperature upper and lower limit is obtained from multiple measurement stability tests and set with reference to the safety measurement limit of the equipment manufacturer. If the gray value shows an obvious outlier situation, perform a re-determination process based on the entropy threshold. The specific method is to first calculate the gray distribution entropy of each thermal pixel. After obtaining the entropy values of all pixels, form an entropy value array M, where n1 represents the number of pixel intervals, and calculate the average value m1 of this array as the initial entropy threshold T. Then, for each pixel record, determine whether it is greater than T. If it meets the condition of being greater than T, exclude the corresponding data. Let a certain function x be an indicator function, which is recorded as 1 when x holds, otherwise 0. If there are still data higher than the threshold after continuous checking, let T decrease downward at a step size δ. The step size of 0.1 is selected after comparing the influence of different decreasing amplitudes on the exclusion result. If no record to be excluded appears after multiple iterations, stop decreasing and determine the final T. In the video pixel comparison information, determine damaged data such as broken frames or frames that cannot obtain a clear target contour based on frame difference. Count the number of occurrences of such frames and check whether it exceeds 5% of the total number of frames. This 5% threshold comes from the comprehensive evaluation value of image readability during testing. If it exceeds, exclude this part of the frame data from the subsequent process. After repeating the above judgment process, aggregate the remaining valid records into a new set to form a preliminarily cleaned data set.

[0066] Based on the preliminarily cleaned data set, first extract the encoding methods of all numerical records and represent them uniformly under the UTF-8 character set. Check whether there are conflicts in the original encoding identifiers of the infrared measurement value, thermal pixel gray value, and video frame index respectively. If it is found that some records are stored in ASCII code or GBK format, convert them to UTF-8 format according to the byte mapping relationship. When processing the timestamp, split the year, month, day, hour, minute, second, and millisecond included in each record, establish a zero reference based on the Gregorian standard time, and calculate the absolute time value by accumulating backward at intervals of 0.01 seconds. This 0.01-second interval is selected after repeatedly testing the coordination between the video frame sampling frequency and the thermal imaging data refresh rate. If the millisecond part is missing in some records, supplement it with the average value, which is obtained by weighted calculation of the adjacent valid timestamps before and after it. When detecting an abnormal time sequence, such as an inversion or illogical error in the year, month, and day fields, exclude this record. Then, add a unified time zone identifier to the timestamps of all records and sort them by number. Finally, rearrange and summarize all the data in the converted format in row order to generate standardized monitoring data.

[0067] The steps for obtaining the feature extraction results are as follows:

[0068] Based on the standardized monitoring data, calculate the information gain value. The calculation formula is:

[0069]

[0070] where T represents the total data set, F represents the feature set, |T f | is the number of data sets under the feature f, |T| is the total number of data, H(T) is the entropy of the data set T, and H(T f ) is the entropy of the data set under the feature f. IG(T,F) represents the information gain value of the total data set T relative to the feature set F;

[0071] Based on the information gain value, screen the feature combination with the highest information gain value to obtain the feature extraction result.

[0072] Specifically, the advantage of the formula is that it measures the amount of information provided by a feature when distinguishing a data set through the entropy value, and combines the difference between the entropy of the total data set and the entropy of the feature subset to measure the importance of the feature to the overall data distribution, thereby providing a quantitative basis for subsequent screening of features with significant difference degrees and discriminative powers.

[0073] The steps for obtaining T:

[0074] T represents the total data set. Taking 80 records continuously collected in the indoor environment monitoring scenario as an example, based on real-time observations within a 24-hour period, each record contains several observation fields such as temperature, humidity, and movement state identifier. These records are obtained from periodic readings of sensor outputs. During the collection process, distortion values caused by equipment failure error reports are excluded. For the stability determination of the observation effectiveness, a 5% record sampling principle is adopted, that is, for any 5% of the samples, each field is detected to see if it is within a reasonable range, and if it is outside the range, the sampling point is discarded. After multiple repeated comparisons, 80 records that meet integrity and consistency are obtained. Therefore, T = {r1, r2, …, r 80} can be determined.

[0075] The steps for obtaining F:

[0076] Let \(F\) represent the feature set. Five features directly related to security events are selected from the above 80 records as examples, including temperature gradient, humidity range, static / dynamic status flag, personnel identification tag, and sound intensity level. These five features are determined through previous analysis to be able to affect the indoor activity pattern. The original data of each feature field is compared, and the range of valid values is statistically obtained respectively, and invalid entries and extreme abnormal readings are excluded. The quantization of each feature field follows a unified coding table. For example, the sound intensity level is divided into the range of 0 to 100 and mapped to decibel values, and the temperature gradient ranges from 0°C to 50°C, etc. Finally, \(F = \{f1, f2, f3, f4, f5\}\).

[0077] |T f | acquisition steps:

[0078] |T f | represents the number of data sets under feature \(f\). To quantify the coverage of each feature in the data set, it is necessary to first determine the value ranges of feature \(f\). For example, for the temperature gradient \(f1\), it is divided into a low-temperature section (0°C to 18°C), a normal-temperature section (18°C to 26°C), a relatively high-temperature section (26°C to 35°C), etc. Then, the temperature observation values of each record in \(T\) are assigned to the corresponding sections, and the number of records included in the section where the current feature value belongs is statistically obtained. When the total number of records is 80, if the low-temperature section contains 22 records, the normal-temperature section contains 40 records, and the relatively high-temperature section contains 18 records, then when feature \(f1\) points to the normal-temperature section

[0079] |T| acquisition steps:

[0080] |T| is the total number of data. According to the 80 records mentioned above, this value is directly equal to 80. This quantity comes from reading the monitoring data every 15 minutes within 24 hours. Each set of multi-field observation results read each time is regarded as a record. After 24 hours, 96 initial records are obtained. Subsequently, 16 records with all missing values in temperature, humidity, static / dynamic status flag, or other key fields are screened out, and the remaining 80 records are determined to be available. |T| is thus determined.

[0081] Steps for obtaining \(H(T)\):

[0082] \(H(T)\) represents the entropy of the data set \(T\), which is used to measure the uncertainty of the overall data distribution. To calculate this value, it is necessary to first count the probability of each possible label appearing. The label can be the distinction of static / dynamic status, the annotation of environmental safety level, or other classification fields. In this example, the 80 records are divided into a safe identification class and a suspicious identification class, a total of 2 labels. The safe identification class accounts for 48 records, and the suspicious identification class accounts for 32 records. At this time, let \(p1 = 48 / 80 = 0.6\) and \(p2 = 32 / 80 = 0.4\). According to the Shannon entropy \(H = -\sum p\) ilog2p i The general formula gives H(T) = -[0.6log2(0.6) + 0.4log2(0.4)], and by calculation, H(T) ≈ 0.971.

[0083] H(T f ) acquisition steps:

[0084] H(T f ) represents the entropy under the subset corresponding to the feature f. To obtain this value, it is necessary to determine the label distribution corresponding to the specific value range of the feature f. Continuing with the example of the normal temperature section of the temperature gradient f1 mentioned above, among the 40 records in this section, there are 24 safety identification types and 16 suspicious identification types. Therefore, the corresponding probabilities and Then, similarly, calculate according to to get H(T f ) ≈ 0.971.

[0085] Calculation process:

[0086] Statistically calculate the entropy H(T) of the current data set. From the previous text, H(T) ≈ 0.971;

[0087] Calculate |T f | corresponding to each feature f, and calculate H(T f );

[0088] Sum up each item ;

[0089] Subtract the above sum result from H(T) to get IG(T,F);

[0090] Taking the two major features of the temperature gradient f1 and the dynamic and static state identification f2 as an example, let (The dynamic and static state feature covers all records, but can be further divided into multiple subsets at the next level of subdivision), and calculate and In this example, let Then:

[0091]

[0092] Furthermore:

[0093] IG(T,F) = H(T) - 1.4355 = 0.971 - 1.4355 = -0.4645

[0094] In this example, a negative value is obtained because the selected feature distribution does not effectively reduce the entropy, or more refined sub-section splitting is required to reflect the difference degree. For other feature combinations, if a positive value is generated, it means information gain.

[0095] The results show that the value of IG(T,F) can measure the importance of features for data classification or determination. When the value is larger, it indicates that the feature can reduce the overall uncertainty more. When the value is too small or negative, it means that the feature has limited value for distinguishing data. According to the size of the IG values of different features, it will directly reflect which features are more significant to be screened subsequently.

[0096] Based on the information gain value, first read the IG values of each feature and sort them from large to small. When the IG values of multiple features are close, the significance of the difference needs to be judged according to the empirical threshold. This empirical threshold can be obtained by statistically averaging the differences after nearly a hundred tests. For example, after multiple tests, it is found that when the difference in IG values is less than 0.02, the impact on the subsequent recognition effect is very small. Therefore, 0.02 is established as the empirical threshold for distinguishing feature priorities. After sorting the IG values, list all the feature numbers from the first to the last. If the difference in IG values of some features among the top six features is less than 0.02, they are regarded as having the same priority. Then, perform a correlation analysis on the features with the same priority using the dynamic and static state identifiers or other observation fields. For example, compare the IG cross-term between the temperature gradient feature and the humidity interval feature, and record whether it is greater than the cross-term IG of another combination. This cross-term IG can be determined by calculating the joint entropy after considering temperature and humidity as a joint feature and taking the difference from the previous entropy value. If the value is higher than 0.15, it is considered to have obvious distinguishability in the cross-analysis. When the cross-term IG is greater than 0.15, the feature combination is preferentially retained. Summarize these selected features item by item and record their corresponding IG values to form a candidate list. At the same time, check the remaining low-IG features previously to confirm whether there are rare features that still have potential value in extreme cases. If there are still items with an IG value greater than 0.05, mark them for alternative in the list. Through the above comparison and cross-check steps, complete the ranking of feature priorities and select a series of feature combinations with the highest information gain value to obtain the feature extraction result.

[0097] The steps for obtaining the behavior recognition result are as follows:

[0098] Input the feature extraction result into a pre-trained deep learning model, and complete data propagation in sequence according to the deep learning model structure, including input layer data mapping, intermediate layer feature transformation, and output layer classification calculation. Execute the activation function operations of each layer of neurons to form a preliminary behavior recognition result;

[0099] Based on the preliminary behavior recognition result, perform anomaly detection on the recognized behavior categories, analyze the matching degree between the current behavior category and the normal behavior pattern, and obtain the behavior recognition result.

[0100] Specifically, the feature extraction results are input into a pre-trained deep learning model one by one. This pre-trained deep learning model is previously trained by collecting 600 labeled home scene monitoring data. Each monitoring data contains multiple fields such as temperature values, personnel locations, and activity trajectory identifiers. During training, 80% of the above monitoring data is divided as training samples and processed in batches. The size of each batch is set to 32, and this batch size is determined by referring to the balance of device processing speed and storage resources. Subsequently, the parameters are adjusted according to the loss function value in each forward propagation. The loss function takes the form based on cross-entropy. For each input sample, the sum of the differences between the true category and the predicted category is calculated and accumulated and averaged. If the accuracy of the model in the validation set is lower than 92% after 10 rounds of iteration, an additional 5 rounds of iteration are added and the convergence stability of the model is tested through the loss curve. After training is completed, the fixed parameters of the model are saved in the device storage space and used for inference in the actual application stage. When the feature extraction results are sent into this model, they will first undergo numerical mapping through the input layer, and then the information of each dimension will undergo multiple weight operations in the middle layer. The initial values of these weights are referred to the results obtained from the previous training and will not be modified after the pre-training is completed. If the output value of some neurons in the middle layer is higher than 0.7, the corresponding feature elements will be regarded as the main inputs. This threshold of 0.7 is selected after statistically comparing the distribution of the model output confidence. When the data continues to be passed backward in the middle layer, batch normalization is used to stretch or scale the values of different dimensions. When passing through the output layer, if the highest probability value of a certain category output by the model exceeds 0.6, then this category is determined as the preliminary recognition result. This threshold of 0.6 is derived from the analysis of the confidence distribution of different categories in the training set and it is confirmed that category confusion is likely to occur when it is lower than 0.6. When the category with the highest probability is confirmed, it is used as the preliminary behavior recognition result.

[0101] Based on the preliminary behavior recognition results, first read the category label and the corresponding confidence level of each recognized entry. Compare with the pre-established abnormal matching benchmark value. If the confidence level is lower than 0.4, it is determined that the category label of this entry is unstable. This 0.4 coefficient is obtained from the statistical ratio of misidentifications and low confidence intervals in a large number of actual measurement data. When the category label is stable, then check its occurrence frequency in the historical records. If the frequency deviates from the normal pattern by more than 30%, it is regarded as an abnormal tendency. The 30% judgment basis is the average floating interval obtained from the analysis of the behavior statistics of the monitored object. When comparing each entry with this benchmark interval, if it is found that multiple consecutive records are all higher than 30%, more attention should be paid. In specific implementation, the time period to which the behavior belongs, the indoor location, and relevant temperature or personnel information can be comprehensively referred to. For example, compare the number of activities during the night time period with the number of activities during the day time period. If the ratio of the two is greater than 1.5, it indicates that the number of night-time behaviors is significantly higher. Then combine this high phenomenon with the permanent location of the personnel for further screening. If the night-time activities are mainly concentrated in key areas such as the door, it is regarded as a high-risk signal during abnormal detection. If the confidence level is insufficient or the deviation degree is not high, only routine monitoring is carried out. If the judgment result conforms to the abnormal situation, this category is marked as abnormal behavior and incorporated into the behavior recognition result.

[0102] The steps to obtain the risk assessment result are as follows:

[0103] Based on the behavior recognition result, extract the behavior type, duration, occurrence times, change range of the movement path, and object interaction situation to construct behavior feature data;

[0104] Based on the behavior feature data, calculate the risk level score. The expression is:

[0105]

[0106] where, R is the risk level score, A j is the duration of the j-th type of behavior, B is the average duration of normal behaviors, C j is the occurrence times of the j-th type of behavior, D j is the change range of the movement path of the j-th type of behavior, E j is the number of object interactions of the j-th type of behavior, ZF j is the change rate of the j-th type of behavior, and m is the total number of behavior categories;

[0107] Based on the risk level score, sort all abnormal behaviors, classify them according to the risk level score, and mark the risk level of abnormal behaviors to obtain the risk assessment result.

[0108] Specifically, based on the previously obtained behavior recognition results, first read the category label corresponding to each behavior and compare it with a pre-established reference document, which consists of a list of activities of different types. The names and corresponding classification codes of each activity are clearly defined. Then, read the duration of each record one by one in combination with the timestamp information contained in the recognition results. If the duration is longer than 30 minutes, it is marked as an overtime behavior and the reason is noted. This 30-minute limit is obtained based on the statistics of past monitoring data and the observation of the peak range of daily activities. Next, query the occurrence times of the corresponding behavior in the recognition results and compare it with the set normal range. For example, the normal range can be defined as that a certain type of behavior appears no more than 10 times within one day. If the number of times on the same day exceeds 10 times, it indicates that the occurrence frequency has increased significantly. For the tracking of the movement path, it is necessary to lock the center point or key positions of the person frame by frame in the monitoring screen and record the position changes. When the position change amplitude exceeds the window or door area or crosses the room passage, it is registered as a certain amplitude of movement. If the cumulative amplitude exceeds the preset 8 meters, it indicates that the movement range is large. This 8-meter value comes from the measured data of the distance between the living room and the bedroom in a typical indoor house type. Through the above steps, information such as behavior type, duration, occurrence times, and movement path change amplitude can be obtained. When counting the interaction situation of the statistical object, it is necessary to record whether it appears in the same video frame with other indoor people at the same time or whether there is an obvious contact situation with pets or items, and summarize the multiple interactions that occur cumulatively within one monitoring period. In this way, the interaction quantity of the behavior object can be annotated in the record. Through the integration process of all the above data, behavior characteristic data is finally formed.

[0109] The benefit of the formula is that by synchronously considering the difference between the behavior duration and the normal average value, the cumulative product of the behavior occurrence times and the movement amplitude, and the proportional relationship between the object interaction quantity and the change rate in the same expression, the difference degree of different behaviors on the overall risk level can be more intuitively reflected, and a quantitative basis is provided for subsequent classification and ranking.

[0110] A j Introduction:

[0111] A j represents the duration of the j-th type of behavior. This duration is usually in minutes or seconds, depending on the interval accuracy of the monitoring sampling and the requirements of the behavior occurrence scenario. For example, in a home environment, the duration can be in seconds to obtain more accurate statistics. To obtain A jIt is necessary to first mark the time for each monitoring record and register the start and end time points, then perform a difference operation to obtain the corresponding duration. After summarizing all the monitoring records, all the durations of a certain type of behavior will be obtained. Finally, these values can be added or averaged to represent the overall duration of this type of behavior. For example, if a certain type of behavior appears 3 times in a day, lasting 45 seconds, 80 seconds, and 60 seconds respectively, then at this time, let A j Take their average value of 61.67 seconds.

[0112] Introduction of B:

[0113] B represents the average duration of normal behavior, which is used to measure the difference between the time length of the current suspicious behavior and the daily normal level. In practice, it can be obtained by collecting a large number of daily behavior records over a long period. For example, add up the durations of all the same type of activities in the past month and then divide by the number of times this activity appears, so as to obtain the historical average duration of this type of activity. In a home monitoring scenario, if it is observed that family members take short indoor walks every day and usually once between 30 seconds and 1 minute, then the average duration in a certain day can be obtained through statistical data as 45 seconds. If the records of one day, three days, and one week are accumulated and averaged, a more representative B value can be obtained. Assuming that the comprehensive result is 50 seconds, then in subsequent calculations, B = 50 can be set.

[0114] C j Introduction of:

[0115] C j represents the occurrence times of the j-th type of behavior. To obtain this value, it can be accumulated and statistically counted according to the time period or the number identifier of each occurrence of this type of behavior in the monitoring data. In most home or indoor monitoring scenarios, some suspicious behaviors, such as frequent opening and closing of doors at night or long-term stay in the room, may occur repeatedly in the specified monitoring window. Therefore, each occurrence needs to be classified and marked and counted during recording. After completing the preliminary classification, calculate the quantity item by item to obtain C j . For example, within the monitoring period from 10 pm to 5 am, if the same type of behavior is detected a total of 8 times, then C j can be assigned a value of 8.

[0116] D j Introduction of:

[0117] D jrepresents the change amplitude of the movement path of the j-th type of behavior. It is necessary to obtain the coordinate trajectory of the behavior object from the surveillance video or other position signals and count the displacement of this trajectory. If the displacement is only within one room, it is usually measured in meters. When the object moves back and forth within the room, the path length can be calculated by comparing the center point positions frame by frame. For example, sample the movement trajectory at a rate of 1 frame per second, and then accumulate the distances between points to obtain the cumulative displacement of this behavior during one occurrence. During multiple detections, the displacements of the same type of behavior are accumulated or averaged to illustrate the overall movement degree. For example, if the path of one behavior totals 5 meters, the path of another behavior totals 12 meters, and the third is 7 meters, then D j =(5 + 12 + 7) / 3 = 8 meters.

[0118] E j Introduction of:

[0119] E j represents the number of object interactions of the j-th type of behavior, mainly recording which people or objects the behavior object has simultaneous appearances or contact behaviors with. To obtain this value, each video frame can be parsed in the surveillance record to check whether the relative distance between the object and other people or items is lower than a certain threshold. For example, use 1 meter as the detection threshold for relative interaction. This 1-meter range can be determined according to the walking area and interpersonal distance habits of the actual indoor environment. If the relative distance between the object and another person or item remains within 1 meter, it is regarded as one contact or approach, and only one record is made during the continuous occurrence. After a complete behavior ends, count the total number of object interactions. For multiple occurrences, accumulate or average. For example, if there are 2 interactions, 5 interactions, and 4 interactions in three detections respectively, we can let E j =(2 + 5 + 4)=11 times.

[0120] ZF j Introduction of:

[0121] ZF j represents the change rate of the j-th type of behavior. In an indoor scene, the state transitions during the occurrence of the behavior can be statistically segmented first. For example, the acceleration and deceleration frequencies during walking or the action rate during the door-opening action can also include actions with an acceleration higher than a certain threshold such as jumping or running. If the threshold is set at 2 meters per second squared, the speed increment can be estimated frame by frame in the surveillance. When it reaches or exceeds 2 meters per second squared, record one change. Subsequently, summarize multiple detections and compare with the number of behavior occurrences within the total recording duration. Finally, quantify the change amplitude into a rate index. If a certain behavior lasts for 60 seconds in one detection and the time period of sudden speed increase reaches 15 seconds, then ZF j is 15 / 60 = 0.25.

[0122] Introduction of m:

[0123] m is the total number of behavior categories, that is, how many categories of activities that do not belong to the normal state are detected in one or more monitoring cycles. Such categories are usually set or automatically generated in the monitoring system in advance, such as staying for a long time at night, frequent opening and closing of the door, suspicious wandering, etc. To obtain m, the recognition results can be classified and marked, and then it can be checked whether it exceeds the normal behavior reference range. After numbering different abnormal activities separately, the total number of categories can be obtained. In a home security scenario, if 6 suspicious behaviors are found and all have certain duration and movement paths and other characteristics, then m = 6 can be recorded.

[0124] Calculation process:

[0125] First, list the parameters of the first type of behavior. For example, A1 = 75 seconds, B = 50 seconds, C1 = 4 times, D1 = 10 meters, E1 = 3 times, ZF1 = 0.2;

[0126] List the parameters of the second type of behavior. For example, A2 = 30 seconds, C2 = 6 times, D2 = 15 meters, E2 = 5 times, ZF2 = 0.3;

[0127] Assume that there are two types of abnormal behaviors currently, then m = 2;

[0128] Substitute the parameters into the formula:

[0129]

[0130] Therefore, the summation term of the first type of behavior is 25 + 6.32 + 2.88 = 34.2;

[0131]

[0132] Therefore, the summation term of the second type of behavior is 20 + 9.49 + 4.59 = 34.08;

[0133] Further sum up the calculation results of the two types of behaviors:

[0134]

[0135] Finally, divide by m = 2 to get:

[0136]

[0137] The results show that when R = 34.14, the combined risk of the two types of suspicious behaviors that have emerged in the current monitoring is relatively high. Subsequently, the scores of other monitoring periods can be compared. If there is a significant increase or decrease in the scores, it indicates a corresponding change in the degree of abnormal behavior. Generally, a benchmark value can be set internally to assist in judgment. For example, when R exceeds 20, it is regarded as an abnormal risk that requires close attention. Different users can select a more appropriate benchmark value based on historical data and experience and incorporate this value into the overall risk assessment process.

[0138] Based on the risk level scores obtained previously, first, summarize each behavior marked as abnormal and read its corresponding score value. List the behaviors with the highest scores and the lower-scoring behaviors for comparison item by item. During this process, the ranking results of the scores will be numbered. If the score differences of several abnormal behaviors are less than 0.5 during the ranking, they will be regarded as the same score range and further subdivided according to the specific degree of difference. This threshold of 0.5 is found through multiple tests and comparisons. When the score difference is less than 0.5, it has little impact on the subsequent risk classification. Then, after obtaining the complete sorted list, check item by item in descending order of scores. Those with scores exceeding 20 are regarded as high-risk categories. This value of 20 comes from the statistical results of past monitoring records. When the score exceeds 20, it often corresponds to more obvious abnormal situations in practical cases. If the score is between 10 and 20, it is recorded as a medium-risk category. When the score is below 10, most of the test samples in this scenario correspond to situations with low vigilance. For each category of abnormal behavior, fill in the remarks information including the associated time period and location of occurrence, and classify and mark them in combination with the characteristic parameters of individual behaviors. If an abnormal behavior appears repeatedly during the monitoring period and similar signs are recorded from multiple device angles, it will be included in the priority attention list during classification. When all abnormal behaviors are sorted and classified, label them according to the risk level, distinguish the high-risk category, medium-risk category, and low-risk category in the record, and summarize them into a risk assessment result, finally forming a total list of abnormal behaviors containing risk level labels.

[0139] The steps to obtain the comprehensive risk assessment result are as follows:

[0140] Based on the risk assessment result, query the historical events in the event database that match the risk characteristics, and extract the occurrence time, geographical location, affected target, affected area, and event duration of the matching historical events to obtain the matching event data;

[0141] Based on the matching event data, calculate the risk matching score, and the expression is:

[0142]

[0143] Among them, Q is the risk matching score, and L k is the kth type of risk assessment value, and Mk is the risk assessment value for the k-th type of historical event, N is the average impact area of all matching events, P is the average duration of all matching events, S is the total number of matching feature categories between the current risk event and historical events, TE is the occurrence time of the current event, U is the occurrence time of the matching historical event, and r is the total number of matching event categories;

[0144] Based on the risk matching score, screen the historical event with the highest matching score, and conduct inductive analysis on the risk assessment result and the historical event with the highest matching score to obtain the comprehensive risk assessment result.

[0145] Specifically, based on the risk assessment results obtained earlier, first read the risk level and corresponding feature descriptions of each abnormal behavior entry. When querying the event database, key feature items need to be filled in the retrieval field, such as the specific time period when the event occurred, the possible spatial range involved, and whether it is associated with multiple locations in the room. This database is a set of historical events accumulated during daily monitoring. For each historical event, key information such as the occurrence time period, involved location, and number of personnel is registered and continuously expanded. If it is found that there are multiple matching fields between the existing historical event records and the features of the current abnormal behavior, record the occurrence time and geographical location of this historical event. In addition, read the impact target of the historical event, such as whether the event involves personnel, property, or indoor facilities. If the correlation with the current abnormal behavior is relatively high, continue to register the size of the impact area and extract the event duration. For example, the duration can generally be defined within the range of 0 minutes to 1440 minutes. Scan the previously stored event records and compare their duration fields with this interval. If it exceeds 1440 minutes, it is regarded as a cross-day event and listed separately in this stage. When all matching fields have a relatively high degree of consistency with the current abnormal behavior, they are regarded as key corresponding objects. Extract and summarize the occurrence time, geographical location, impact target, impact area, and event duration of these historical events as matching event data, so that the correlation analysis of the current abnormal behavior can be carried out according to the matching event data in the subsequent steps.

[0146] The benefit of the formula is that by simultaneously considering the total difference in the evaluation values between the current risk event and the matching historical event and the proximity of the event occurrence times, and combining the two references of the average impact range and average duration of all matching events, it can more objectively evaluate the correlation strength between the current risk event and the historical event and provide a more refined basis for risk matching scores.

[0147] L k Introduction of

[0148] L kDenote the risk assessment value of the k-th category, which is used to quantify the risk level of a specific category in the current risk event. This value is usually selected based on the previously obtained risk assessment results and can be calculated by comprehensively considering indicators such as the duration, occurrence frequency, and movement amplitude of abnormal behaviors. When there are multiple types of suspicious behaviors with different characteristics, a corresponding risk assessment value will be recorded separately for each type of behavior. In a home scenario, if we detect two different abnormal phenomena such as continuous wandering at night and repeated door opening and closing, two risk assessment values will be generated and denoted as L1 and L2 respectively. For example, after a certain monitoring, the overall score of the continuous wandering behavior at night is 27, and the score of the repeated door opening and closing behavior is 18, corresponding to L1 = 27 and L2 = 18.

[0149] M k Introduction of:

[0150] M k Denote the risk assessment value of the k-th category of historical events, and relevant numerical values need to be evaluated or extracted from the event database for past abnormal events that have occurred and been confirmed. When the on-site conditions and the final determined risk score of a historical event are recorded in the database, the score can be directly extracted as M. k , if the risk score of the historical event is not yet clear, the historical data can be reprocessed using the same evaluation method as the current system. For example, numerical analysis is performed on previously stored wandering behaviors, door opening and closing abnormalities, etc., and features such as their duration, frequency, and movement amplitude are integrated to obtain the historical risk value. If a certain historical event registration indicates that the door opened violently multiple times and lasted for a cumulative 120 seconds at that time, and after calculation, the score is 30, then M k = 30.

[0151] Introduction of N:

[0152] N represents the average affected area of all matching events. After querying the event database, historical records with high similarity to the current risk event need to be extracted, and numerical statistics are performed on their affected area fields. If the affected area can be measured by the indoor area or the number of affected locations, the area involved in each matching event can be extracted first. For example, one event affects 15 square meters, and another affects 25 square meters. After accumulating multiple events, divide by the number of matching events to obtain the average affected area. In a home scenario, the affected range generally may be from 5 square meters to 40 square meters. If it is found that 3 matching events are 10, 20, and 25 square meters respectively, then N = (10 + 20 + 25) / 3 = 18.33 square meters.

[0153] Introduction of P:

[0154] Let \(P\) denote the average duration of all matching events. It is necessary to further extract the above - matched historical events in the time dimension. By taking the difference between the start and end times of each event, the specific duration can be obtained. In an indoor environment, if the records are usually in minutes, the durations of all matching events can be aggregated at once and the mean value can be calculated. For example, if the durations of the 3 matched events are 60 minutes, 90 minutes, and 120 minutes respectively, then \(P=(60 + 90+120) / 3 = 90\) minutes.

[0155] Introduction of \(S\):

[0156] Let \(S\) be the total number of matching feature categories between the current risk event and historical events, which indicates how many features are common in the comparison process. In home security, several features can be listed, such as the time period of occurrence at night, repeated entry and exit, indoor personnel identity information, etc. When the matching events exactly match the current event in these features, 1 point is accumulated, and the number of all matchable features is added up to get \(S\). For example, if the current event has three features: "occurrence at night", "relatively large movement amplitude", and "involving two people", and the same three items are also found in a historical event, then \(S = 3\).

[0157] Introduction of \(TE\):

[0158] Let \(TE\) be the occurrence time of the current event, which is generally recorded in the form of a timestamp or a specific date plus hours and minutes. In the monitoring system, as long as the date and time period of the current abnormal event are confirmed, a quantitative time value can be stored in the database. If in integer form, 0:00 am on a certain day can be used as the starting point, and minutes or seconds can be used as the smallest unit to calculate until the moment of the event occurrence. For example, at 10 pm on a certain day, if calculated in minutes, then \(TE = 22\times60=1320\) minutes.

[0159] Introduction of \(U\):

[0160] Let \(U\) be the occurrence time of the matching historical event. The acquisition method is similar to \(TE\). When querying the event database, the historical records that meet the feature conditions are retrieved in sequence and their occurrence times are read. The time can also be converted into a value with the same measurement method as \(TE\). For example, if a historical event occurs at 8 pm, then \(U = 20\times60 = 1200\) minutes.

[0161] Introduction of \(r\):

[0162] Let \(r\) be the total number of matching event categories. After retrieving several historical events from the database, if it is found in the comparison that these historical events are not all the same as each other, they may belong to different abnormal categories or are only similar to the current event in some features. Therefore, each historical event will provide several types of category-matching data. By counting the quantities of all the categories found in the comparison, \(r\) can be obtained. For example, in a search, three historical events are found, corresponding to three different categories: nighttime break-in, long-term loitering, and abnormal noise. Then \(r = 3\) can be set.

[0163] Calculation process:

[0164] Let \(r = 2\). After querying the event database, it is found that two historical events have common features with the current risk event, corresponding to two risk assessment values \(M1 = 25\) and \(M2 = 28\) respectively;

[0165] Two assessment values of the current risk event are \(L1 = 30\) and \(L2 = 32\);

[0166] Calculation

[0167] Obtain the average impact area \(N\) and average duration \(P\) of the matching events. For example, \(N = 18\) square meters and \(P = 60\) minutes. Then

[0168] Let \(S = 2\), indicating that the current event coincides with the historical events in two key features. Let \(TE = 1320\) minutes and \(U = 1200\) minutes. Then \(|TE - U| = 120\);

[0169] Substitute the above values into the formula:

[0170]

[0171] The result shows that when \(Q\approx0.2825\), the matching degree between the current risk event and these two historical events is relatively low. A benchmark value such as \(0.5\) can be set internally for screening. If \(Q\) is greater than \(0.5\), it indicates that the current risk event has a higher coincidence degree with the historical events. If it is less than \(0.5\), it indicates a weaker correlation. Different users can adjust or select this benchmark value appropriately according to the scale and distribution of historical events in their respective environments to provide data support for subsequent disposal strategies.

[0172] Based on the risk matching scores obtained previously, first sort all the matching scores from largest to smallest, select the historical event with the highest score, and at the same time check the events with similar scores to determine if there are ties. If it is found that the scores of multiple historical events differ by no more than 0.02, they are all listed as priority matching objects of the same level. The determination range of 0.02 is derived from the analysis of the score distribution of each historical event in the system. After multiple comparisons, it is found that when the score difference is less than 0.02, the impact on the overall matching result is small. Then, check the field records of each high-score event in terms of occurrence time and affected targets, and re-compare these fields with the characteristics of abnormal behaviors in the previously obtained risk assessment results. If there is also a high degree of overlap in terms of spatial location and affected area, etc., it is included as an important reference event. Then, organize all the judgment details of this important reference event and merge it with the current risk event into the same report. During the merging process, check whether there are contradictions or omissions in the key fields. If some fields still lack necessary parameters, conduct a supplementary query in the database and add the relevant information. Finally, complete the inductive analysis of the current risk event and the historical event with the highest matching score to obtain the comprehensive risk assessment result.

[0173] The specific steps to receive the comprehensive risk assessment result and determine whether to activate the alarm and notify the user are as follows:

[0174] Receive the comprehensive risk assessment result, extract the risk level, the size of the affected area, the expected duration, and the number of affected targets, and generate a risk decision input;

[0175] Based on the risk decision input, calculate the alarm activation index, and the expression is:

[0176]

[0177] where I represents the alarm activation index, f i represents the risk level of the i-th risk event, l i represents the expected duration of the i-th risk event, LT is the sum of the durations of all risk events, a i represents the size of the area affected by the i-th risk event, A total represents the total area that can be affected, and n is the total number of events in the assessment;

[0178] Based on the alarm activation index, determine whether it exceeds the set activation threshold. If it exceeds the activation threshold, execute the alarm activation program and notify the user. If it does not exceed, maintain the monitoring status.

[0179] Specifically, for each risk level corresponding to abnormal behavior and the affected scenario elements in the comprehensive risk assessment results, first read the size of the affected area among these elements and compare its range with a pre-established benchmark. For example, in a home scenario, an area of 0 square meters to 50 square meters is often regarded as a small area, 50 square meters to 150 square meters as a medium area, and an area exceeding 150 square meters as a large area. If it is detected that the size of the affected area is between 60 square meters and 80 square meters, it is recorded as a medium area. At the same time, for the expected duration, it is also read based on the observation records. If the corresponding time value exceeds 120 minutes, it is marked as an overtime situation. This 120-minute threshold is obtained by adding 3 times the standard deviation to the mean of long-term home monitoring data. For behaviors with a duration of less than 10 minutes, they are marked as short-term phenomena. If repetitions occur in multiple device video images or multiple monitoring periods, the total number of minutes can be accumulated to form the final assessment time. Then, when synchronously reviewing the number of affected targets, list the number of people and the number of key materials affected. If the number of people exceeds 3, record their distribution. If pets or special objects are involved, add a note in the registration information. After completing the above data collection, summarize according to the risk level and form a set of input items for alarm decision-making. This input item includes four core fields: risk level, size of the affected area, expected duration, and number of affected targets, and finally obtain the risk decision input.

[0180] The advantage of the formula is that it takes into account both the cumulative effect of the risk event level and its duration, as well as the sum of squares of each event's occupancy in the affected area. By summing and then taking the square root operation to unify the index dimension, it is convenient to quickly estimate the overall risk situation in subsequent judgments.

[0181] f i Steps for obtaining:

[0182] f i represents the risk level value of the i-th risk event, which is read through the previously obtained risk assessment results. If multiple abnormal behaviors are located during the monitoring process, several risk scores will be generated. Each score comes from the comprehensive measurement based on quantitative indicators such as duration, movement amplitude, and interaction times mentioned above. In the context of home security, for example: a behavior of repeatedly opening and closing the door at night accompanied by a relatively loud noise, the risk score is obtained by counting its occurrence times and abnormal indicators as 22, and another abnormal behavior of staying indoors for a long time at night has a score of 27, which respectively correspond to f1 = 22 and f2 = 27.

[0183] l i Steps for obtaining:

[0184] l irepresents the estimated duration of the i-th risk event. The value is usually saved in minutes or seconds. During the monitoring of this event, its start and end times are marked by timestamps and subtracted to obtain the total duration of the event. The so-called "estimated duration" may also refer to historical statistical information, calculating the average value or peak interval of similar types of behaviors in a large number of monitoring records and performing weighted processing. For example, nighttime wandering usually lasts from 20 minutes to 40 minutes. If the final "estimated duration" of this abnormal behavior is 35 minutes through on-site video logs and historical reference analysis, then let l i = 35.

[0185] Steps to obtain LT:

[0186] LT represents the sum of the durations of all risk events, that is, in a comprehensive assessment, the durations of multiple abnormal behaviors are counted item by item and accumulated. This value can reflect the total persistence of multiple risk events during this monitoring period. In practice, if a total of n abnormal events are identified and the durations of each event are l1, l2,... l n , then let LT = l1 + l2 +... + l n , for example, within a day of home monitoring, 3 abnormalities are detected, and their durations are 20 minutes, 35 minutes, and 55 minutes respectively. Then LT = 20 + 35 + 55 = 110 minutes.

[0187] a i Steps to obtain:

[0188] a i represents the size of the area affected by the i-th risk event. It is necessary to first obtain the physical range involved in this event in the monitoring data and numericalize its planar area. For example, the activity areas in the living room and bedroom can be estimated separately. If the event involves frequent entry and exit between two rooms, then the sum of the areas of the two rooms plus the area of the aisle can be regarded as the total affected range. For example, the living room area is 20 square meters, the bedroom is 15 square meters, and the aisle is 5 square meters. Then the combined a i = 40 square meters. If another risk event only occurs within a 2-square-meter range in the entrance hall, then a2 = 2.

[0189] A total Steps to obtain:

[0190] A totalIt represents the total area that can be affected. When initially modeling the monitoring system, it is necessary to survey the indoor environment. Sum up the areas of all rooms, corridors, balconies and other places where activities may occur. A larger area can also be selected to include the entire residential area. In a home scenario, if the main monitoring areas are the living room, bedroom, kitchen and bathroom, first measure the length and width of each area. For example, the living room is 25 square meters, the bedroom is 15 square meters, the kitchen is 10 square meters, and the bathroom is 6 square meters. If the aisle is not included, the total is 56 square meters as A. total 。

[0191] Steps to obtain n:

[0192] n is the total number of events in the assessment, which refers to the number of all abnormal events recorded within the current monitoring period or during this risk analysis process, including different types or the same type of events that occur repeatedly. In practice, the de-duplication process can be carried out first through the previously obtained risk event identification results, and then the truly different or repeatedly occurring abnormal behaviors are counted item by item to obtain the final n. For example, if 3 types of abnormalities such as "wandering at the door late at night", "speaking loudly multiple times", and "frequently entering and leaving the kitchen" are found during a day's monitoring, then n = 3 can be recorded.

[0193] Calculation process:

[0194] Assume that during an assessment, n = 2 risk events are detected, denoted as event 1 and event 2 respectively;

[0195] The risk level of event 1, f1 = 25, the expected duration, l1 = 30 minutes, and the affected area, a1 = 25 square meters;

[0196] The risk level of event 2, f2 = 18, the expected duration, l2 = 50 minutes, and the affected area, a2 = 15 square meters;

[0197] Let LT = l1 + l2 = 30 + 50 = 80 minutes, and set the total area that can be affected as A total = 100 square meters. All the above values are selected according to the actual room area in the home monitoring scenario and the statistical range obtained after multiple risk observations.

[0198] First, calculate

[0199] Then, calculate

[0200] Then, find and

[0201]

[0202] And calculate

[0203] Substitute into the formula:

[0204]

[0205] The result shows that when the alarm activation index I≈20.62, it is possible to compare with the internally set activation threshold to determine whether to trigger an alarm. For example, if the specified threshold is 15, it means that when I exceeds 15, the alarm and subsequent notification process will be immediately activated. If the internal threshold is set at 25, it can also be concluded that the situation is still within the scope of safety monitoring at this time. Different users can adjust the threshold accordingly based on the cumulative observation results or industry standards. After substituting more events or a larger house area, I will also increase or decrease accordingly, and finally obtain the judgment basis for whether to activate the alarm.

[0206] The present invention provides a home security integration method, including the following steps:

[0207] Collect indoor and outdoor temperature, infrared data, and video images through infrared sensors, thermal imaging cameras, and standard video cameras to generate an environmental data fusion result;

[0208] Perform outlier detection, null value processing, and data format standardization on the environmental data fusion result to generate standardized monitoring data;

[0209] Based on the standardized monitoring data, perform feature point marking, extract dynamic features in the environment, and use a pre-trained deep learning network to analyze behavior patterns, identify abnormal behaviors, and generate a behavior recognition result;

[0210] Based on the behavior recognition result, divide the risk levels of each behavior pattern, identify risk behaviors, and perform risk analysis by matching with historical event data to generate a comprehensive risk assessment result;

[0211] Receive the comprehensive risk assessment result, determine whether to trigger an alarm according to the risk level, and select SMS or application push to inform the user of the current security status to generate a response and notification result.

Claims

1. An artificial intelligence-based home security integration system, characterized in that, The system includes: A multi-source data acquisition module that collects environmental data through infrared sensors, thermal imaging cameras, and standard video cameras, performs data fusion to generate an environmental data fusion result; and based on the environmental data fusion result, performs data cleaning and format unification processing to obtain standardized monitoring data. An intelligent analysis and recognition module that uses the standardized monitoring data to perform feature extraction to obtain a feature extraction result; and based on the feature extraction result, uses a pre-trained deep learning model to perform behavior recognition and anomaly detection to generate a behavior recognition result. A risk assessment and early warning module that, based on the behavior recognition result, performs risk level assessment to obtain a risk assessment result; and matches the risk assessment result with an event database to generate a comprehensive risk assessment result. A response and notification module that receives the comprehensive risk assessment result and determines whether to activate an alarm and notify the user.

2. The home security integration system based on artificial intelligence according to claim 1, characterized in that, The steps for obtaining the environmental data fusion result are as follows: Collect infrared data, thermal images, and visible light video streams through infrared sensors, thermal imaging cameras, and standard video cameras respectively to form an original environmental data set. Based on the original environmental data set, perform spatio-temporal alignment on the data from the sensors and eliminate data redundancy to obtain the environmental data fusion result.

3. The home security integration system based on artificial intelligence according to claim 1, wherein The steps for obtaining the standardized monitoring data are as follows: Based on the environmental data fusion result, identify and delete noisy or corrupted data to form a preliminarily cleaned data set. Based on the preliminarily cleaned data set, convert all data to a unified encoding format and synchronize all timestamps to generate standardized monitoring data.

4. The home security integration system based on artificial intelligence according to claim 1, characterized in that, The steps for obtaining the feature extraction result are as follows: Based on the standardized monitoring data, calculate the information gain value, and the calculation formula is: Among them, T represents the total dataset, F represents the feature set, |T f | is the number of datasets under the feature f, |T| is the total number of data, H(T) is the entropy of the dataset T, H(T f ) is the entropy of the dataset under the feature f, and IG(T,F) represents the information gain value of the total dataset T relative to the feature set F; Based on the information gain value, screen the feature combination with the highest information gain value to obtain the feature extraction result.

5. The home security integration system based on artificial intelligence according to claim 1, characterized in that The steps for obtaining the behavior recognition result are as follows: Input the feature extraction result into a pre-trained deep learning model, and sequentially complete data propagation according to the deep learning model structure, including input layer data mapping, intermediate layer feature transformation, and output layer classification calculation, and perform the activation function operation of each layer of neurons to form a preliminary behavior recognition result. Based on the preliminary behavior recognition result, perform anomaly detection on the recognized behavior categories, analyze the matching degree between the current behavior category and the normal behavior pattern, and obtain the behavior recognition result.

6. The home security integration system based on artificial intelligence according to claim 1, characterized in that The steps for obtaining the risk assessment result are as follows: Based on the behavior recognition result, extract the behavior type, duration, occurrence times, change range of the movement path, and object interaction situation to construct behavior feature data. Based on the behavior feature data, calculate the risk level score, and the expression is: Among them, R is the risk level score, A j is the duration of the j-th type of behavior, B is the average duration of normal behavior, C j is the occurrence frequency of the j-th type of behavior, D j is the change range of the movement path of the j-th type of behavior, E j is the number of object interactions of the j-th type of behavior, ZF j is the change rate of the j-th type of behavior, and m is the total number of behavior categories; Based on the risk level score, sort all abnormal behaviors, classify them according to the risk level score, and label the risk levels of abnormal behaviors to obtain the risk assessment result.

7. The home security integration system based on artificial intelligence according to claim 1, characterized in that, The steps for obtaining the comprehensive risk assessment result are as follows: Based on the risk assessment result, query the historical events in the event database that match the risk features, and extract the occurrence time, geographical location, affected target, affected area, and event duration period of the matching historical events to obtain the matching event data. Based on the matching event data, calculate the risk matching score, with the expression: Among them, Q is the risk matching score, and L k is the risk assessment value of the k-th type of risk, M k is the risk assessment value of the k-th type of historical event, N is the average impact area of all matching events, P is the average duration of all matching events, S is the total number of matching feature categories between the current risk event and the historical event, TE is the occurrence time of the current event, U is the occurrence time of the matching historical event, and r is the total number of matching event categories; Based on the risk matching score, screen the historical event with the highest matching score, and conduct inductive analysis on the risk assessment result and the historical event with the highest matching score to obtain the comprehensive risk assessment result.

8. The home security integration system based on artificial intelligence according to claim 1, wherein The specific steps for receiving the comprehensive risk assessment result and determining whether to activate the alarm and notify the user are as follows: Receive the comprehensive risk assessment result, extract the risk level, the size of the affected area, the expected duration, and the number of affected targets, and generate a risk decision input; Based on the risk decision input, calculate the alarm activation index, with the expression: Among them, I represents the alarm activation index, f i represents the risk level of the i-th risk event, l i represents the expected duration of the i-th risk event, LT is the sum of the durations of all risk events, a i represents the area size affected by the i-th risk event, A total represents the total affected area, and n is the total number of events in the assessment; Based on the alarm activation index, determine whether it exceeds the set activation threshold. If it exceeds the activation threshold, execute the alarm activation procedure and notify the user. If it does not exceed, maintain the monitoring status.

9. The home security integration method of the home security integration system based on artificial intelligence according to any one of claims 1-8, characterized in that, Include the following steps: Collect the indoor and outdoor temperature, infrared data, and video images through an infrared sensor, a thermal imaging camera, and a standard video camera to generate an environmental data fusion result; Perform outlier detection, null value processing, and data format standardization on the environmental data fusion result to generate standardized monitoring data; Based on the standardized monitoring data, mark the feature points, extract the dynamic features in the environment, and use a pre-trained deep learning network to analyze the behavior patterns, identify abnormal behaviors, and generate a behavior recognition result; Based on the behavior recognition result, classify the risk levels of each behavior pattern, identify risk behaviors, and conduct risk analysis by matching with historical event data to generate a comprehensive risk assessment result; Receive the comprehensive risk assessment result, determine whether to trigger an alarm according to the risk level, and select SMS or application push to inform the user of the current security status, generating a response and notification result.

Images