Remote monitoring alarm system and method for intelligent safety box

Abstract

The invention relates to the field of safe case remote monitoring, and particularly discloses a remote monitoring alarm system and method for an intelligent safe case, and the method comprises the steps: firstly obtaining a safe case monitoring image collected by a 360-degree 3D panoramic infrared night vision camera and vibration information, collected by a vibration sensor, of the safe case at a plurality of preset time points; and then feature extraction and correlation analysis are carried out on the two by using a deep learning technology, and finally, whether a safe case remote early warning is sent out is judged through a classifier, so that the safety of the safe case is greatly improved, the loss and risk are reduced, and meanwhile, the response efficiency and timeliness are improved.

Description

Technical Field

[0001] The present application relates to the field of remote monitoring of safes, and more specifically, to a remote monitoring alarm system and method for smart safes. Background Art

[0002] A safe is a special container, which is mainly divided into fireproof safes, anti-theft safes, anti-magnetic safes, fireproof anti-magnetic safes, etc. according to its function. Each type of safe has its national standard. Safes are generally equipped with monitors to record the criminal behavior of stealing the safe, thereby improving the confidentiality and safety of use.

[0003] Although existing safes are equipped with monitors, they cannot be fully remotely monitored, the monitoring angles are incomplete, and remote comprehensive warning reminders cannot be performed. The safety factor is low and the remote operability is poor.

[0004] Therefore, a remote monitoring alarm system and method for a smart safe is desired. Summary of the invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a remote monitoring alarm system and method for a smart safe, which first obtains the safe monitoring image collected by a 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by a vibration sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally uses a classifier to determine whether to issue a remote warning of the safe, thereby greatly improving the safety of the safe, reducing losses and risks, and improving the efficiency and timeliness of the response.

[0006] According to one aspect of the present application, a remote monitoring alarm system for a smart safe is provided, comprising:

[0007] A safe monitoring data acquisition module is used to acquire the safe monitoring images collected by a 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by a vibration sensor;

[0008] A safe monitoring data extraction module, used to extract a safe monitoring noise reduction vector and optimize a safe vibration information multimodal feature vector from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor;

[0009] The safe remote warning judgment module is used to judge whether to issue a safe remote warning based on the safe monitoring noise reduction vector and the optimized safe vibration information multi-modal feature vector.

[0010] According to another aspect of the present application, a remote monitoring alarm method for a smart safe is provided, comprising:

[0011] Acquire the monitoring images of the safe collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor;

[0012] Extracting a safe monitoring noise reduction vector and optimizing a safe vibration information multimodal feature vector from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor;

[0013] Based on the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector, it is determined whether to issue a safe remote warning.

[0014] Compared with the prior art, the present application provides a remote monitoring alarm system and method for a smart safe, which first obtains the safe monitoring image collected by a 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by a vibration sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two, and finally uses a classifier to determine whether to issue a remote warning of the safe, thereby greatly improving the safety of the safe, reducing losses and risks, and at the same time improving the efficiency and timeliness of the response. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 The block diagram is a remote monitoring alarm system for a smart safe according to an embodiment of the present application.

[0017] Figure 2 The present invention is a block diagram of a safe monitoring data extraction module in a remote monitoring alarm system for a smart safe according to an embodiment of the present application.

[0018] Figure 3 The present invention is a block diagram of a safe remote early warning judgment module in a remote monitoring alarm system for a smart safe according to an embodiment of the present application.

[0019] Figure 4 The present invention is a flowchart of a remote monitoring and alarm method for a smart safe according to an embodiment of the present application.

[0020] Figure 5 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0022] Figure 1 FIG. 1 is a block diagram of a remote monitoring alarm system for a smart safe according to an embodiment of the present application. Figure 1 As shown, according to the embodiment of the present application, the remote monitoring alarm system 100 for a smart safe includes: a safe monitoring data acquisition module 110, which is used to obtain the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor; a safe monitoring data extraction module 120, which is used to extract the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor; a safe remote early warning judgment module 130, which is used to judge whether to issue a safe remote early warning based on the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector.

[0023] In the above-mentioned remote monitoring alarm system 100 for smart safes, the safe monitoring data acquisition module 110 is used to obtain the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor. It should be understood that the safe, as a special safety container, is divided into fireproof, anti-theft, anti-magnetic and comprehensive fireproof and anti-magnetic types according to different functions, each of which is designed according to national standards. Modern safes are usually equipped with monitoring facilities to record potential intrusion behaviors to improve the confidentiality and security of use. However, the existing safe monitoring system has some limitations: the monitoring coverage is not comprehensive, it is impossible to achieve all-round remote monitoring and comprehensive warning reminders, and the safety and convenience of operation need to be improved. Therefore, in the technical solution of the present application, by obtaining the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor, and combining the deep learning technology, it is judged to issue a remote warning of the safe to enhance the safety of the safe, reduce the possible losses and risks, and significantly improve the response speed and real-time performance.

[0024] Specifically, through the 360-degree panoramic camera, the system can capture and record visual information inside and outside the safe in real time, including subtle movements and changes, whether it is day or night, and the infrared night vision function ensures that clear images can be obtained even in low-light environments. This visual data can not only be used for real-time monitoring, but also extract features through image processing technology to identify abnormal events and identities. At the same time, the data collected by the vibration sensor provides real-time feedback on the status of the safe, including opening and closing doors, movement or other potential unauthorized access. By combining these two data sources, the system can more accurately analyze and identify potential threats or abnormal behaviors, take preventive measures or sound alarms in time, and effectively protect the safety of valuables in the safe. This comprehensive monitoring system not only improves security, but also provides users with a reliable means to ensure that their property is effectively protected and supervised.

[0025] In the above-mentioned remote monitoring alarm system 100 for smart safes, the safe monitoring data extraction module 120 is used to extract the safe monitoring noise reduction vector and optimize the safe vibration information multimodal feature vector from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor. It should be understood that the safe monitoring noise reduction vector and the vibration information multimodal feature vector are extracted from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of multiple predetermined time points collected by the vibration sensor, in order to achieve efficient monitoring and security management of smart safes. By comprehensively utilizing the monitoring noise reduction vector and the vibration information multimodal feature vector, the system can achieve more accurate security assessment and risk management. It is not just a simple response to a single monitoring indicator, but through comprehensive analysis of information from different data sources, it realizes comprehensive control of complex environments and potential threats. Therefore, this data extraction and processing method not only improves the intelligence level of the safe monitoring system, but also ensures the safety and protection effect of valuables in the safe.

[0026] Figure 2 FIG. 1 is a block diagram of a safe monitoring data extraction module in a remote monitoring alarm system for a smart safe according to an embodiment of the present application. Figure 4As shown, in a specific embodiment of the present application, the safe monitoring data extraction module 120 includes: a safe monitoring image feature extraction unit 121, which is used to extract features from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera to obtain the safe monitoring noise reduction vector; a vibration information frequency domain statistical feature encoding unit 122, which is used to perform frequency domain statistical feature encoding on the vibration information of the safe at multiple predetermined time points collected by the vibration sensor to obtain the safe vibration frequency domain feature vector; a vibration information vibration waveform feature encoding unit 123, which is used to perform vibration waveform feature encoding on the vibration information of the safe at multiple predetermined time points collected by the vibration sensor to obtain the safe vibration signal waveform feature vector; a safe vibration information feature construction unit 124, which is used to construct the safe vibration frequency domain feature vector and the safe vibration signal waveform feature vector to obtain the optimized safe vibration information multimodal feature vector.

[0027] It should be understood that the 360-degree 3D panoramic camera combined with infrared night vision technology provides a wide monitoring field of view and image capture capabilities under different lighting conditions. These cameras are able to capture the overall environment around the safe at the same time, including possible intruders, abnormal behavior or other potential threats. However, due to the complexity of the monitoring environment and the technical limitations of the camera itself, the original image collected by the camera may contain a lot of irrelevant information or noise, which will affect the performance and accuracy of the monitoring system. Through the feature extraction process, the monitoring system is able to extract important information and specific patterns from the image, such as human outlines, moving objects, specific actions, etc. These features can include color histograms, texture features, shape descriptors, etc. Through the combination and analysis of these features, the system can generate a safe monitoring noise reduction vector. This vectorized representation not only helps to reduce noise and unnecessary details in the image, but also enables the system to focus more on the monitoring and response of key events, improving monitoring efficiency and real-time performance.

[0028] Furthermore, the frequency domain statistical feature encoding of the safe vibration information collected by the vibration sensor is intended to achieve comprehensive monitoring and analysis of the safe status and behavior by deeply analyzing the frequency domain characteristics of the vibration signal. The vibration sensor can capture various vibrations occurring inside the safe in real time, such as door opening and closing, object movement, or unauthorized access attempts. These vibration data can be converted into frequency and amplitude distribution characteristics through frequency domain analysis, thereby revealing the key information hidden behind the vibration signal. Frequency domain statistical feature encoding involves converting the vibration signal to the frequency domain, and converting the time domain signal into a frequency domain energy distribution by applying Fourier transform or other frequency domain analysis techniques. The purpose of this is to extract the main frequency components and energy distribution in the vibration signal, which can reflect the characteristics of different vibration sources.

[0029] Furthermore, the vibration information of the safe collected by the vibration sensor at multiple predetermined time points is encoded with vibration waveform features, aiming to achieve detailed monitoring and identification of activities and abnormal conditions inside the safe by analyzing and encoding the time domain waveform of the vibration signal. The vibration sensor can capture various vibration events occurring inside the safe, such as the opening and closing of the door, the movement of objects or external force interference, and these signals can be converted into numerical vectors with specific meanings through vibration waveform feature encoding. Among them, vibration waveform feature encoding involves extracting key waveform features from the original time domain vibration signal. These features may include the amplitude, frequency, waveform shape, duration, etc. of the vibration. Through different signal processing and analysis techniques (such as wavelet transform, time domain feature extraction algorithm, etc.), the complex vibration waveform is converted into a feature vector that can be quantified and compared. Generating vibration signal waveform feature vectors enables the monitoring system to more accurately identify and classify different types of vibration behaviors. For example, the vibration pattern of normal operation and the vibration pattern of abnormal conditions (such as unauthorized unlocking attempts or violent lock picking behaviors) will show different waveform features. By analyzing and comparing these feature vectors, the system can detect potential security issues in real time and respond in a timely manner.

[0030] In particular, the fusion of the safe vibration frequency domain feature vector and the vibration signal waveform feature vector is used to obtain the optimized safe vibration information multimodal feature vector, aiming to achieve a comprehensive analysis and evaluation of the internal activities and status of the safe by comprehensively utilizing information in both the frequency domain and the time domain. The vibration frequency domain feature vector provides a detailed description of the frequency distribution and energy distribution of the vibration signal, which can reflect the frequency characteristics and changing trends of different vibration events. The vibration signal waveform feature vector reveals the specific dynamics and patterns of the vibration event through parameters such as the shape, amplitude and duration of the time domain waveform.

[0031] By fusing these two types of feature vectors, the multifaceted characteristics of the vibration signal can be described and represented in a comprehensive feature space. For example, by combining frequency domain features with time domain features, different types of vibration behaviors, such as normal operations, unauthorized attempts, or destructive behaviors, can be more accurately distinguished. The generation of this multimodal feature vector enables the monitoring system to more accurately identify and classify various safe internal activities in different scenarios and vibration backgrounds.

[0032] In a specific embodiment of the present application, the safe monitoring image feature extraction unit 121 includes: passing the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera through a safe monitoring image denoising module to obtain a safe monitoring denoised image; passing the safe monitoring denoising image through a safe monitoring denoising convolutional neural network model as a feature encoder to obtain the safe monitoring denoising vector.

[0033] It should be understood that the safe monitoring image noise reduction module can effectively remove or reduce noise and unnecessary details in the image through a series of image processing techniques, such as filtering, dimensionality reduction, pattern recognition, etc. These technologies can not only remove visual noise in the image, but also improve the contrast and clarity of the image, so that the monitoring system can still maintain efficient monitoring capabilities under complex lighting conditions. Specifically, the noise reduction module can automatically or semi-automatically adjust the brightness, contrast and color balance of the image according to the specific attributes of the image and preset standards, thereby optimizing the visual effect of the image. This optimization not only improves the visual perception during manual operation, but also provides clearer and more accurate input data for image recognition and analysis algorithms, thereby improving the overall performance and efficiency of the monitoring system.

[0034] Furthermore, the safe monitoring denoising image is processed by the safe monitoring denoising convolutional neural network model as a feature encoder to obtain the safe monitoring denoising vector, aiming to use deep learning technology to extract and encode key features in the image, so as to achieve efficient representation and analysis of monitoring data. After the safe monitoring image is denoised, it usually still contains a large amount of complex visual information and details, which need to be further abstracted and encoded for subsequent data analysis and security event detection. Among them, the safe monitoring denoising convolutional neural network model, as a feature encoder, can effectively capture the spatial hierarchy and structural features in the image through multi-layer convolution and pooling operations. These features include but are not limited to visual attributes such as shape, texture, and edge of the object. By learning and optimizing the weights and connections of the neural network, the model can automatically extract the most informative features, which are crucial for the recognition and classification of activities inside the safe. Specifically, the safe monitoring denoising convolutional neural network model converts the input denoised image into a high-dimensional feature vector. These vectors not only contain the abstract representation of the image at multiple levels in the convolutional neural network, but also are processed and normalized by the fully connected layer to ensure the validity and stability of the features. Such a processing process enables the original image data to be mapped into a vector space with more semantic information and abstract expression, and these vectors can be directly used for subsequent monitoring event recognition, anomaly detection and prediction tasks. Specifically, each layer of the safe monitoring denoising convolutional neural network model used as a feature encoder performs convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward pass of the layer to output the safe monitoring denoising vector by the last layer of the safe monitoring denoising convolutional neural network model used as a feature encoder, wherein the input of the safe monitoring denoising convolutional neural network model used as a feature encoder is the safe monitoring denoising image.

[0035] In a specific embodiment of the present application, the vibration information frequency domain statistical feature encoding unit 122 includes: performing Fourier transform on the vibration information of the safe at multiple predetermined time points collected by the vibration sensor to obtain multiple safe vibration frequency domain statistical feature values; passing the multiple safe vibration frequency domain statistical feature values ​​through the sequence encoder of the safe vibration information joint encoder to obtain the safe vibration frequency domain feature vector.

[0036] It should be understood that, through Fourier transform, the time domain vibration signal collected by the vibration sensor can be decomposed into vibration components at different frequencies. These frequency components reflect the vibration characteristics of the safe under different activities or interference conditions, such as opening and closing doors, moving objects or external impacts. By calculating the vibration signal after Fourier transform, the vibration intensity or energy value corresponding to each frequency can be obtained, thereby forming multiple frequency domain statistical eigenvalues.

[0037] Furthermore, multiple safe vibration frequency domain statistical eigenvalues ​​are processed by the sequence encoder of the safe vibration information joint encoder, aiming to effectively capture and encode the time series information and correlation in the vibration eigenvalue sequence through the structure and algorithm of the sequence encoder, thereby improving the deep understanding and analysis ability of the safe vibration behavior. The vibration frequency domain statistical eigenvalues ​​reflect the characteristics of the vibration signal in the frequency domain at different time points, including frequency components, energy distribution and change trends. The sequence encoder of the safe vibration information joint encoder can use structures such as recurrent neural network (RNN) or long short-term memory network (LSTM) to gradually process and learn the input vibration frequency domain statistical eigenvalue sequence. These sequence encoders can maintain the time order of vibration data and effectively capture the time series dependency between vibration eigenvalues ​​during the encoding process, thereby generating a more representative and information-rich vibration frequency domain feature vector. The vibration frequency domain feature vector generated by the sequence encoder not only retains the time series information of the original data, but also organically integrates the frequency domain statistical features of multiple time points of the vibration event. This comprehensive feature vector is not only more representative, but also can reflect the overall pattern and change trend of the safe's vibration behavior at a higher level, providing a more reliable and accurate basis for subsequent pattern recognition, anomaly detection and predictive analysis. The generation of the frequency domain feature vector of the safe vibration is also helpful to improve the efficiency of data processing and the real-time response capability of the system through the sequence encoder. Compared with simple feature splicing or averaging processing, the sequence encoder can effectively extract important time series patterns and change trends in the data, providing a more refined and intelligent vibration analysis capability for the monitoring system. Specifically, the embedding layer of the sequence encoder of the safe vibration information joint encoder is used to map the multiple safe vibration frequency domain statistical feature values ​​to word embedding vectors to obtain a sequence of word embedding vectors; the sequence encoder of the safe vibration information joint encoder is used to perform global context semantic encoding on the sequence of word embedding vectors to obtain multiple feature vectors; the multiple feature vectors are cascaded to obtain the frequency domain feature vector of the safe vibration.

[0038] In a specific embodiment of the present application, the sequence encoder of the safe vibration information joint encoder is a safe vibration information context encoder including an embedding layer.

[0039] It should be understood that the embedding layer transforms each eigenvalue into a more compact and information-rich vector representation by learning the semantic relationship and mutual influence between the vibration frequency domain statistical eigenvalues. This transformation enables the sequence encoder to understand the characteristics and changes of the vibration data at a higher level, rather than just the original frequency domain statistical values. As a context encoder, the sequence encoder not only considers the arrangement order of the vibration information in the time series, but also provides additional semantic information and context associations through the embedding layer. This combination enables the vibration data to comprehensively consider the time dependency and semantic association during the encoding process, thereby generating a more accurate and complete vibration frequency domain feature vector. In addition, the embedding layer also helps to reduce the dimension and complexity of the data, and improve the processing efficiency and calculation speed of the sequence encoder for vibration information. By effectively compressing and representing the vibration frequency domain features, the embedding layer can not only optimize the computational cost of the sequence encoder, but also reduce the noise and unnecessary information loss in the data processing process. As a safe vibration information context encoder containing an embedding layer, the sequence encoder of the safe vibration information joint encoder effectively improves the understanding and expression capabilities of vibration data by combining embedding technology and sequence learning algorithms. The application of this technology provides a more flexible and efficient solution for the intelligent monitoring system in vibration data analysis and anomaly detection, and promotes the technological progress and intelligent development of the safe security management system.

[0040] In a specific embodiment of the present application, the vibration information vibration waveform feature encoding unit 123 includes: extracting a waveform graph of the safe vibration signal from the vibration information of the safe at multiple predetermined time points collected by the vibration sensor; passing the waveform graph of the safe vibration signal through the image encoder of the safe vibration information joint encoder to obtain a safe vibration signal waveform feature graph; and pooling the safe vibration signal waveform feature graph to obtain the safe vibration signal waveform feature vector.

[0041] It should be understood that the waveform diagram can intuitively display the vibration behavior of the safe at different time points by presenting the time domain vibration signal in a graphical manner. This intuitive display helps operators or analysis experts quickly identify and understand the basic characteristics of the vibration signal, such as the intensity of the vibration, the change in frequency, and the duration of the vibration, thereby providing a reliable basis for the evaluation of the system operation status and the monitoring of the safety status. The waveform diagram can help identify and distinguish between normal vibration and abnormal vibration events. By comparing and analyzing the vibration waveforms in different time periods, sudden vibration changes or abnormal patterns, such as unauthorized access, object movement, or accidental collisions, can be detected, and then necessary safety measures and countermeasures can be taken in time to ensure the safety of the property and data inside the safe. The waveform diagram also provides an important data basis for subsequent vibration data analysis. Through quantitative analysis of the waveform diagram, the time domain characteristics of the vibration signal, such as peak value, period, amplitude change, etc., can be extracted. These characteristics are an important basis for further application of signal processing algorithms and machine learning models for vibration pattern recognition and anomaly detection.

[0042] Furthermore, the waveform of the safe vibration signal is processed by the image encoder of the safe vibration information joint encoder, aiming to use image coding technology to convert the time domain vibration data into a more abstract and advanced representation. The image encoder can extract key features such as vibration mode, frequency distribution and duration in the waveform to form a waveform feature map with more information and representation. The image encoder can convert the pixel information of the time domain vibration waveform image into a compact feature vector through a deep learning algorithm. This feature vector not only contains the visual features in the waveform, such as shape, curve and color distribution, but also captures the complex patterns and correlations hidden in the vibration signal, providing a higher level of data expression. The vibration signal waveform feature map generated by the image encoder can effectively reduce the dimension and complexity of the data while maintaining the richness and accuracy of the information. This data conversion not only helps to optimize the storage and transmission efficiency, but also improves the speed and accuracy of subsequent data analysis and pattern recognition, providing more powerful technical support for the real-time response and safety management of the vibration monitoring system. Specifically, each layer of the image encoder of the safe vibration information joint encoder performs convolution processing, mean pooling processing based on the local feature matrix and nonlinear activation processing on the input data in the forward transmission of the layer, so that the last layer of the image encoder of the safe vibration information joint encoder outputs the waveform feature map of the safe vibration signal, wherein the input of the image encoder of the safe vibration information joint encoder is the waveform map of the safe vibration signal.

[0043] Furthermore, the pooling operation combines the eigenvalues ​​of each image region into a single value or vector by applying a pooling kernel (such as maximum pooling or average pooling) on ​​the waveform feature map. This operation not only reduces the size of the data, but also retains important waveform features, such as the main frequency components of the vibration, amplitude changes, and the overall shape of the waveform, thereby ensuring that the feature vector can effectively capture and express the key information of the vibration signal. The vibration signal waveform feature vectors generated by pooling are more abstract and versatile. These feature vectors can not only effectively maintain the integrity and representativeness of the information while greatly reducing the amount of data, but also provide better input data for subsequent machine learning models. For example, feature vectors can more accurately distinguish different types of vibration patterns or identify abnormal events, thereby improving the accuracy and efficiency of the safe security monitoring system.

[0044] In the above-mentioned remote monitoring alarm system 100 for smart safes, the safe remote warning judgment module 130 is used to determine whether to issue a safe remote warning based on the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector. It should be understood that the monitoring noise reduction vector removes background noise and irrelevant information by processing and filtering signal data from various sensors, extracting key features related to vibration events. These features may include the spectral distribution, energy change and time series pattern recognition of vibration signals, so as to effectively detect and classify abnormal vibrations. The vibration information multimodal feature vector combines vibration data obtained from different sensors, such as acceleration sensors, pressure sensors, etc., and fuses various types of data into a comprehensive feature vector through multimodal fusion technology. This fusion can comprehensively consider the changes and mutual relationships of multiple vibration features in the time domain and frequency domain, and provide the system with a more comprehensive and accurate vibration feature expression. The process of determining whether to issue a remote warning mainly depends on pre-set security rules and algorithms. For example, a threshold based on vibration frequency can be set, and when the frequency detected in the monitoring noise reduction vector exceeds the preset range or abnormal frequency distribution, the system will trigger an alarm. At the same time, the use of multimodal feature vectors of vibration information can enhance the accuracy of abnormal situations, such as combining changes in vibration patterns and spectral energy distribution to confirm whether there is suspicious activity or attack behavior.

[0045] In a specific embodiment of the present application, the safe vibration information feature construction unit includes: fusing the safe vibration frequency domain feature vector and the safe vibration signal waveform feature vector to obtain a safe vibration information multimodal feature vector; based on the safe monitoring noise reduction vector, enhancing the causal correlation of the safe vibration information multimodal feature vector based on reverse optimization perception to obtain an optimized safe vibration information multimodal feature vector.

[0046] In the technical solution of the present application, it is considered that the safe monitoring denoising vector is extracted based on visual information (image), while the safe vibration information multimodal feature vector is extracted based on physical vibration data. These two data types reflect different physical phenomena, which may lead to differences in their feature expressions. The safe monitoring denoising vector extracts features through a convolutional neural network model, focusing on capturing visual features such as texture and shape in the image; while the safe vibration information multimodal feature vector extracts frequency domain statistical features through Fourier transform and sequence encoder, and extracts waveform features through an image encoder. Different feature extraction methods may lead to semantic differences in feature vectors. : Vibration information is dynamic data that changes over time, while the monitoring image is static. This difference in the time dimension may cause the vibration feature vector to be unable to fully reflect the dynamic changes in the monitoring image. When fusing the safe monitoring denoising vector and the safe vibration information multimodal feature vector, if the fusion strategy does not fully consider the temporal and spatial characteristics of the two, it may cause the safe vibration information multimodal feature vector to have a missing a posteriori-priori causal relationship relative to the safe monitoring denoising vector. In order to solve this problem, in the technical solution of the present application, based on the safe monitoring noise reduction vector, the causal correlation of the multimodal feature vector of the safe vibration information is enhanced based on the reverse optimization perception.

[0047] Specifically, based on the safe monitoring noise reduction vector, the causal correlation enhancement based on the inverse optimization perception is performed on the multimodal feature vector of the safe vibration information to obtain an optimized multimodal feature vector of the safe vibration information, including: calculating the position difference between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector to obtain a first viewpoint measurement representation vector; calculating the position point multiplication between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector to obtain a second viewpoint measurement representation vector; calculating the position sum between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector to obtain a third viewpoint measurement representation vector. ; calculating the gradient amplitude of each position in the first viewpoint measurement representation vector, the second viewpoint measurement representation vector and the third viewpoint measurement representation vector to obtain the first viewpoint measurement perception vector, the second viewpoint measurement perception vector and the third viewpoint measurement perception vector; inputting the first viewpoint measurement perception vector, the second viewpoint measurement perception vector and the third viewpoint measurement perception vector into the backward machine learning reverse multi-viewpoint perception optimization module to obtain the reverse multi-viewpoint perception optimization representation vector; calculating the position point multiplication between the reverse multi-viewpoint perception optimization representation vector and the safe vibration information multi-modal feature vector to obtain the optimized safe vibration information multi-modal feature vector.

[0048] The backward machine learning reverse multi-view perception optimization module is expressed as follows:

[0049]

[0050] in, and represents the eigenvalues ​​of the i-th position of the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector, respectively, and μ represents the three eigenvalues ​​of the corresponding positions in the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector , σ is the standard deviation of the three eigenvalues ​​at corresponding positions in the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector, represents the maximum value of the three eigenvalues ​​at the corresponding positions in the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector, ReLU represents a linear rectification function, softmax represents a normalized exponential function, and v ri Represents the eigenvalue of the i-th position of the inverse multi-view perceptual optimization representation vector.

[0051] Specifically, in the technical solution of the present application, based on the safe monitoring noise reduction vector, the multimodal feature vector of the safe vibration information is optimized for causal correlation enhancement based on reverse optimization perception, and the process includes first calculating the position difference, point multiplication and sum between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector through multi-view analysis (Multi-View Analysis) to obtain three viewpoint metric representation vectors, and quantify the relationship between features from different angles. Then, the gradient amplitude of each position in these viewpoint metric representation vectors is calculated through backward error perception to identify key change points to provide a basis for feature optimization.

[0052] Then, the first viewpoint measurement perception vector, the second viewpoint measurement perception vector and the third viewpoint measurement perception vector are input into the backward machine learning reverse multi-viewpoint perception optimization module, wherein the backward machine learning reverse multi-viewpoint perception optimization module captures the significant comprehensive ability factor of the feature importance perception (Feature Importance Perception) of the feature sub-dimension distribution of the corresponding position in the first to third viewpoint measurement perception vectors by adopting a squeeze-incentive mechanism composed of -functions. Then, the position point multiplication between the reverse multi-viewpoint perception optimization representation vector and the multi-modal feature vector of the safe vibration information is calculated to perform backward optimization perception (Backward Optimization Perception) on the multi-modal feature vector of the safe vibration information to adjust the multi-modal feature vector of the safe vibration information so as to better associate it with the safe monitoring noise reduction vector, thereby enhancing the causal relationship between the multi-modal feature vector of the safe vibration information and the safe monitoring noise reduction vector.

[0053] Figure 3 FIG. 1 is a block diagram of a remote warning judgment module for a safe in a remote monitoring alarm system for a smart safe according to an embodiment of the present application. Figure 3 As shown, in a specific embodiment of the present application, the safe remote warning judgment module 130 includes: a safe monitoring information feature association unit 131, which is used to associate the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector to obtain a safe remote warning judgment classification feature vector; a safe remote warning judgment classification unit 132, which is used to classify the safe remote warning judgment classification feature vector to obtain a classification result, and the classification result is used to determine whether to issue a safe remote warning.

[0054] It should be understood that the main purpose of associating the monitoring noise reduction vector and the vibration information multimodal feature vector is to use these comprehensive feature vectors for remote early warning judgment. The early warning judgment classification feature vector can learn and identify different vibration modes and abnormal event types from historical data by setting pre-defined security rules and algorithms, such as based on machine learning models or deep learning algorithms. These feature vectors can help the system automatically detect and analyze the vibration signals of the safe, identify whether there are abnormal behaviors or security threats, so as to take necessary security measures and early warning notifications in time. Associating the safe monitoring noise reduction vector and the vibration information multimodal feature vector to generate a remote early warning judgment classification feature vector can not only comprehensively utilize multiple data sources and information features to improve the comprehensiveness and accuracy of security assessment, but also enhance the system's security protection capabilities and emergency response efficiency through intelligent analysis and prediction.

[0055] Furthermore, the classification feature vector of the remote warning judgment of the safe is classified to obtain the classification result, and then determine whether to issue a remote warning of the safe, in order to map the complex feature vector to the preset security rules and standards through effective pattern recognition and classification algorithms, so as to realize the automatic evaluation and warning triggering of the security status of the safe. Using the classification results for remote warning judgment not only simplifies the process of security management, but also improves the reaction speed and decision-making efficiency. This intelligent security warning system can effectively reduce human errors and omissions, while providing a safer and more reliable protection environment for safe users, thereby ensuring the security of valuables and sensitive information.

[0056] In summary, by processing the classification feature vector of the safe remote early warning judgment through a classification algorithm, it is possible to realize intelligent analysis and evaluation of vibration data, effectively judge and predict potential safety risks, and thus take necessary preventive measures and safety response strategies in a timely manner.

[0057] In summary, the embodiment of the present application first obtains the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor, and then uses deep learning technology to perform feature extraction and correlation analysis on the two. Finally, a classifier is used to determine whether to issue a remote warning for the safe, thereby greatly improving the safety of the safe, reducing losses and risks, and at the same time improving the efficiency and timeliness of the response.

[0058] As described above, the remote monitoring alarm system 100 for a smart safe according to an embodiment of the present application can be implemented in various terminal devices. In one example, the remote monitoring alarm system 100 for a smart safe can be integrated into a terminal device as a software module and / or a hardware module. For example, the remote monitoring alarm system 100 for a smart safe can be a software module in the operating system of the terminal device, or can be an application developed for the terminal device; of course, the remote monitoring alarm system 100 for a smart safe can also be one of the many hardware modules of the terminal device.

[0059] Alternatively, in another example, the remote monitoring alarm system 100 for the smart safe and the terminal device may be separate devices, and the remote monitoring alarm system 100 for the smart safe may be connected to the terminal device via a wired and / or wireless network and transmit interactive information in accordance with an agreed data format.

[0060] Figure 4 FIG. 1 is a flow chart of a remote monitoring alarm method for a smart safe according to an embodiment of the present application. Figure 4As shown, according to the remote monitoring and alarm method for a smart safe according to an embodiment of the present application, it includes: S110, acquiring a safe monitoring image captured by a 360-degree 3D panoramic camera with infrared night vision and vibration information of the safe at multiple predetermined time points captured by a vibration sensor; S120, extracting a safe monitoring noise reduction vector and optimizing the safe vibration information multimodal feature vector from the safe monitoring image captured by the 360-degree 3D panoramic camera with infrared night vision and the vibration information of the safe at multiple predetermined time points captured by the vibration sensor; S130, judging whether to issue a remote warning of the safe based on the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector.

[0061] Here, those skilled in the art can understand that the specific operations of each step in the remote monitoring alarm method for a smart safe have been described in the above reference. Figures 1 to 3 The remote monitoring alarm system for smart safes has been described in detail, and therefore, its repeated description will be omitted.

[0062] Below, reference Figure 5 To describe an electronic device according to an embodiment of the present application.

[0063] Figure 5 A block diagram of an electronic device according to an embodiment of the present application is illustrated.

[0064] like Figure 5 As shown, the electronic device 10 includes an input device 11, an input interface 12, a central processing unit 13, a memory 14, an output interface 15, an output device 16, and a bus 17. The input interface 12, the central processing unit 13, the memory 14, and the output interface 15 are connected to each other through the bus 17, and the input device 11 and the output device 16 are connected to the bus 17 through the input interface 14 and the output interface 15, respectively, and then connected to other components of the electronic device 10.

[0065] Specifically, the input device 11 receives input information from the outside and transmits the input information to the central processing unit 13 through the input interface 12; the central processing unit 13 processes the input information based on the computer executable instructions stored in the memory 14 to generate output information, temporarily or permanently stores the output information in the memory 14, and then transmits the output information to the output device 16 through the output interface 15; the output device 16 outputs the output information to the outside of the electronic device 10 for user use.

[0066] In one embodiment, Figure 5The electronic device 10 shown can be implemented as a network device, which may include: a memory configured to store programs; a processor configured to run the programs stored in the memory to execute any one of the remote monitoring and alarm methods for a smart safe described in the above embodiments.

[0067] According to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network, and / or installed from a removable storage medium.

[0068] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0069] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present application, but the present application is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of the present application, and these modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. A remote monitoring alarm system for a smart safe, characterized in that: include: A safe monitoring data acquisition module is used to acquire the safe monitoring images collected by a 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by a vibration sensor; A safe monitoring data extraction module, used to extract a safe monitoring noise reduction vector and optimize a safe vibration information multimodal feature vector from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor; The safe remote warning judgment module is used to judge whether to issue a safe remote warning based on the safe monitoring noise reduction vector and the optimized safe vibration information multi-modal feature vector.

2. The remote monitoring alarm system for a smart safe according to claim 1, characterized in that: The safe monitoring data extraction module includes: A safe monitoring image feature extraction unit, used for extracting features from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera to obtain the safe monitoring noise reduction vector; A vibration information frequency domain statistical feature encoding unit, used for performing frequency domain statistical feature encoding on the vibration information of the safe at a plurality of predetermined time points collected by the vibration sensor to obtain a frequency domain feature vector of the vibration of the safe; A vibration information vibration waveform feature encoding unit, used for performing vibration waveform feature encoding on the vibration information of the safe at a plurality of predetermined time points collected by the vibration sensor to obtain a waveform feature vector of the vibration signal of the safe; The safe vibration information feature construction unit is used to construct the safe vibration frequency domain feature vector and the safe vibration signal waveform feature vector to obtain the optimized safe vibration information multi-modal feature vector.

3. The remote monitoring alarm system for a smart safe according to claim 2, characterized in that: The safe monitoring image feature extraction unit comprises: The safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera is passed through a safe monitoring image noise reduction module to obtain a safe monitoring noise reduction image; The safe monitoring denoising image is passed through a safe monitoring denoising convolutional neural network model as a feature encoder to obtain the safe monitoring denoising vector.

4. The remote monitoring alarm system for a smart safe according to claim 3, characterized in that: The vibration information frequency domain statistical feature encoding unit comprises: Performing Fourier transformation on the vibration information of the safe at multiple predetermined time points collected by the vibration sensor to obtain multiple frequency domain statistical characteristic values ​​of the vibration of the safe; The plurality of safe vibration frequency domain statistical characteristic values ​​are passed through a sequence encoder of a safe vibration information joint encoder to obtain the safe vibration frequency domain characteristic vector.

5. The remote monitoring alarm system for a smart safe according to claim 4, characterized in that: The sequence encoder of the safe vibration information joint encoder is a safe vibration information context encoder including an embedding layer.

6. The remote monitoring alarm system for a smart safe according to claim 5, characterized in that: The vibration information vibration waveform feature encoding unit comprises: Extracting a waveform diagram of a vibration signal of the safe from the vibration information of the safe at a plurality of predetermined time points collected by the vibration sensor; The waveform of the safe vibration signal is passed through the image encoder of the safe vibration information combined encoder to obtain a safe vibration signal waveform characteristic diagram; The waveform feature map of the safe vibration signal is pooled to obtain a waveform feature vector of the safe vibration signal.

7. The remote monitoring alarm system for a smart safe according to claim 6, characterized in that: The safe vibration information feature construction unit comprises: Fusion of the safe vibration frequency domain feature vector and the safe vibration signal waveform feature vector to obtain a safe vibration information multimodal feature vector; Based on the safe monitoring noise reduction vector, the causal correlation enhancement based on the inverse optimization perception is performed on the multimodal feature vector of the safe vibration information to obtain the optimized multimodal feature vector of the safe vibration information.

8. The remote monitoring alarm system for a smart safe according to claim 7, characterized in that: Based on the safe monitoring noise reduction vector, the causal correlation enhancement based on the reverse optimization perception is performed on the multi-modal feature vector of the safe vibration information to obtain the optimized multi-modal feature vector of the safe vibration information, including: Calculating the position difference between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector to obtain a first viewpoint metric representation vector; Calculate the position point multiplication between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector to obtain a second viewpoint metric representation vector; Calculating the positional sum between the multimodal feature vector of the safe vibration information and the safe monitoring noise reduction vector to obtain a third viewpoint metric representation vector; Calculating the gradient magnitude of each position in the first viewpoint metric representation vector, the second viewpoint metric representation vector, and the third viewpoint metric representation vector to obtain a first viewpoint metric perception vector, a second viewpoint metric perception vector, and a third viewpoint metric perception vector; Inputting the first viewpoint metric perception vector, the second viewpoint metric perception vector and the third viewpoint metric perception vector into a backward machine learning inverse multi-view perception optimization module to obtain a reverse multi-view perception optimization representation vector; The optimized safe vibration information multimodal feature vector is obtained by calculating the position point multiplication between the inverse multi-view perception optimization representation vector and the safe vibration information multimodal feature vector.

9. The remote monitoring alarm system for a smart safe according to claim 8, characterized in that: The safe remote early warning judgment module includes: A safe monitoring information feature association unit, used to associate the safe monitoring noise reduction vector with the optimized safe vibration information multi-modal feature vector to obtain a safe remote early warning judgment classification feature vector; The safe remote warning judgment classification unit is used to classify the safe remote warning judgment classification feature vector to obtain a classification result, and the classification result is used to determine whether to issue a safe remote warning.

10. A remote monitoring alarm method for a smart safe, characterized in that: include: Acquire the monitoring images of the safe collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor; Extracting a safe monitoring noise reduction vector and optimizing a safe vibration information multimodal feature vector from the safe monitoring image collected by the 360-degree 3D panoramic infrared night vision camera and the vibration information of the safe at multiple predetermined time points collected by the vibration sensor; Based on the safe monitoring noise reduction vector and the optimized safe vibration information multimodal feature vector, it is determined whether to issue a safe remote warning.

Images