User behavior analysis method and device for burglary prevention of mobile equipment in exhibition hall

Abstract

The invention relates to a user behavior analysis method and device for burglary prevention of mobile equipment in an exhibition hall, and the method comprises the steps: carrying out the collection and preprocessing of Bluetooth signal intensity data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data, and taking the sampling rate of the preprocessed acceleration and angular velocity data as reference time; mapping the preprocessed Bluetooth signal intensity data, geomagnetic sensor data and ambient light sensor data to the same timestamp; constructing a user behavior database according to the defined user behaviors; a user behavior recognition result is obtained through the trained CNN-LSTM hybrid recognition model by means of multi-modal sensor data collected in real time; a comprehensive score is calculated by distributing weights for the user behavior recognition result and the Bluetooth signal strength distance; and judging whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, triggering an anti-theft alarm. The problems that in an existing mobile phone display anti-theft system, the false alarm rate is high, and a single sensor is prone to being interfered are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile device security, and in particular relates to a user behavior analysis method and device for preventing theft of mobile devices in exhibition halls. Background Art

[0002] With the popularity of mobile devices such as smartphones, the importance of device anti-theft and user behavior analysis technologies has become increasingly prominent. Although the existing exhibition hall mobile phone anti-theft systems provide device protection and user identity recognition functions to a certain extent, most of them rely on a single sensor or specific technology and have many limitations. The existing technologies mainly include the following categories:

[0003] (1) Wired anti-theft system: Traditional wired anti-theft systems rely on the physical connection of the device to a fixed location, such as connecting the device to a fixed point via a cable. Once the cable is disconnected or the device is removed, the system will sound an alarm. Although this method is effective in certain specific scenarios (such as store display devices), it is not applicable to mobile devices used in daily life (such as smartphones) because it severely limits the mobility and portability of the device.

[0004] (2) BLE (Bluetooth Low Energy) anti-theft system: The Bluetooth anti-theft system relies on the signal strength between the smartphone and the paired Bluetooth device (such as smart watches, headphones, etc.) to determine whether the device is within a safe range. When the Bluetooth signal weakens or disconnects, the system will trigger the anti-theft alarm. However, Bluetooth signals are easily affected by environmental interference (such as obstacles and signal reflections) and are unstable, resulting in a high false alarm rate. At the same time, a single Bluetooth anti-theft system cannot provide the precise location of the device and cannot effectively deal with complex anti-theft scenarios.

[0005] (3) UWB (Ultra-Wideband) Technology: UWB is a short-range wireless communication technology with high accuracy and low power consumption, which is particularly suitable for precise positioning and anti-theft applications. UWB is often used in high-precision indoor positioning systems to determine whether a device has been moved by measuring the distance between devices. However, the cost of UWB devices is relatively high and has not yet been widely used in smartphones. In addition, the accuracy of UWB technology is limited in open outdoor environments, making it difficult to fully apply to anti-theft needs in various scenarios.

[0006] (4) RFID (Radio Frequency Identification) Anti-theft System: RFID technology relies on wireless communication between radio frequency tags (Tags) and readers (Readers), and is usually used in logistics, commodity anti-theft and other scenarios. The advantage of RFID is that passive tag devices do not require batteries, are low-cost, and can detect the presence of devices within a short distance. However, the disadvantage of RFID technology is that the effective distance is limited, usually not more than a few meters, and once the device leaves the effective range of the reader, the anti-theft function becomes invalid. In addition, RFID cannot continuously track the real-time location of the device, making it difficult to apply to anti-theft scenarios for mobile devices.

[0007] (5) Single motion or positioning sensor technology: In addition to the above technologies, single sensors built into smartphones (such as GPS, accelerometers, gyroscopes, etc.) are also commonly used for anti-theft and user behavior analysis. However, single sensor systems have obvious limitations. For example, GPS signals are weak or cannot obtain accurate locations in indoor environments, while accelerometers or gyroscopes may generate false alarms due to normal use of the device (such as holding it in hand, putting it in a bag, etc.). In addition, relying solely on touch screens or accelerometers for user behavior analysis cannot effectively cope with complex behavior patterns and scene changes.

[0008] In summary, it is necessary to develop a user behavior analysis technology solution for anti-theft of mobile devices in exhibition halls. Summary of the invention

[0009] To this end, the present invention provides a user behavior analysis method and device for anti-theft of mobile devices in exhibition halls, so as to solve the problems of high false alarm rate and susceptibility of single sensor to interference in existing mobile phone display anti-theft systems, improve anti-theft accuracy by integrating multiple sensor data, and provide rich user behavior analysis data while ensuring safety.

[0010] In order to achieve the above object, the present invention provides the following technical solution: a user behavior analysis method for anti-theft of mobile devices in exhibition halls, comprising:

[0011] Multimodal sensor data collection: collect Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data

[0012] Multimodal sensor data preprocessing: Kalman filtering is used to smooth the collected Bluetooth signal strength data; zero-phase filter is used to preprocess the collected acceleration and angular velocity data; third-order Savitzky-Golay filter is used to preprocess the collected geomagnetic sensor data; exponentially weighted moving average filter is used to preprocess the collected ambient light sensor data;

[0013] Multimodal data time alignment fusion: using the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, mapping the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp;

[0014] User behavior identification: defining user behavior through the acceleration, angular velocity data, the geomagnetic sensor data and the ambient light sensor data;

[0015] User behavior database construction: Build a user behavior database based on defined user behaviors;

[0016] Hybrid recognition model construction and training: construct a CNN-LSTM hybrid recognition model, which includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain user behavior recognition results through the trained CNN-LSTM hybrid recognition model using multimodal sensor data collected in real time;

[0017] User behavior and Bluetooth signal strength fusion judgment: Calculate the comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determine whether the comprehensive score exceeds the preset threshold. If the comprehensive score exceeds the preset threshold, the anti-theft alarm is triggered.

[0018] As a preferred solution of the user behavior analysis method for anti-theft of mobile devices in exhibition halls, in the process of performing Kalman filtering to smooth the collected Bluetooth signal strength data, the true value of the current Bluetooth signal strength data is estimated through a prediction step and an update step;

[0019] In the prediction step, the value of the Bluetooth signal strength data at the current moment is predicted and error covariance P k|k-1 :

[0020]

[0021] P k|k-1 =A·P k-1|k-1 ·A T +Q

[0022] Where A is the state transfer matrix; Q is the process noise covariance;

[0023] In the update step, the Kalman gain K is calculated k , according to the Kalman gain K k Update the value of the Bluetooth signal strength data and error covariance Pk|k-1 :

[0024]

[0025]

[0026] P k|k =(1-K k )·P k|k-1

[0027] In the formula, z k is the currently measured Bluetooth signal strength data value, and R is the observed noise covariance; The Bluetooth signal strength data value after filtering at the current time t; P k|k The error covariance of the filtered Bluetooth signal strength data at the current time t;

[0028] The formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is:

[0029]

[0030] Where, d bt (t) is the distance between the mobile device and the display stand estimated by Bluetooth; A` is the Bluetooth signal strength data value at the reference distance); n is the path loss index of signal propagation.

[0031] As a preferred solution of the user behavior analysis method for anti-theft of mobile devices in exhibition halls, in the process of preprocessing the collected acceleration and angular velocity data with a zero-phase filter, bidirectional filtering is adopted, that is, the zero-phase filter is applied once in the forward direction and once in the backward direction to offset the phase offset;

[0032] The zero-phase filter is a Butterworth filter or a Chebyshev filter.

[0033] As a preferred solution of the user behavior analysis method for anti-theft of mobile devices in exhibition halls, the formula for preprocessing the collected geomagnetic sensor data with a third-order Savitzky-Golay filter is:

[0034]

[0035] In the formula, c k is the coefficient of the filter; K is the window size, and the appropriate window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing by the third-order Savitzky-Golay filter; m(t+k) is the geomagnetic sensor data before preprocessing by the third-order Savitzky-Golay filter;

[0036] The formula for performing exponentially weighted moving average filter preprocessing on the collected ambient light sensor data is:

[0037] L filtered (t) = γ·L(t) + (1-γ)·L filtered (t-1)

[0038] Where L(t) is the ambient light intensity at the current moment, L filtered (t) is the light intensity after filtering at the current moment; L filtered (t-1) is the light intensity after filtering at the previous moment; γ is the smoothing coefficient of the filter;

[0039] The preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp by linear interpolation or nearest neighbor interpolation method.

[0040] As a preferred solution of the user behavior analysis method for exhibition hall mobile device theft prevention, the input data of the user behavior database is multimodal sensor data within a time window; the output data of the user behavior database represents the category of user behavior;

[0041] The data X and the corresponding behavior label y in each time window together constitute a sample and are added to the data set D. The size N of the data set D depends on the acquisition time and the window sliding setting. The form of the data set D is:

[0042] D={(X (i) ,y (i) )|i=1,2,…,N}

[0043] In the formula, X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d;y (i) is the behavior label of the i-th sample.

[0044] As a preferred solution for the user behavior analysis method for anti-theft of mobile devices in exhibition halls, by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is:

[0045] S(t)=w1·f behavior (y pred )+w2·f distance (d bt (t))

[0046] In the formula, f behavior is the user behavior recognition result y pred The corresponding risk score; f distance is the Bluetooth signal strength distance dbt (t) The corresponding risk score; w1, w2 are the behavior recognition results y pred and Bluetooth signal strength distance d bt The weight of (t);

[0047] It also includes dividing the risk score into a behavior identification risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

[0048] As a preferred solution of the user behavior analysis method for preventing theft of mobile devices in exhibition halls, it also includes recording and storing user behavior data, identifying and storing each user's interaction behavior with the displayed mobile device as a behavior sequence, and the behavior sequence includes the behavior category, the timestamp of the behavior, the duration of the behavior, and the location where the behavior occurs;

[0049] It also includes analyzing the interaction frequency between users and displayed mobile phones, counting the distribution of users' behaviors in the display area, and analyzing the duration of users' behaviors; analyzing the behavior paths of users in the display area by constructing a behavior transfer matrix M;

[0050] It also includes establishing a user preference analysis model, which uses a user-mobile phone interaction matrix to predict the user's potential interests and purchase intentions.

[0051] The present invention also provides a user behavior analysis device for preventing theft of mobile devices in exhibition halls, comprising:

[0052] Multimodal sensor data acquisition module, used to collect Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data

[0053] A multimodal sensor data preprocessing module is used to perform Kalman filtering to smooth the collected Bluetooth signal strength data; to preprocess the collected acceleration and angular velocity data using a zero-phase filter; to preprocess the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; and to preprocess the collected ambient light sensor data using an exponentially weighted moving average filter.

[0054] A multimodal data time alignment fusion module, used to map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp using the sampling rate of the preprocessed acceleration and angular velocity data as a reference time;

[0055] A user behavior recognition module, used to define user behavior through the acceleration, angular velocity data, the geomagnetic sensor data and the ambient light sensor data;

[0056] A user behavior database construction module, used to construct a user behavior database according to defined user behaviors;

[0057] A hybrid recognition model construction training module is used to construct a CNN-LSTM hybrid recognition model, wherein the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; the constructed CNN-LSTM hybrid recognition model is trained according to a user behavior database, and the user behavior recognition result is obtained by using the multimodal sensor data collected in real time through the trained CNN-LSTM hybrid recognition model;

[0058] The user behavior-Bluetooth signal strength fusion judgment module is used to calculate the comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determine whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, trigger an anti-theft alarm.

[0059] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the multimodal sensor data preprocessing module, the real value of the current Bluetooth signal strength data is estimated through a prediction step and an update step;

[0060] In the prediction step, the value of the Bluetooth signal strength data at the current moment is predicted and error covariance P k|k-1 :

[0061]

[0062] P k|k-1 =A·P k-1|k-1 ·A T +Q

[0063] Where A is the state transfer matrix; Q is the process noise covariance;

[0064] In the update step, the Kalman gain K is calculated k , according to the Kalman gain K k Update the value of the Bluetooth signal strength data and error covariance P k|k-1 :

[0065]

[0066]

[0067] P k|k =(1-K k )·P k|k-1

[0068] In the formula, zk is the currently measured Bluetooth signal strength data value, and R is the observed noise covariance; The Bluetooth signal strength data value after filtering at the current time t; P k|k Error covariance of the filtered Bluetooth signal strength data at the current time t.

[0069] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the multimodal sensor data preprocessing module, the formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is:

[0070]

[0071] Where, d bt (t) is the distance between the mobile device and the display stand estimated by Bluetooth; A` is the Bluetooth signal strength data value at the reference distance); n is the path loss index of signal propagation.

[0072] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the multimodal sensor data preprocessing module, bidirectional filtering is adopted, that is, a zero-phase filter is applied once in the forward direction and once in the backward direction to offset the phase offset;

[0073] The zero-phase filter adopts a Butterworth filter or a Chebyshev filter;

[0074] In the multimodal sensor data preprocessing module, the formula for preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter is:

[0075]

[0076] In the formula, c k is the coefficient of the filter; K is the window size, and the appropriate window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing with a third-order Savitzky-Golay filter; m(t+k) is the geomagnetic sensor data before preprocessing with a third-order Savitzky-Golay filter.

[0077] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the multimodal sensor data preprocessing module, the formula for performing exponentially weighted moving average filter preprocessing on the collected ambient light sensor data is:

[0078] L filtered (t) = γ·L(t) + (1-γ)·L filtered (t-1)

[0079] Where L(t) is the ambient light intensity at the current moment, L filtered (t) is the light intensity after filtering at the current moment; L filtered (t-1) is the light intensity after filtering at the previous moment; γ is the smoothing coefficient of the filter.

[0080] As a preferred solution for the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the multimodal data time alignment fusion module, the pre-processed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp through linear interpolation or nearest neighbor interpolation method.

[0081] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the user behavior database construction module, the input data of the user behavior database is multimodal sensor data within a time window; the output data of the user behavior database represents the category of user behavior;

[0082] The data X and the corresponding behavior label y in each time window together constitute a sample and are added to the data set D. The size N of the data set D depends on the acquisition time and the window sliding setting. The form of the data set D is:

[0083]

[0084] In the formula, X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d;y (i) is the behavior label of the i-th sample.

[0085] As a preferred solution of the user behavior analysis device for exhibition hall mobile device theft prevention, in the user behavior-Bluetooth signal strength fusion judgment module:

[0086] By assigning weights to the user behavior recognition results and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is:

[0087] S(t)=w1·f behavior (y pred )+w2·f distance (d bt (t))

[0088] In the formula, f behavior is the user behavior recognition result y pred The corresponding risk score; f distance is the Bluetooth signal strength distance d bt (t) The corresponding risk score; w1, w2 are the behavior recognition results y pred and Bluetooth signal strength distance dbt The weight of (t).

[0089] As a preferred solution for the user behavior analysis device for anti-theft of mobile devices in exhibition halls, in the user behavior-Bluetooth signal strength fusion judgment module, the risk score is divided into a behavior identification risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

[0090] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in exhibition halls, it also includes a user behavior preference analysis module, which is used to record and store user behavior data, identify and store each user's interaction behavior with the display mobile device as a behavior sequence, and the behavior sequence includes the behavior category, the timestamp of the behavior, the behavior duration and the location where the behavior occurs;

[0091] The user behavior preference analysis module is also used to analyze the interaction frequency between the user and the displayed mobile phone, count the user's behavior distribution in the display area, and analyze the user's behavior duration; by constructing a behavior transfer matrix M, analyze the user's behavior path in the display area;

[0092] The user usage behavior preference analysis module is also used to establish a user preference analysis model, which uses the user-mobile phone interaction matrix to predict the user's potential interests and purchase intentions.

[0093] The present invention has the following advantages:

[0094] First, multi-modal sensor fusion makes anti-theft monitoring more accurate:

[0095] The present invention can accurately judge the relative movement and position change of mobile phones in the exhibition hall by fusing Bluetooth signal strength RSSI data, IMU acceleration, angular velocity data, geomagnetic sensor and ambient light sensor data, avoiding the problem that a single sensor is susceptible to interference and has a high false alarm rate; in particular, through a weighted judgment method, combining user behavior recognition results with Bluetooth signal strength RSSI data, the anti-theft system can intelligently analyze user behavior and the physical location of the mobile phone, significantly improving the accuracy of anti-theft judgment; the judgment strategy can be flexibly adjusted according to different behaviors and Bluetooth signal strengths, thereby reducing false alarms and missed alarms.

[0096] Second, intelligent anti-theft mechanism to adapt to different scenarios:

[0097] The present invention proposes a dynamic weight adjustment and threshold adaptive mechanism, which can adjust the weight and threshold of anti-theft judgment according to the activity level of the display area, the time period and the behavioral trend of the mobile phone; it can adaptively adjust the anti-theft sensitivity to ensure a rapid response to potential theft during peak hours and maintain a low false alarm rate during off-peak hours; through time-dependent analysis, it can identify behavioral trends in a short period of time, thereby more quickly identifying situations where the mobile phone has been stolen or abnormally moved.

[0098] Third, real-time identification of user behavior to improve user experience:

[0099] With the help of CNN-LSTM model and attention mechanism, the present invention can identify various behaviors of users in the display area in real time, including picking up the phone to check, light operation, taking pictures, rotating the phone, etc. This behavior recognition not only helps to prevent theft, but also helps stores understand users' behavior patterns and preferences. The behavior recognition results can help stores optimize the layout of the display area and improve the user experience. For example, it can identify the mobile phone models or functions that users use more frequently to help adjust the display focus.

[0100] Fourth, user preference analysis to support personalized marketing:

[0101] By recording and analyzing user behaviors, the present invention can obtain information such as user behavior frequency, stay duration, and behavior sequence, and provide data support for user preference prediction. It can accurately predict users' potential points of interest and make personalized recommendations based on user behavior data. Based on these analysis results, stores can provide customized promotional plans and product recommendations for different user groups to improve users' purchase intentions and transaction rates.

[0102] Fifth, efficient system scalability and adaptability:

[0103] The present invention can adapt to different display environments and equipment types, can adjust and optimize the model according to the needs of different display areas, has high scalability, can be easily integrated into the existing display area anti-theft system, and can achieve functional expansion through software upgrades without the need for additional hardware modifications, thereby reducing implementation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0105] Figure 1A schematic diagram showing a method flow of a user behavior analysis method for preventing theft of mobile devices in an exhibition hall provided in an embodiment of the present invention;

[0106] Figure 2 The figure is a schematic diagram of the architecture of a user behavior analysis device for preventing theft of mobile devices in exhibition halls provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0107] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0108] Example 1

[0109] See also Figure 1 Embodiment 1 of the present invention provides a user behavior analysis method for preventing theft of mobile devices in an exhibition hall, comprising the following steps:

[0110] Multimodal sensor data collection: collect Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data

[0111] Multimodal sensor data preprocessing: Kalman filtering is used to smooth the collected Bluetooth signal strength data; zero-phase filter is used to preprocess the collected acceleration and angular velocity data; third-order Savitzky-Golay filter is used to preprocess the collected geomagnetic sensor data; exponentially weighted moving average filter is used to preprocess the collected ambient light sensor data;

[0112] Multimodal data time alignment fusion: using the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, mapping the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp;

[0113] User behavior identification: defining user behavior through the acceleration, angular velocity data, the geomagnetic sensor data and the ambient light sensor data;

[0114] User behavior database construction: Build a user behavior database based on defined user behaviors;

[0115] Hybrid recognition model construction and training: construct a CNN-LSTM hybrid recognition model, which includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain user behavior recognition results through the trained CNN-LSTM hybrid recognition model using multimodal sensor data collected in real time;

[0116] User behavior and Bluetooth signal strength fusion judgment: Calculate the comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determine whether the comprehensive score exceeds the preset threshold. If the comprehensive score exceeds the preset threshold, the anti-theft alarm is triggered.

[0117] In this embodiment, in order to adapt to the different sampling rates of Bluetooth, IMU, geomagnetic and ambient light sensors, the data of each sensor is first preprocessed, and then consistent time series data is generated through time alignment and fusion for use by subsequent algorithms. The specific process is as follows:

[0118] A1. Bluetooth signal strength collection and processing

[0119] The sampling rate of the Bluetooth signal is set to 10Hz. The Bluetooth signal strength RSSI usually fluctuates greatly due to environmental interference. Therefore, Kalman filtering is used to smooth the RSSI data. The true value of the current RSSI is estimated through two steps: prediction and update.

[0120] Specifically, predict the RSSI value at the current moment and error covariance P k|k-1

[0121]

[0122] P k|k-1 =A·P k-1|k-1 ·A T +Q

[0123] Where A is the state transition matrix, and A is usually set to 1. Q is the process noise covariance, which reflects the change in RSSI. Generally, if the RSSI changes relatively smoothly, Q can be set to a smaller value, such as Q = 0.01. If the RSSI changes greatly, Q can be set to a larger value, such as Q = 0.1. Considering the characteristics of this scenario, Q is set to 0.1.

[0124] Specifically, in the update step, the Kalman gain K is first calculated k :

[0125]

[0126] Update RSSI value and error covariance P k|k-1 :

[0127]

[0128] P k|k =(1-K k )·P k|k-1

[0129] Among them, z k is the currently measured RSSI value, and R is the observation noise covariance, which reflects the uncertainty of RSSI measurement. Normally, if the measurement accuracy is high, R takes a smaller value, such as R = 0.1. If the measurement noise is large, R takes a larger value, such as R = 1. Considering the characteristics of this scenario, R is set to 0.5.

[0130] In addition, the initial state Set to the first measured RSSI value. Initial error covariance P 0|0 A larger value, such as 1, indicates a higher uncertainty about the initial state. The RSSI value after Kalman filtering can be used for subsequent distance estimation.

[0131]

[0132] Among them, d bt (t) is the distance between the mobile phone and the display stand estimated by Bluetooth (unit: meter). A` is the RSSI value at the reference distance (usually 1 meter), which is a constant corrected according to the environment; is the RSSI value after filtering at the current time t (unit: dBm); n is the path loss index of signal propagation, which is also corrected according to the environment.

[0133] B1. Inertial Measurement Unit (IMU) Data Acquisition and Processing

[0134] The sampling rate of IMU data is 100Hz, and IMU includes accelerometer data and gyroscope data. In order to filter noise and retain motion characteristics, the acceleration and angular velocity data are preprocessed using a zero-phase IIR filter. The purpose of the zero-phase IIR filter is to smooth the signal without introducing phase distortion. Conventional IIR filters usually introduce phase offsets, which can cause distortion of the temporal characteristics of sensor data. In order to avoid phase distortion, an embodiment of the present invention uses bidirectional filtering, that is, an IIR filter is applied once in the forward and backward directions to offset the phase offset.

[0135] Specifically, for the accelerometer data a(t), we have:

[0136] Forward filtering: a f (t) = IIR(a(t))

[0137] Post filtering: a filtered (t) = IIR(a f (t))

[0138] Among them, IIR is a predefined filter function, usually a low-pass filter. Similarly, for the gyroscope data ω(t), forward and backward filtering is also performed to obtain the filtered angular velocity ω filtered (t).

[0139] Specifically, IIR filter design: A low-pass filter is usually used to smooth acceleration data and remove high-frequency noise. Common IIR filter types include Butterworth filters or Chebyshev filters. In motion detection, Butterworth filters are often used because of their smooth frequency response curves. Therefore, a 2nd-order Butterworth filter is selected in this embodiment. A lower order can reduce the complexity and amount of calculation of the filter while maintaining a better smoothing effect. Since human body motion is usually in the low-frequency range (1-20Hz), the cutoff frequency can be set to 5-10Hz to retain the characteristics of human body motion while filtering out high-frequency noise. The frequency of change of gyroscope data may be slightly higher, and the cutoff frequency can usually be set to 10-20Hz to retain the characteristics of angular velocity changes.

[0140] C1. Geomagnetic sensor data acquisition and processing

[0141] The sampling rate of the geomagnetic sensor is 50Hz, and the geomagnetic sensor is used to detect the directional changes of the mobile phone. In order to ensure the smoothness and stability of the data, the geomagnetic data m(t) is processed by a third-order Savitzky-Golay filter.

[0142]

[0143] Among them, c k is the coefficient of the filter (determined by the order of the local polynomial fit and the window size). For most sensor signals, a third-order polynomial is usually selected, that is, n=3. The third-order polynomial can well preserve the trend of the signal while smoothing high-frequency noise. K is the window size, which is selected according to the characteristics of the signal and the application scenario. Comprehensive sampling rate, select a window size of 7 points.

[0144] D1. Ambient light sensor data collection and processing

[0145] The sampling rate of the ambient light sensor is 20 Hz. In order to process the instantaneous spikes or transition changes in the ambient light data, an exponentially weighted moving average (EWMA) filter is used for processing.

[0146] L filtered(t) = γ·L(t) + (1-γ)·L filtered (t-1)

[0147] Where L(t) is the ambient light intensity at the current moment. filtered (t) is the light intensity after filtering. γ is the smoothing coefficient of the filter, which is usually 0.2 to 0.5.

[0148] E1. Multimodal Data Synchronization and Time Alignment

[0149] Since the data sampling frequencies of each sensor are different, it is necessary to time align these data. The sampling rate of IMU (100Hz) is used as the reference time t k ,The data from other sensors are mapped to the same timestamp by linear interpolation or nearest neighbor interpolation method.

[0150] Among them, (a1) Bluetooth data interpolation uses nearest neighbor interpolation:

[0151] RSSI mapped (t k )=RSSI filtered (t k′ )where t k′ ≤t k <t k′+1

[0152] Where, RSSI mapped (t k ) is the target time point t k Mapped RSSI value; RSSI filtered .t k′ / is time point t k′ RSSI data that has been filtered;

[0153] (b1) Geomagnetic data interpolation, using linear interpolation:

[0154]

[0155] In the formula, m mapped (t k ) is the target time point t k The geomagnetic sensor data obtained by interpolation; m filtered (t i ) is the time point t i Above, the geomagnetic sensor data after filtering; m filtered (t i+1 ) is the time point t i+1 Above: Geomagnetic sensor data after filtering;

[0156] (c1) Ambient light data interpolation, using linear interpolation:

[0157]

[0158] Where, L mapped (t k ) is the target time point t k The light intensity data obtained by interpolation; L filtered (t j ) is the time point t j The light intensity data after filtering; L filtered (t j+1 ) is the time point t j+1 The filtered light intensity data.

[0159] In this embodiment, the specific process of user behavior identification is as follows:

[0160] A2. User Behavior Definition

[0161] By combining data from IMU (accelerometer and gyroscope), geomagnetic sensor and ambient light sensor, a variety of detailed user behaviors are defined. These behaviors include but are not limited to:

[0162] (a2) Static state:

[0163] The phone was placed on a display stand and was not touched by the user.

[0164] (b2) Light operation (viewing, sliding):

[0165] The user stands in front of the display stand and touches the screen to slide, click, or view information.

[0166] (c2) Pick up your phone and check:

[0167] The user picks up the phone to experience it, perhaps to check the phone's appearance, weight, etc., while holding the phone but not performing any other operations.

[0168] (d2) Raise your phone to take photos or videos:

[0169] When a user raises their phone to take a photo or video, the angle of the phone changes significantly.

[0170] (e2) Rotate the phone (to view the back or side of the phone):

[0171] The user rotates the phone to view its back, sides, etc.

[0172] (f2) The phone is placed in a pocket or bag:

[0173] Users put their phones in their pockets or bags, or block out light to experience it.

[0174] (g2) The phone is forcibly moved:

[0175] The phone is moved quickly or removed from the display stand, which may be unauthorized removal.

[0176] B2. Construction of user behavior database

[0177] (a3) Database structure

[0178] Input data: X is a multimodal sensor data within a time window with dimension T w ×d, where T w is the time window length, and d is the dimension of the sensor data (acceleration, angular velocity, geomagnetic field, and ambient light).

[0179] X(t k )=(a filtered (t k ),ω filtered (t k ),m filtered (t k ),L filtered (t k ))

[0180] Output label: y, indicating the category of user behavior. The behavior categories are as follows:

[0181] y∈{0,1,2,3,4,5,6}

[0182] (b3) Data collection process

[0183] Set the time window length T w ,The sampling rate is determined by the sensor (100 Hz for IMU, 50 Hz for geomagnetism, and 20 Hz for ambient light), and the collected data is processed by interpolation and filtering.

[0184] The data X in each time window and the corresponding behavior label y together constitute a sample and are added to the data set D.

[0185] The size of the dataset N depends on the acquisition duration and the window sliding setting.

[0186] Finally, the dataset D is in the form of:

[0187] D={(X (i) ,y (i) )|i=1,2,…,N}

[0188] X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d.

[0189] y (i)is the behavior label of the i-th sample.

[0190] C2. Model structure and specific parameters

[0191] In this embodiment, a CNN-LSTM hybrid model is used, combined with an attention mechanism, to fully utilize the local and global features of time series data. The following is the specific architecture and parameter settings of the model.

[0192] (a4) CNN module parameters

[0193] The convolutional layer (Conv1D) is used to extract local features from sensor input data. Where: Input dimension: T w ×d(time series length T w and feature dimension d).

[0194] Convolution kernel size: 3

[0195] Number of convolution kernels (filters): 64

[0196] stride: 1

[0197] Activation function: ReLU

[0198] Output dimension: (T w -2)×64 (the time step is reduced by 2 and the number of channels is 64)

[0199] The pooling layer is used to downsample the convolution features to reduce the amount of calculation, and the pooling window size is set to 2.

[0200] (b4) LSTM module parameters

[0201] The LSTM layer is used to capture temporal dependencies.

[0202] Input dimension: ((T w -2) / 2)×64

[0203] Number of hidden units: 128

[0204] Output dimension: 128 (the hidden state dimension of each time step output)

[0205] Dropout: 0.5 (to prevent overfitting)

[0206] (c4) Attention mechanism parameters

[0207] Attention weight calculation: For each time step t, calculate the attention weight α t :

[0208]

[0209] Among them, W a is the learnable weight matrix, h t is the hidden state of the LSTM.

[0210] Context vector calculation: Calculate the global context vector c through weighted summation:

[0211]

[0212] (d4) Fully connected layer and output layer

[0213] The fully connected layer is used to map the context vector to the behavior classification space:

[0214] Input dimension: 128 (the dimension of the context vector)

[0215] Output dimension: 64

[0216] Activation function: ReLU

[0217] The output layer uses the softmax activation function to output the probability distribution of the behavior category.

[0218] Input dimension: 64

[0219] Output dimension: 7 (number of behavior categories)

[0220] Activation function: softmax

[0221] Output: Probability distribution of behavior categories

[0222] D2. Training process and parameter setting

[0223] (a5) Loss Function

[0224] The cross entropy loss function is used to measure the difference between the model prediction and the true label:

[0225]

[0226] in, is the true label of the i-th sample (using one-hot encoding). is the model predicted probability distribution for the ith input

[0227] (b5) Optimization algorithm

[0228] Use the Adam optimizer to update the model parameters:

[0229] Initial learning rate α = 0.001; β1 = 0.9, β2 = 0.999

[0230] Among them, β1 controls the calculation of the first-order momentum (accumulation of gradients) to accelerate gradient descent. β2 controls the calculation of the second-order momentum (accumulation of squared gradients) to adjust the adaptive changes of the learning rate.

[0231] Learning rate decay: The learning rate decays by 0.1 every 10 epochs to ensure rapid convergence in the early stage and more precise search for the optimal solution in the later stage.

[0232] (c5) Training parameters

[0233] Batch size: 64

[0234] Number of training epochs: 50

[0235] Dropout: 0.5 (used in LSTM layer to prevent overfitting)

[0236] Regularization: L2 regularization coefficient is 0.001

[0237] (d5) Dataset Division

[0238] Training set: 80% of the data is used to train the model.

[0239] Validation set: 10% of the data is used to tune model hyperparameters.

[0240] Test set: 10% of the data is used to evaluate the model performance.

[0241] (e5) Data enhancement and preprocessing

[0242] The sensor data is normalized to have a mean of 0 and a variance of 1.

[0243]

[0244] Among them, μ x and σ x are the mean and standard deviation of the sensor data, respectively. In addition, the data is enhanced by adding noise, random shearing, or time reversal to improve the generalization ability of the model.

[0245] (f5) The final output behavior label

[0246] After data processing and model training, and then inputting data, the probability distribution obtained by the softmax function is:

[0247]

[0248] The final output behavior label:

[0249]

[0250] In the formula, is the predicted probability distribution of the behavior category; W o is the weight matrix of the linear classifier, with a dimension of K×D, where K is the total number of behavior categories and D is the input feature vector; c represents the feature representation vector obtained after data processing and model training; b o Represents the bias vector of the linear classifier, with dimension K.

[0251] In this embodiment, the specific process of the fusion judgment of user behavior and Bluetooth signal strength RSSI is as follows:

[0252] (a6) Weighted judgment alarm method

[0253] By identifying the result y for user behavior pred and Bluetooth signal strength RSSI distance d bt (t) assigns different weights and calculates a comprehensive score S(t) for mobile phone anti-theft judgment. If the comprehensive score S(t) exceeds a preset threshold S threshold , the burglar alarm is triggered:

[0254] S(t)=w1·f behavior (y pred )+w2·f distance (d bt (t))

[0255] Among them, f behavior is the risk score corresponding to the user behavior recognition result. distance is the risk score corresponding to the Bluetooth RSSI distance. w1 and w2 are the weights of the behavior recognition result and RSSI distance, reflecting their respective impact on the comprehensive score.

[0256] (b6) Mapping of risk scores

[0257] In order to enhance the flexibility of the system, risk scores are divided more finely according to different behavior categories and RSSI distances:

[0258] Behavior Identification Risk Score f behavior (y pred ):

[0259] High-risk behavior: behavior category y pred =6 (the phone was forcibly moved or stolen), the score is 1.0.

[0260] Behavior category y pred = 5 (the phone is put into a pocket or bag), the score is 0.7. This behavior has a certain risk, but it is not necessarily theft.

[0261] Low-risk behavior: behavior category ypred {0,1,2,3,,4} (stationary, light operation, picking up to check, rotating the phone), with a score of 0.5.

[0262] RSSI distance risk score f distance (d bt (t)):

[0263] High risk distance: When RSSI distance d bt (t)>d threshold , the phone is too far from the display stand and the score is 1.0.

[0264] Medium risk distance: When the RSSI distance is close to the threshold d bt (t)∈[d threshold -δ,d threshold ], indicating that the phone may be taken away from the display stand, with a score of 0.7.

[0265] Low risk distance: When RSSI distance d bt (t)≤d threshold -δ, indicating that the phone is still near the display stand, with a score of 0.3.

[0266] (c6) Dynamic weight adjustment mechanism

[0267] In order to enhance the intelligence of the system, a dynamic adjustment mechanism is adopted to change the weights w1 and w2 of the behavior recognition results and RSSI distance based on the following factors:

[0268] Time Factor:

[0269] During peak hours of mobile phone display (e.g., busy hours in a store), the weight w1 of user behavior should be increased because user behavior in the display area is more frequent and diverse. During off-peak hours, the weight w2 of Bluetooth RSSI should be higher because there is less activity in the display area and the risk of mobile phones being forcibly moved or stolen is higher.

[0270] Mobile phone trends:

[0271] If the phone continuously detects high-risk behaviors in a short period of time, the system should increase the weight w1 of behavior recognition, because the risk of user behavior is higher in this case. Conversely, if the phone maintains low-risk behavior (such as stillness or light operation) in a short period of time, the weight w2 of RSSI should be increased.

[0272] In this embodiment, the specific process of user usage behavior and preference analysis is as follows:

[0273] The analysis of user behavior and preferences in the mobile phone display area can help stores gain a deeper understanding of customers’ interaction methods, interests, and potential purchase intentions. This information can not only optimize the layout of mobile phone displays, but also provide data support for personalized recommendations and marketing campaign design. pred And sensor data, combined with multi-dimensional data analysis methods, to conduct in-depth analysis of user usage behavior and preferences.

[0274] (a7) Recording and storage of user behavior data

[0275] In the display area, every user interaction will generate a series of sensor data and behavior recognition results. In order to analyze user behavior and preferences, the system needs to record and store this data.

[0276] Behavior record: Each user's interaction with the display phone will be identified and stored as a behavior sequence, including: behavior category, timestamp of the behavior, duration of the behavior, and location of the behavior.

[0277] (b7) User Behavior Analysis Dimension

[0278] Analyze the frequency of users' interactions with displayed mobile phones to help merchants understand which users are more interested in the mobile phone display area and which behaviors are most frequent. The specific dimensions are as follows:

[0279] Behavior distribution: Statistics on the distribution of various user behaviors in the display area, such as the proportion of static, light operation, photo taking, rotating the phone, etc.

[0280]

[0281] Where N(y) is the number of occurrences of behavior category y. total is the total number of all behaviors.

[0282] Behavior duration analysis: Behavior duration can reflect the user's interest in certain interactions.

[0283] Generally speaking, the longer users stay, the more interested they are in a particular phone or function.

[0284]

[0285] in, is the duration of behavior y.

[0286] Behavior sequence and pattern analysis: The sequence and pattern of user behavior can reveal the typical interaction process of users. For example, a user may first pick up the phone to check the appearance, then perform some light operations, and finally raise the phone to take a photo. By constructing the behavior transfer matrix M, the typical behavior path of users in the display area is analyzed:

[0287]

[0288] Among them, N(y i →y i ) is the user's behavior y i Transfer to y i The number of times. N(y i ) Behavior i The total number of occurrences.

[0289] Through the behavior transfer matrix, we can identify the typical interaction patterns of users. For example, users may be more inclined to raise their phones to take photos immediately after a light operation, which means they may have a higher interest in the phone's operating experience and camera functions.

[0290] (c7) User Preference Analysis Model

[0291] Based on the user behavior analysis, a user preference analysis model is further established to predict the user's potential interests and purchase intentions. The embodiment of the present invention uses collaborative filtering technology to mine user preferences. The collaborative filtering model can predict the user's potential interest in other mobile phone models or functions based on the user's historical behavior data.

[0292] Specifically, we construct a user-mobile phone interaction matrix R, where the element R ui Indicates the interaction frequency or dwell time between user u and mobile phone i:

[0293]

[0294] in, is the duration of the interaction.

[0295] Interest prediction: Based on the user-mobile phone interaction matrix, collaborative filtering algorithms (such as matrix decomposition) are used to predict the user's potential interest in other mobile phones. It can recommend mobile phone models that users may be interested in.

[0296] In summary, the present invention collects Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data; performs Kalman filtering to smooth the collected Bluetooth signal strength data; pre-processes the collected acceleration and angular velocity data using a zero-phase filter; pre-processes the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; pre-processes the collected ambient light sensor data using an exponentially weighted moving average filter; uses the sampling rate of the pre-processed acceleration and angular velocity data as the reference time, and maps the pre-processed Bluetooth signal strength data, the geomagnetic sensor data and the ambient light sensor data to the same timestamp; uses the acceleration and angular velocity data, The geomagnetic sensor data and the ambient light sensor data define user behavior; a user behavior database is constructed according to the defined user behavior; a CNN-LSTM hybrid recognition model is constructed, and the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer and an output layer; the constructed CNN-LSTM hybrid recognition model is trained according to the user behavior database, and the user behavior recognition result is obtained by the trained CNN-LSTM hybrid recognition model using the multimodal sensor data collected in real time; a comprehensive score is calculated by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; and it is determined whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, an anti-theft alarm is triggered. The present invention can accurately judge the relative movement and position change of the mobile phone in the exhibition hall by integrating the Bluetooth signal strength RSSI data, the acceleration and angular velocity data of the IMU, the data of the geomagnetic sensor and the ambient light sensor, avoiding the problem that a single sensor is susceptible to interference and has a high false alarm rate; in particular, through the weighted judgment method, combining the user behavior recognition results with the Bluetooth signal strength RSSI data, the anti-theft system can intelligently analyze the user behavior and the physical location of the mobile phone, significantly improving the accuracy of the anti-theft judgment; it can flexibly adjust the judgment strategy according to different behaviors and Bluetooth signal strengths, thereby reducing false alarms and missed alarms. The present invention proposes a dynamic weight adjustment and threshold adaptive mechanism, which can adjust the weight and threshold of the anti-theft judgment according to the activity level, time period and behavioral trend of the display area; it can adaptively adjust the anti-theft sensitivity to ensure rapid response to potential theft behaviors during peak hours, while maintaining a low false alarm rate during off-peak hours; through time-dependent analysis, it can identify behavioral trends in a short period of time, thereby more quickly identifying the situation where the mobile phone is stolen or moved abnormally. With the help of CNN-LSTM model and attention mechanism, the present invention can identify various behaviors of users in the display area in real time, including picking up the phone to check, light operation, taking pictures, rotating the phone, etc. This behavior recognition not only helps prevent theft, but also helps stores understand users' behavior patterns and preferences.The behavior recognition results can help stores optimize the display area layout and improve user experience. For example, it can identify the mobile phone models or functions that users use more frequently to help adjust the display focus. By recording and analyzing user behavior, the present invention can obtain information such as user behavior frequency, stay duration, behavior sequence, etc., and provide data support for user preference prediction. It can accurately predict the user's potential points of interest and make personalized recommendations based on user behavior data. Based on these analysis results, stores can provide customized promotional plans and product recommendations for different user groups to improve users' purchasing intentions and transaction rates. The present invention can adapt to different display environments and equipment types, can adjust and optimize the model according to different display area requirements, has a high degree of scalability, can be easily integrated into the existing display area anti-theft system, and can achieve functional expansion through software upgrades without the need for additional hardware modifications, reducing implementation costs.

[0297] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0298] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0299] Example 2

[0300] See also Figure 2 Embodiment 2 of the present invention further provides a user behavior analysis device for preventing theft of mobile devices in exhibition halls, comprising:

[0301] Multimodal sensor data acquisition module 001, used to collect Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data

[0302] The multimodal sensor data preprocessing module 002 is used to perform Kalman filtering to smooth the collected Bluetooth signal strength data; to preprocess the collected acceleration and angular velocity data using a zero-phase filter; to preprocess the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; and to preprocess the collected ambient light sensor data using an exponentially weighted moving average filter.

[0303] The multimodal data time alignment fusion module 003 is used to map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp using the sampling rate of the preprocessed acceleration and angular velocity data as a reference time;

[0304] A user behavior recognition module 004, used to define user behavior through the acceleration, angular velocity data, the geomagnetic sensor data and the ambient light sensor data;

[0305] A user behavior database construction module 005, used to construct a user behavior database according to defined user behaviors;

[0306] The hybrid recognition model construction training module 006 is used to construct a CNN-LSTM hybrid recognition model, wherein the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; the constructed CNN-LSTM hybrid recognition model is trained according to the user behavior database, and the user behavior recognition result is obtained by the trained CNN-LSTM hybrid recognition model using the multimodal sensor data collected in real time;

[0307] The user behavior-Bluetooth signal strength fusion judgment module 007 is used to calculate a comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determine whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, trigger an anti-theft alarm.

[0308] In this embodiment, in the multimodal sensor data preprocessing module 002, the real value of the current Bluetooth signal strength data is estimated through a prediction step and an update step;

[0309] In the prediction step, the value of the Bluetooth signal strength data at the current moment is predicted and error covariance P k|k-1 :

[0310]

[0311] P k|k-1 =A·P k-1|k-1 ·AT +Q

[0312] Where A is the state transfer matrix; Q is the process noise covariance;

[0313] In the update step, the Kalman gain K is calculated k , according to the Kalman gain K k Update the value of the Bluetooth signal strength data and error covariance P k|k-1 :

[0314]

[0315]

[0316] P k|k =(1-K k )·P k|k-1

[0317] In the formula, z k is the currently measured Bluetooth signal strength data value, and R is the observation noise covariance.

[0318] In this embodiment, in the multimodal sensor data preprocessing module 002, the formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is:

[0319]

[0320] Where, d bt (t) is the distance between the mobile device and the display stand estimated by Bluetooth; A` is the Bluetooth signal strength data value at the reference distance); n is the path loss index of signal propagation.

[0321] In this embodiment, in the multimodal sensor data preprocessing module 002, bidirectional filtering is adopted, that is, a zero-phase filter is applied once in the forward direction and once in the backward direction to offset the phase offset;

[0322] The zero-phase filter adopts a Butterworth filter or a Chebyshev filter;

[0323] In the multimodal sensor data preprocessing module 002, the formula for preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter is:

[0324]

[0325] In the formula, c k is the coefficient of the filter; K is the window size, and the appropriate window size is selected according to the characteristics of the signal and the application scenario; m filtered(t) is the geomagnetic sensor data after preprocessing with a third-order Savitzky-Golay filter; m(t+k) is the geomagnetic sensor data before preprocessing with a third-order Savitzky-Golay filter.

[0326] In this embodiment, in the multimodal sensor data preprocessing module 002, the formula for performing exponentially weighted moving average filter preprocessing on the collected ambient light sensor data is:

[0327] L filtered (t) = γ·L(t) + (1-γ)·L filtered (t-1)

[0328] Where L(t) is the ambient light intensity at the current moment, L filtered (t) is the light intensity after filtering at the current moment; L filtered (t-1) is the light intensity after filtering at the previous moment; γ is the smoothing coefficient of the filter.

[0329] In this embodiment, in the multimodal data time alignment fusion module 003, the pre-processed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp through linear interpolation or nearest neighbor interpolation method.

[0330] In this embodiment, in the user behavior database construction module 005, the input data of the user behavior database is multimodal sensor data within a time window; the output data of the user behavior database represents the category of user behavior;

[0331] The data X and the corresponding behavior label y in each time window together constitute a sample and are added to the data set D. The size N of the data set D depends on the acquisition time and the window sliding setting. The form of the data set D is:

[0332]

[0333] In the formula, X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d;y (i) is the behavior label of the i-th sample.

[0334] In this embodiment, in the user behavior-Bluetooth signal strength fusion judgment module 007:

[0335] By assigning weights to the user behavior recognition results and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is:

[0336] S(t)=w1·f behavior (y pred)+w2·f distance (d bt (t))

[0337] In the formula, f behavior is the user behavior recognition result y pred The corresponding risk score; f distance is the Bluetooth signal strength distance d bt (t) The corresponding risk score; w1, w2 are the behavior recognition results y pred and Bluetooth signal strength distance d bt The weight of (t).

[0338] In this embodiment, in the user behavior-Bluetooth signal strength fusion judgment module 007, the risk score is divided into a behavior identification risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

[0339] In this embodiment, it also includes a user behavior preference analysis module 008, which is used to record and store user behavior data, identify and store each user's interaction behavior with the display mobile device as a behavior sequence, and the behavior sequence includes a behavior category, a timestamp of the behavior, a behavior duration, and a location where the behavior occurs;

[0340] The user usage behavior preference analysis module 008 is also used to analyze the interaction frequency between the user and the displayed mobile phone, count the user's behavior distribution in the display area, and analyze the user's behavior duration; by constructing a behavior transfer matrix M, analyze the user's behavior path in the display area;

[0341] The user usage behavior preference analysis module 008 is also used to establish a user preference analysis model, which uses the user-mobile phone interaction matrix to predict the user's potential interests and purchase intentions.

[0342] It should be noted that the information interaction, execution process and other contents between the above-mentioned device modules are based on the same concept as the method embodiment in Example 1 of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the present application, and will not be repeated here.

[0343] Example 3

[0344] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, in which a program code for a user behavior analysis method for preventing theft of mobile devices in exhibition halls is stored. The program code includes instructions for executing embodiment 1 or any possible implementation thereof, a user behavior analysis method for preventing theft of mobile devices in exhibition halls.

[0345] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0346] Example 4

[0347] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor;

[0348] The processor and the memory communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute a user behavior analysis method for preventing theft of mobile devices in an exhibition hall in accordance with embodiment 1 or any possible implementation thereof.

[0349] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor implemented by reading software codes stored in a memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0350] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0351] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented by a program code executable by a computing system, so that they can be stored in a storage system and executed by the computing system, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0352] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A user behavior analysis method for anti-theft of mobile devices in exhibition halls, characterized in that: include: Multimodal sensor data collection: collect Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data Multimodal sensor data preprocessing: Kalman filtering is performed on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; the collected acceleration and angular velocity data are preprocessed using a zero-phase filter; the collected geomagnetic sensor data is preprocessed using a third-order Savitzky-Golay filter; Performing exponentially weighted moving average filter preprocessing on the collected ambient light sensor data; Multimodal data time alignment fusion: using the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, mapping the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp; User behavior identification: defining user behavior through the acceleration, angular velocity data, the geomagnetic sensor data and the ambient light sensor data; User behavior database construction: Build a user behavior database based on defined user behaviors; Hybrid recognition model construction and training: construct a CNN-LSTM hybrid recognition model, which includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain user behavior recognition results through the trained CNN-LSTM hybrid recognition model using multimodal sensor data collected in real time; User behavior and Bluetooth signal strength fusion judgment: Calculate the comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determine whether the comprehensive score exceeds the preset threshold. If the comprehensive score exceeds the preset threshold, the anti-theft alarm is triggered.

2. A user behavior analysis method for anti-theft of mobile devices in exhibition halls according to claim 1, characterized in that: In the process of performing Kalman filtering to smooth the collected Bluetooth signal strength data, a true value of the current Bluetooth signal strength data is estimated through a prediction step and an update step; In the prediction step, the value of the Bluetooth signal strength data at the current moment is predicted and error covariance P k|k-1 : P k|k-1 =A·P k-1|k-1 ·A T +Q Where A is the state transfer matrix; Q is the process noise covariance; In the update step, the Kalman gain K is calculated k , according to the Kalman gain K k Update the value of the Bluetooth signal strength data and error covariance P k|k-1 : P k|k =(1-K k )·P k|k-1 In the formula, z k is the currently measured Bluetooth signal strength data value, and R is the observed noise covariance; The Bluetooth signal strength data value after filtering at the current time t; P k|k The error covariance of the filtered Bluetooth signal strength data at the current time t; The formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is: Where, d bt (t) is the distance between the mobile device and the display stand estimated by Bluetooth; A` is the Bluetooth signal strength data value at the reference distance); n is the path loss index of signal propagation.

3. The user behavior analysis method for anti-theft of mobile devices in exhibition halls according to claim 1 is characterized in that: In the process of preprocessing the collected acceleration and angular velocity data using a zero-phase filter, bidirectional filtering is used, that is, the zero-phase filter is applied once in the forward direction and once in the backward direction to offset the phase offset; The zero-phase filter is a Butterworth filter or a Chebyshev filter.

4. The user behavior analysis method for anti-theft of mobile devices in exhibition halls according to claim 1 is characterized in that: The formula for preprocessing the collected geomagnetic sensor data with a third-order Savitzky-Golay filter is: In the formula, c k is the coefficient of the filter; K is the window size, and the appropriate window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing by the third-order Savitzky-Golay filter; m(t+k) is the geomagnetic sensor data before preprocessing by the third-order Savitzky-Golay filter; The formula for performing exponentially weighted moving average filter preprocessing on the collected ambient light sensor data is: L filtered (t)=γ·L(t)+(1-γ)·L filtered (t-1) Where L(t) is the ambient light intensity at the current moment, L filtered (t) is the light intensity after filtering at the current moment; L filtered (t-1) is the light intensity after filtering at the previous moment; γ is the smoothing coefficient of the filter; The preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp by linear interpolation or nearest neighbor interpolation method.

5. The user behavior analysis method for anti-theft of mobile devices in exhibition halls according to claim 1 is characterized in that: The input data of the user behavior database is multimodal sensor data within a time window; the output data of the user behavior database represents the category of user behavior; The data X in each time window and the corresponding behavior label y together constitute a sample and are added to the dataset. Medium; Dataset The size N depends on the acquisition time and window sliding settings; the data set The form is: Where, X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d;y (i) is the behavior label of the i-th sample.

6. The user behavior analysis method for anti-theft of mobile devices in exhibition halls according to claim 1 is characterized in that: By assigning weights to the user behavior recognition results and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is: S(t)=w1·f behavior (y pred )+w2·f distance (d bt (t)) In the formula, f behavior is the user behavior recognition result y pred The corresponding risk score; f distance is the Bluetooth signal strength distance d bt (t) The corresponding risk score; w1, w2 are the behavior recognition results y pred and Bluetooth signal strength distance d bt The weight of (t); It also includes dividing the risk score into a behavior identification risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

7. The user behavior analysis method for anti-theft of mobile devices in exhibition halls according to claim 1 is characterized in that: It also includes recording and storing user behavior data, identifying and storing each user's interaction behavior with the display mobile device as a behavior sequence, the behavior sequence including the behavior category, the timestamp of the behavior, the duration of the behavior, and the location where the behavior occurs; It also includes analyzing the frequency of interaction between users and the displayed mobile phones, counting the distribution of users' behaviors in the display area, and analyzing the duration of users' behaviors; By constructing the behavior transfer matrix M, the user's behavior path in the display area is analyzed; It also includes establishing a user preference analysis model, which uses a user-mobile phone interaction matrix to predict the user's potential interests and purchase intentions.

8. A user behavior analysis device for preventing theft of mobile devices in exhibition halls, characterized in that: include: Multimodal sensor data acquisition module, used to collect Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data and ambient light sensor data A multimodal sensor data preprocessing module is used to perform Kalman filtering to smooth the collected Bluetooth signal strength data; to preprocess the collected acceleration and angular velocity data using a zero-phase filter; to preprocess the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; and to preprocess the collected ambient light sensor data using an exponentially weighted moving average filter. A multimodal data time alignment fusion module, used to map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp using the sampling rate of the preprocessed acceleration and angular velocity data as a reference time; A user behavior recognition module, used to define user behavior through the acceleration, angular velocity data, the geomagnetic sensor data and the ambient light sensor data; A user behavior database construction module, used to construct a user behavior database according to defined user behaviors; A hybrid recognition model construction training module is used to construct a CNN-LSTM hybrid recognition model, wherein the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; the constructed CNN-LSTM hybrid recognition model is trained according to a user behavior database, and the user behavior recognition result is obtained by using the multimodal sensor data collected in real time through the trained CNN-LSTM hybrid recognition model; The user behavior-Bluetooth signal strength fusion judgment module is used to calculate the comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determine whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, trigger an anti-theft alarm.

9. The user behavior analysis device for preventing theft of mobile devices in exhibition halls according to claim 8, characterized in that: In the multimodal sensor data preprocessing module, the true value of the current Bluetooth signal strength data is estimated through a prediction step and an update step; In the prediction step, the value of the Bluetooth signal strength data at the current moment is predicted and error covariance P k|k-1 : P k|k-1 =A·P k-1|k-1 ·A T +Q Where A is the state transfer matrix; Q is the process noise covariance; In the update step, the Kalman gain K is calculated k , according to the Kalman gain K k Update the value of the Bluetooth signal strength data and error covariance P k|k-1 : P k|k =(1-K k )·P k|k-1 In the formula, z k is the currently measured Bluetooth signal strength data value, and R is the observed noise covariance; The Bluetooth signal strength data value after filtering at the current time t; P k|k The error covariance of the filtered Bluetooth signal strength data at the current time t; In the multimodal sensor data preprocessing module, the formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is: Where, d bt (t) is the distance between the mobile device and the display stand estimated by Bluetooth; A` is the Bluetooth signal strength data value at the reference distance); n is the path loss index of signal propagation; In the multimodal sensor data preprocessing module, bidirectional filtering is adopted, that is, a zero-phase filter is applied once in the forward direction and once in the backward direction to offset the phase offset; The zero-phase filter adopts a Butterworth filter or a Chebyshev filter; In the multimodal sensor data preprocessing module, the formula for preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter is: In the formula, c k is the coefficient of the filter; K is the window size, and the appropriate window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing by the third-order Savitzky-Golay filter; m(t+k) is the geomagnetic sensor data before preprocessing by the third-order Savitzky-Golay filter; In the multimodal sensor data preprocessing module, the formula for performing exponentially weighted moving average filter preprocessing on the collected ambient light sensor data is: L filtered (t)=γ·L(t)+(1-γ)·L filtered (t-1) Where L(t) is the ambient light intensity at the current moment, L filtered (t) is the light intensity after filtering at the current moment; L filtered (t-1) is the light intensity after filtering at the previous moment; γ is the smoothing coefficient of the filter; In the multimodal data time alignment fusion module, the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp by linear interpolation or nearest neighbor interpolation method; In the user behavior database construction module, the input data of the user behavior database is multimodal sensor data within a time window; The output data of the user behavior database represents the category of user behavior; The data X in each time window and the corresponding behavior label y together constitute a sample and are added to the dataset. Medium; Dataset The size N depends on the acquisition time and window sliding settings; the data set The form is: Where, X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d;y (i) is the behavior label of the i-th sample; In the user behavior-Bluetooth signal strength fusion judgment module: By assigning weights to the user behavior recognition results and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is: S(t)=w1·f behavior (y pred )+w2·f distance (d bt (t)) In the formula, f behavior is the user behavior recognition result y pred The corresponding risk score; f distance is the Bluetooth signal strength distance d bt (t) The corresponding risk score; w1, w2 are the behavior recognition results y pred and Bluetooth signal strength distance d bt The weight of (t); In the user behavior-Bluetooth signal strength fusion judgment module, the risk score is divided into a behavior identification risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

10. The user behavior analysis device for preventing theft of mobile devices in exhibition halls according to claim 8, characterized in that: It also includes a user behavior preference analysis module, which is used to record and store user behavior data, identify and store each user's interaction behavior with the display mobile device as a behavior sequence, and the behavior sequence includes the behavior category, the timestamp of the behavior, the behavior duration and the location where the behavior occurs; The user behavior preference analysis module is also used to analyze the interaction frequency between the user and the displayed mobile phone, count the user's behavior distribution in the display area, and analyze the user's behavior duration; by constructing a behavior transfer matrix M, analyze the user's behavior path in the display area; The user usage behavior preference analysis module is also used to establish a user preference analysis model, which uses the user-mobile phone interaction matrix to predict the user's potential interests and purchase intentions.

Images