A method and device for user behavior analysis for anti-theft of mobile devices in exhibition halls

Abstract

A user behavior analysis method and device for anti-theft of mobile devices in exhibition halls. This method collects and preprocesses Bluetooth signal strength data, acceleration and angular velocity data, geomagnetic sensor data, and ambient light sensor data. Taking the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, it maps the preprocessed Bluetooth signal strength data, geomagnetic sensor data, and ambient light sensor data to the same time stamp; constructs a user behavior database according to the defined user behaviors; obtains the user behavior recognition result by using the trained CNN-LSTM hybrid recognition model with real-time collected multi-modal sensor data; calculates the comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; determines whether the comprehensive score exceeds the preset threshold, and if the comprehensive score exceeds the preset threshold, triggers an anti-theft alarm. The present invention solves the problems of high false alarm rate and easy interference of single sensors in the existing mobile phone display anti-theft system.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile device security, and particularly relates to a user behavior analysis method and device for anti-theft of mobile devices in exhibition halls. Background Art

[0002] With the popularization of mobile devices such as smart phones, the importance of device anti-theft and user behavior analysis technologies has become increasingly prominent. Although existing anti-theft systems for mobile phones in exhibition halls provide device protection and user identity recognition functions to a certain extent, most rely on a single sensor or specific technology and have many limitations. The existing technologies mainly include the following categories:

[0003] (1) Wired anti-theft system: The traditional wired anti-theft system relies on the physical connection of the device to a fixed position. For example, the device is connected to a fixed point through a cable. Once the cable is disconnected or the device is removed, the system will issue an alarm. Although this method is effective in some specific scenarios (such as store display devices), it is not applicable to mobile devices in daily use (such as smart phones) because it severely limits the mobility and portability of the device.

[0004] (2) BLE (Bluetooth Low Energy) anti-theft system: The Bluetooth anti-theft system relies on the signal strength between the smart phone and paired Bluetooth devices (such as smart watches, earphones, etc.) to determine whether the device is within the safe range. When the Bluetooth signal weakens or disconnects, the system will trigger an anti-theft alarm. However, the Bluetooth signal is easily interfered by the environment (such as obstacles, signal reflection) and is unstable, resulting in a high false alarm rate. At the same time, a single Bluetooth anti-theft system cannot provide the precise location of the device and cannot effectively cope with complex anti-theft scenarios.

[0005] (3) UWB (Ultra-Wideband) technology: UWB is a short-range wireless communication technology with high precision and low power consumption, especially suitable for precise positioning and anti-theft applications. UWB is often used in high-precision indoor positioning systems to determine whether a device has been moved by measuring the distance between devices. However, the cost of UWB devices is relatively high and they have not been widely used in smart phones. In addition, the accuracy of UWB technology is limited in an open outdoor environment and it is difficult to fully meet the anti-theft requirements of various scenarios.

[0006] (4) RFID (Radio Frequency Identification) anti-theft system: The RFID technology relies on the wireless communication between radio frequency tags (Tags) and readers, and is usually used in scenarios such as logistics and commodity anti-theft. The advantages of RFID are that the passive tag devices do not require batteries, have low costs, and can detect the presence of devices within a short distance. However, the disadvantages of RFID technology are that the effective distance is limited, usually not exceeding several meters, and once the device leaves the effective range of the reader, the anti-theft function fails. In addition, RFID cannot continuously track the real-time position of the device, making it difficult to be applied to the anti-theft scenarios of mobile devices.

[0007] (5) Single motion or positioning sensor technology: In addition to the above technologies, single sensors (such as GPS, accelerometers, gyroscopes, etc.) built into smartphones are usually also used for anti-theft and user behavior analysis. However, single sensor systems have obvious limitations. For example, GPS has weak signals or cannot obtain accurate positions in indoor environments, while accelerometers or gyroscopes may generate false alarms due to the normal use of the device (such as being held in the hand, placed in a bag, etc.). In addition, relying solely on the touch screen or accelerometer for user behavior analysis cannot effectively handle complex behavior patterns and scenario changes.

[0008] In summary, it is necessary to develop a technical solution for user behavior analysis for anti-theft of mobile devices in exhibition halls. Summary of the Invention

[0009] Therefore, the present invention provides a method and device for user behavior analysis for anti-theft of mobile devices in exhibition halls to solve the problems of high false alarm rates and susceptibility to interference of single sensors in existing mobile phone display anti-theft systems, improve anti-theft accuracy by fusing multiple sensor data, and provide rich user behavior analysis data while ensuring security.

[0010] To achieve the above object, the present invention provides the following technical solution: A method for user behavior analysis for anti-theft of mobile devices in exhibition halls, including:

[0011] Multi-modal sensor data acquisition: Collect Bluetooth signal strength data, acceleration and angular velocity data, geomagnetic sensor data, and ambient light sensor data.

[0012] Multi-modal sensor data preprocessing: Perform Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; perform preprocessing on the collected acceleration and angular velocity data using a zero-phase filter; perform preprocessing on the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; perform preprocessing on the collected ambient light sensor data using an exponentially weighted moving average filter.

[0013] Multi-modal data temporal alignment and fusion: Using the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp;

[0014] User behavior recognition: Define user behavior through the acceleration, angular velocity data, the geomagnetic sensor data, and the ambient light sensor data;

[0015] User behavior database construction: Construct a user behavior database according to the defined user behavior;

[0016] Hybrid recognition model construction and training: Construct a CNN-LSTM hybrid recognition model, which includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; Train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain the user behavior recognition result by using the real-time collected multi-modal sensor data through the trained CNN-LSTM hybrid recognition model;

[0017] User behavior and Bluetooth signal strength fusion judgment: Calculate the comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; Judge whether the comprehensive score exceeds the preset threshold, and if the comprehensive score exceeds the preset threshold, trigger the anti-theft alarm.

[0018] As a preferred solution for the user behavior analysis method for anti-theft of exhibition hall mobile devices, during the process of performing Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data, estimate the true value of the current Bluetooth signal strength data through the prediction step and the update step;

[0019] In the prediction step, predict the value of the Bluetooth signal strength data at the current moment and the error covariance P k|k-1 :

[0020]

[0021] P k|k-1 = A · P k-1|k-1 · A T + Q

[0022] In the formula, A is the state transition matrix; Q is the process noise covariance;

[0023] In the update step, calculate the Kalman gain K k , and update the value of the Bluetooth signal strength data according to the Kalman gain K k and the error covariance P and the error covariance Pk|k-1 :

[0024]

[0025]

[0026] P k|k = (1 - K k )·P k|k-1

[0027] Where z k is the currently measured Bluetooth signal strength data value, and R is the observation noise covariance; The Bluetooth signal strength data value after filtering at the current time t; P k|k The error covariance of the Bluetooth signal strength data after filtering at the current time t;

[0028] The formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is:

[0029]

[0030] Where d bt (t) is the distance between the mobile device estimated by Bluetooth and the display stand; A` is the Bluetooth signal strength data value at the reference distance); n is the path loss exponent of signal propagation.

[0031] As an optimal solution for the user behavior analysis method for anti-theft of mobile devices in the exhibition hall, during the preprocessing of the collected acceleration and angular velocity data using a zero-phase filter, two-way filtering is adopted, that is, the zero-phase filter is applied once forward and once backward to cancel the phase shift;

[0032] The zero-phase filter uses a Butterworth filter or a Chebyshev filter.

[0033] As an optimal solution for the user behavior analysis method for anti-theft of mobile devices in the exhibition hall, the formula for preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter is:

[0034]

[0035] Where c k is the coefficient of the filter; K is the window size, and a suitable window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing using a third-order Savitzky-Golay filter; m(t + k) is the geomagnetic sensor data before preprocessing using a third-order Savitzky-Golay filter;

[0036] The formula for preprocessing the collected ambient light sensor data with an exponentially weighted moving average filter is as follows:

[0037] L filtered (t) = γ·L(t) + (1 - γ)·L filtered (t - 1)

[0038] In the formula, L(t) is the ambient light intensity at the current moment, and L filtered (t) is the filtered light intensity at the current moment; L filtered (t - 1) is the filtered light intensity at the previous moment; γ is the smoothing coefficient of the filter;

[0039] Map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp through linear interpolation or nearest neighbor interpolation method.

[0040] As a preferred solution for the user behavior analysis method for anti-theft of mobile devices in the exhibition hall, the input data of the user behavior database is multi-modal sensor data within a time window; the output data of the user behavior database represents the category of user behavior;

[0041] The data X within each time window and the corresponding behavior label y together form a sample and are added to the dataset D; the size N of the dataset D depends on the acquisition duration and the window sliding setting; the form of the dataset D is:

[0042] D = {(X (i) , y (i) ) ∣ i = 1, 2, …, N}

[0043] In the formula, X (i) is the multi-modal sensor data of the i-th sample, with a dimension of T w ×d; y (i) is the behavior label of the i-th sample.

[0044] As a preferred solution for the user behavior analysis method for anti-theft of mobile devices in the exhibition hall, the formula for calculating the comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance is:

[0045] S(t) = w1·f behavior (y pred ) + w2·f distance (d bt (t))

[0046] In the formula, f behavior is the risk score corresponding to the user behavior recognition result y pred ; f distance is the Bluetooth signal strength distance dbt (t) corresponding risk score; w1 and w2 are the weights of the behavior recognition result y pred and the Bluetooth signal strength distance d bt (t);

[0047] It also includes dividing the risk score into a behavior recognition risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

[0048] As an optimal solution for the user behavior analysis method for anti-theft of mobile devices in the exhibition hall, it also includes recording and storing user behavior data, identifying and storing the interaction behavior between each user and the exhibition mobile device as a behavior sequence, and the behavior sequence includes the behavior category, the timestamp of the behavior occurrence, the behavior duration, and the location where the behavior occurs;

[0049] It also includes analyzing the interaction frequency between the user and the exhibition mobile phone, counting the behavior distribution of the user in the exhibition area, and analyzing the behavior duration of the user; by constructing a behavior transition matrix M, analyzing the behavior path of the user in the exhibition area;

[0050] It also includes establishing a user preference analysis model, and the user preference analysis model uses the user-mobile phone interaction matrix to predict the potential interest points and purchase intentions of the user.

[0051] The present invention also provides a user behavior analysis device for anti-theft of mobile devices in the exhibition hall, including:

[0052] A multi-modal sensor data acquisition module for collecting Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data, and ambient light sensor data

[0053] A multi-modal sensor data preprocessing module for performing Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; preprocessing the collected acceleration and angular velocity data using a zero-phase filter; preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; preprocessing the collected ambient light sensor data using an exponentially weighted moving average filter;

[0054] A multi-modal data time alignment and fusion module for using the sampling rate of the preprocessed acceleration and angular velocity data as the reference time to map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp;

[0055] A user behavior recognition module for defining user behavior through the acceleration, angular velocity data, the geomagnetic sensor data, and the ambient light sensor data;

[0056] A user behavior database construction module for constructing a user behavior database according to defined user behaviors;

[0057] A hybrid recognition model construction and training module for constructing a CNN-LSTM hybrid recognition model, where the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; training the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtaining a user behavior recognition result by using the real-time collected multi-modal sensor data through the trained CNN-LSTM hybrid recognition model;

[0058] A user behavior-Bluetooth signal strength fusion judgment module for calculating a comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; judging whether the comprehensive score exceeds a preset threshold, and triggering an anti-theft alarm if the comprehensive score exceeds the preset threshold.

[0059] As an optimal solution for the user behavior analysis device for anti-theft of exhibition hall mobile devices, in the multi-modal sensor data preprocessing module, estimate the true value of the current Bluetooth signal strength data through a prediction step and an update step;

[0060] In the prediction step, predict the value of the Bluetooth signal strength data at the current moment and the error covariance P k|k-1 :

[0061]

[0062] P k|k-1 = A·P k-1|k-1 ·A T + Q

[0063] In the formula, A is the state transition matrix; Q is the process noise covariance;

[0064] In the update step, calculate the Kalman gain K k , and update the value of the Bluetooth signal strength data according to the Kalman gain K k and the error covariance P and error covariance P k|k-1 :

[0065]

[0066]

[0067] P k|k =(1 - K k )·P k|k-1

[0068] In the formula, zk is the currently measured Bluetooth signal strength data value, and R is the observation noise covariance; The Bluetooth signal strength data value after filtering at the current time t; P k|k The error covariance of the Bluetooth signal strength data after filtering at the current time t.

[0069] As a preferred solution for the user behavior analysis device for preventing theft of mobile devices in the exhibition hall, in the multi-modal sensor data preprocessing module, the formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is:

[0070]

[0071] In the formula, d bt (t) is the distance between the mobile device estimated by Bluetooth and the display stand; A` is the Bluetooth signal strength data value at the reference distance; n is the path loss exponent of signal propagation.

[0072] As a preferred solution for the user behavior analysis device for preventing theft of mobile devices in the exhibition hall, in the multi-modal sensor data preprocessing module, two-way filtering is used, that is, a zero-phase filter is applied once in the forward and backward directions to cancel the phase shift;

[0073] The zero-phase filter uses a Butterworth filter or a Chebyshev filter;

[0074] In the multi-modal sensor data preprocessing module, the formula for preprocessing the collected geomagnetic sensor data with a third-order Savitzky-Golay filter is:

[0075]

[0076] In the formula, c k is the coefficient of the filter; K is the window size, and a suitable window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing with a third-order Savitzky-Golay filter; m(t + k) is the geomagnetic sensor data before preprocessing with a third-order Savitzky-Golay filter.

[0077] As a preferred solution for the user behavior analysis device for preventing theft of mobile devices in the exhibition hall, in the multi-modal sensor data preprocessing module, the formula for preprocessing the collected ambient light sensor data with an exponentially weighted moving average filter is:

[0078] L filtered (t) = γ·L(t) + (1 - γ)·L filtered (t - 1)

[0079] where L(t) is the ambient light intensity at the current moment, and L filtered (t) is the filtered light intensity at the current moment; L filtered (t - 1) is the filtered light intensity at the previous moment; γ is the smoothing coefficient of the filtering.

[0080] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in the exhibition hall, in the multi-modal data time alignment and fusion module, the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same time stamp by linear interpolation or nearest neighbor interpolation methods.

[0081] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in the exhibition hall, in the user behavior database construction module, the input data of the user behavior database is the multi-modal sensor data within a time window; the output data of the user behavior database represents the category of user behavior;

[0082] The data X within each time window and the corresponding behavior label y together form a sample and are added to the data set D; the size N of the data set D depends on the acquisition duration and the window sliding setting; the form of the data set D is:

[0083]

[0084] where X (i) is the multi-modal sensor data of the i-th sample, with a dimension of T w ×d; y (i) is the behavior label of the i-th sample.

[0085] As a preferred solution of the user behavior - Bluetooth signal strength fusion and judgment module for anti-theft of mobile devices in the exhibition hall:

[0086] By assigning weights to the user behavior recognition result and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is:

[0087] S(t) = w1·f behavior (y pred ) + w2·f distance (d bt (t))

[0088] where f behavior is the risk score corresponding to the user behavior recognition result y pred ; f distance is the risk score corresponding to the Bluetooth signal strength distance d bt (t); w1, w2 are the weights of the behavior recognition result y pred and the Bluetooth signal strength distance dbt The weight of (t).

[0089] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in the exhibition hall, in the user behavior-Bluetooth signal strength fusion judgment module, the risk score is divided into a behavior recognition risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

[0090] As a preferred solution of the user behavior analysis device for anti-theft of mobile devices in the exhibition hall, it further includes a user usage behavior preference analysis module, which is used to record and store user behavior data, and identify and store the interaction behavior between each user and the exhibited mobile device as a behavior sequence. The behavior sequence includes the behavior category, the timestamp of the behavior occurrence, the behavior duration, and the location where the behavior occurs.

[0091] The user usage behavior preference analysis module is also used to analyze the interaction frequency between the user and the exhibited mobile phone, count the behavior distribution of the user in the exhibition area, and analyze the behavior duration of the user; by constructing a behavior transition matrix M, analyze the behavior path of the user in the exhibition area.

[0092] The user usage behavior preference analysis module is also used to establish a user preference analysis model, and the user preference analysis model uses the user-mobile phone interaction matrix to predict the potential interest points and purchase intentions of the user.

[0093] The present invention has the following advantages:

[0094] First, multi-modal sensor fusion, more accurate anti-theft monitoring:

[0095] By fusing the Bluetooth signal strength RSSI data, the acceleration and angular velocity data of the IMU, the geomagnetic sensor, and the ambient light sensor data, the present invention can accurately judge the relative movement and position change of the mobile phone in the exhibition hall, avoiding the problems of single sensor being vulnerable to interference and high false alarm rate; especially through the weighted judgment method, combining the user behavior recognition result with the Bluetooth signal strength RSSI data, enabling the anti-theft system to intelligently analyze the user behavior and the physical position of the mobile phone, significantly improving the accuracy of anti-theft judgment; it can flexibly adjust the judgment strategy according to different behaviors and Bluetooth signal strengths, thereby reducing false alarms and missed alarms.

[0096] Second, intelligent anti-theft mechanism, adapting to different scenarios:

[0097] The present invention proposes a dynamic weight adjustment and threshold self - adaptation mechanism, which can adjust the weights and thresholds of anti - theft judgment according to the activity level of the display area, time period, and the behavior trends of the mobile phone; it can adaptively adjust the anti - theft sensitivity to ensure rapid response to potential theft behavior during peak hours while maintaining a low false - alarm rate during off - peak hours; through time - dependent analysis, it can identify behavior trends within a short period, thus more quickly identifying the situation where the mobile phone is stolen or abnormally moved.

[0098] Third, real - time identification of user behavior to enhance user experience:

[0099] With the help of the CNN - LSTM model and attention mechanism, the present invention can real - time identify various behaviors of users in the display area, including picking up the mobile phone to view, light operation, taking pictures, rotating the mobile phone, etc. This behavior identification not only helps with anti - theft but also enables the store to understand the behavior patterns and preferences of users. The behavior identification results can help the store optimize the layout of the display area and improve the user experience. For example, it can identify the mobile phone models or functions that users use more frequently, helping to adjust the display focus.

[0100] Fourth, analysis of user preferences to support personalized marketing:

[0101] By recording and analyzing user behaviors, the present invention can obtain information such as the frequency of user behaviors, stay duration, and behavior sequence, providing data support for predicting user preferences. It can accurately predict the potential interest points of users and make personalized recommendations based on user behavior data. Based on these analysis results, the store can provide customized promotion plans and product recommendations for different user groups, increasing the user's purchase intention and conversion rate.

[0102] Fifth, high - efficiency system scalability and adaptability:

[0103] The present invention can adapt to different display environments and device types, can adjust and optimize the model according to the requirements of different display areas, has high scalability, can be easily integrated into the existing display area anti - theft system, and realizes function expansion through software upgrade without additional hardware modification, reducing the implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0104] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are merely exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained based on the provided drawings.

[0105] Figure 1It is a schematic flowchart of a user behavior analysis method for anti-theft of mobile devices in the exhibition hall provided in the embodiments of the present invention;

[0106] Figure 2 It is a schematic diagram of the architecture of a user behavior analysis device for anti-theft of mobile devices in the exhibition hall provided in the embodiments of the present invention. Specific embodiments

[0107] The following specific embodiments illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0108] Embodiment 1

[0109] See Figure 1 , Embodiment 1 of the present invention provides a user behavior analysis method for anti-theft of mobile devices in the exhibition hall, including the following steps:

[0110] Multi-modal sensor data acquisition: Collect Bluetooth signal strength data, acceleration and angular velocity data, geomagnetic sensor data, and ambient light sensor data

[0111] Multi-modal sensor data preprocessing: Perform Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; perform preprocessing on the collected acceleration and angular velocity data using a zero-phase filter; perform preprocessing on the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; perform preprocessing on the collected ambient light sensor data using an exponentially weighted moving average filter;

[0112] Multi-modal data time alignment and fusion: Use the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, and map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same time stamp;

[0113] User behavior recognition: Define user behavior through the acceleration and angular velocity data, the geomagnetic sensor data, and the ambient light sensor data;

[0114] User behavior database construction: Construct a user behavior database according to the defined user behavior;

[0115] Construction and Training of Hybrid Recognition Model: Construct a CNN-LSTM hybrid recognition model, which includes CNN module parameters, LSTM module parameters, attention mechanism parameters, fully connected layer and output layer; Train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain the user behavior recognition result by using the multi-modal sensor data collected in real time through the trained CNN-LSTM hybrid recognition model;

[0116] Fusion Judgment of User Behavior and Bluetooth Signal Strength: Assign weights to the user behavior recognition result and the Bluetooth signal strength distance, and calculate the comprehensive score; Judge whether the comprehensive score exceeds the preset threshold, and if the comprehensive score exceeds the preset threshold, trigger the anti-theft alarm.

[0117] In this embodiment, in order to adapt to the different sampling rates of Bluetooth, IMU, geomagnetism and ambient light sensors, the data of each sensor is preprocessed first, and then through time alignment and fusion, consistent time series data is generated for subsequent algorithms. The specific process is as follows:

[0118] A1. Acquisition and Processing of Bluetooth Signal Strength

[0119] The sampling rate of the Bluetooth signal is set to 10Hz. Since the Bluetooth signal strength RSSI is usually greatly fluctuated due to environmental interference, the Kalman filter is used to smooth the RSSI data, and the true value of the current RSSI is estimated through two steps of prediction and update.

[0120] Specifically, predict the RSSI value at the current moment and the error covariance P k|k-1

[0121]

[0122] P k|k-1 = A·P k-1|k-1 ·A T + Q

[0123] where A is the state transition matrix, usually taking A = 1. Q is the process noise covariance, reflecting the change amount of RSSI. Conventionally, if the RSSI change is relatively stable, Q can be set to a smaller value, such as Q = 0.01. If the RSSI change is large, Q can be set to a larger value, such as Q = 0.1. Considering the characteristics of this scenario, Q is set to 0.1.

[0124] Specifically, in the update step, first calculate the Kalman gain K k :

[0125]

[0126] Update the RSSI value and the error covariance P k|k-1 :

[0127]

[0128] P k|k =(1 - K k )·P k|k-1

[0129] where z k is the currently measured RSSI value, R is the observation noise covariance, reflecting the uncertainty of RSSI measurement. Conventionally, if the measurement accuracy is high, R takes a smaller value, such as R = 0.1. If the measurement noise is large, R takes a larger value, such as R = 1. Considering the characteristics of this scenario, R is set to 0.5.

[0130] In addition, the initial state is set to the first measured RSSI value. The initial error covariance P 0|0 is set to a larger value, such as 1, indicating a higher uncertainty about the initial state. The RSSI value after Kalman filtering can be used for subsequent distance estimation

[0131]

[0132] where d bt (t) is the distance (unit: meter) between the mobile phone and the display stand estimated by Bluetooth. A` is the RSSI value at the reference distance (usually 1 meter), which is a constant corrected according to the environment; is the filtered RSSI value (unit: dBm) at the current moment t; n is the path loss exponent of signal propagation, which is also corrected according to the environment.

[0133] B1. Inertial Measurement Unit (IMU) Data Acquisition and Processing

[0134] The sampling rate of IMU data is 100Hz, and IMU includes accelerometer data and gyroscope data. To filter noise and retain motion characteristics, zero-phase IIR filters are used for preprocessing the acceleration and angular velocity data. The purpose of the zero-phase IIR filter is to smooth the signal without introducing phase distortion. Conventional IIR filters usually introduce phase shift, which will cause distortion of the time characteristics of sensor data. To avoid phase distortion, the embodiments of the present invention use bidirectional filtering, that is, applying the IIR filter once forward and once backward, so as to cancel the phase shift.

[0135] Specifically, for the accelerometer data a(t), there is:

[0136] Forward filtering: a f (t)=IIR(a(t))

[0137] Backward filtering: a filtered (t) = IIR(a f (t))

[0138] Wherein, IIR is a predefined filtering function, usually a low-pass filter. Similarly, for the gyroscope data ω(t), backward and forward filtering is also performed to obtain the filtered angular velocity ω filtered (t).

[0139] Specifically, the design of the IIR filter: Usually, a low-pass filter is used to smooth the acceleration data and remove high-frequency noise. Commonly used types of IIR filters include Butterworth filters or Chebyshev filters. In motion detection, Butterworth filters are often used because of their smooth frequency response curve. Therefore, a second-order Butterworth filter is selected in this embodiment. A lower order can reduce the complexity and computational amount of the filter while maintaining a good smoothing effect. Since human motion is usually in the low-frequency range (1 - 20 Hz), the cut-off frequency can be set to 5 - 10 Hz to retain the characteristics of human motion while filtering out high-frequency noise. The change frequency of the gyroscope data may be slightly higher, and the cut-off frequency can usually be set to 10 - 20 Hz to retain the characteristics of the angular velocity change.

[0140] C1. Acquisition and processing of geomagnetic sensor data

[0141] The sampling rate of the geomagnetic sensor is 50 Hz, and the geomagnetic sensor is used to detect the directional change of the mobile phone. In order to ensure the smoothness and stability of the data, the geomagnetic data m(t) is processed by a third-order Savitzky-Golay filter.

[0142]

[0143] Wherein, c k is the coefficient of the filter (determined according to the order of the local polynomial fitting and the window size). For most sensor signals, a third-order polynomial is usually selected, that is, n = 3. The third-order polynomial can well retain the trend of the signal while smoothing high-frequency noise. K is the window size, and a suitable window size is selected according to the characteristics of the signal and the application scenario. Considering the sampling rate, a window size of 7 points is selected.

[0144] D1. Acquisition and processing of ambient light sensor data

[0145] The sampling rate of the ambient light sensor is 20 Hz. In order to process the instantaneous spikes or transitional changes in the ambient light data, an exponentially weighted moving average (EWMA) filter is used for processing.

[0146] L filteredL(t) = γ·L(t)+(1 - γ)·L filtered (t - 1)

[0147] where L(t) is the ambient light intensity at the current moment. L filtered (t) is the filtered light intensity. γ is the smoothing coefficient of the filter, usually taking a value between 0.2 and 0.5.

[0148] E1. Multimodal Data Synchronization and Time Alignment

[0149] Since the data sampling frequencies of the sensors are different, it is necessary to align these data in time. Taking the sampling rate of the IMU (100Hz) as the reference time t k , the data of other sensors are mapped to the same timestamp through linear interpolation or nearest neighbor interpolation methods.

[0150] Among them, (a1) Bluetooth data interpolation, using nearest neighbor interpolation:

[0151] RSSI mapped (t k ) = RSSI filtered (t k′ ) where t k′ ≤t k <t k′+1

[0152] In the formula, RSSI mapped (t k ) is the RSSI value mapped to the target time point t k ; RSSI filtered .t k′ / is the RSSI data that has been filtered at the time point t k′ ;

[0153] (b1) Geomagnetic data interpolation, using linear interpolation:

[0154]

[0155] In the formula, m mapped (t k ) is the geomagnetic sensor data interpolated at the target time point t k ; m filtered (t i ) is the geomagnetic sensor data that has been filtered at the time point t i ; m filtered (t i+1 ) is the geomagnetic sensor data that has been filtered at the time point t i+1 ;

[0156] (c1) Ambient light data interpolation, using linear interpolation:

[0157]

[0158] Wherein, L mapped (t k ) is the light intensity data interpolated at the target time point t k ; L filtered (t j ) is the light intensity data after filtering at the time point t j ; L filtered (t j+1 ) is the light intensity data after filtering at the time point t j+1 .

[0159] In this embodiment, the specific process of user behavior recognition is as follows:

[0160] A2. User behavior definition

[0161] By combining the data of the IMU (accelerometer and gyroscope), geomagnetic sensor, and ambient light sensor, a variety of detailed user behaviors are defined. These behaviors include but are not limited to:

[0162] (a2) Static state:

[0163] The mobile phone is placed on the display stand and not touched by the user.

[0164] (b2) Mild operation (viewing, swiping):

[0165] The user stands in front of the display stand and touches the screen gently to perform operations such as swiping, clicking, or viewing information.

[0166] (c2) Pick up the mobile phone to view:

[0167] The user picks up the mobile phone to experience, which may be to view the appearance, weight, etc. of the mobile phone, holding the mobile phone without performing other operations.

[0168] (d2) Lift the mobile phone to take a photo or video:

[0169] The user lifts the mobile phone to take a photo or video, and the angle of the mobile phone changes significantly.

[0170] (e2) Rotate the mobile phone (view the back or side of the mobile phone):

[0171] The user rotates the mobile phone to view its back, side, etc.

[0172] (f2) The mobile phone is put into the pocket or bag:

[0173] The user puts the mobile phone into the pocket or bag, or blocks the light for experience.

[0174] (g2) The mobile phone is forcibly moved:

[0175] The mobile phone is quickly moved or taken away from the display stand, which may be an unauthorized movement.

[0176] B2. Construction of the user behavior database

[0177] (a3) Database structure

[0178] Input data: X is the multi-modal sensor data within a time window, with a dimension of T w ×d, where T w is the time window length, and d is the dimension of the sensor data (acceleration, angular velocity, geomagnetism, and ambient light).

[0179] X(t k ) = (a filtered (t k ), ω filtered (t k ), m filtered (t k ), L filtered (t k ))

[0180] Output label: y, representing the category of the user behavior. The behavior categories are as follows:

[0181] y ∈ {0, 1, 2, 3, 4, 5, 6}

[0182] (b3) Data acquisition process

[0183] Set the time window length T w , and the sampling rate is determined by the sensor (100Hz for IMU, 50Hz for geomagnetism, and 20Hz for ambient light). The collected data is processed through interpolation and filtering.

[0184] The data X within each time window and the corresponding behavior label y together form a sample and are added to the dataset D.

[0185] The size N of the dataset depends on the acquisition duration and the window sliding setting.

[0186] Finally, the form of the dataset D is:

[0187] D = {(X (i) , y (i) ) | i = 1, 2, …, N}

[0188] X (i) is the multi-modal sensor data of the i-th sample, with a dimension of T w ×d.

[0189] y (i)is the behavior label of the i-th sample.

[0190] C2. Model Structure and Specific Parameters

[0191] In this embodiment, a CNN-LSTM hybrid model is used, combined with an attention mechanism, to fully utilize the local and global features of time series data. The following are the specific architecture and parameter settings of the model.

[0192] (a4) CNN Module Parameters

[0193] The convolutional layer (Conv1D) is used to extract local features from the sensor input data. Among them: Input dimension: T w ×d (time series length T w and feature dimension d).

[0194] Kernel size: 3

[0195] Number of filters: 64

[0196] Stride: 1

[0197] Activation function: ReLU

[0198] Output dimension: (T w -2) × 64 (time steps reduced by 2, number of channels is 64)

[0199] The pooling layer is used for downsampling convolutional features to reduce the computational amount, and the pooling window size is set to 2.

[0200] (b4) LSTM Module Parameters

[0201] The LSTM layer is used to capture temporal dependencies. Among them,

[0202] Input dimension: ((T w -2) / 2) × 64

[0203] Number of hidden units: 128

[0204] Output dimension: 128 (dimension of the hidden state output for each time step)

[0205] Dropout: 0.5 (to prevent overfitting)

[0206] (c4) Attention Mechanism Parameters

[0207] Attention weight calculation: For each time step t, calculate the attention weight α t :

[0208]

[0209] Among them, W a is a learnable weight matrix, and h t is the hidden state of the LSTM.

[0210] Context vector calculation: Calculate the global context vector c through weighted summation:

[0211]

[0212] (d4) Fully connected layer and output layer

[0213] The fully connected layer is used to map the context vector to the behavior classification space:

[0214] Input dimension: 128 (dimension of the context vector)

[0215] Output dimension: 64

[0216] Activation function: ReLU

[0217] The output layer uses the softmax activation function to output the probability distribution of the behavior categories.

[0218] Input dimension: 64

[0219] Output dimension: 7 (number of behavior categories)

[0220] Activation function: softmax

[0221] Output: Probability distribution of behavior categories

[0222] D2. Training process and parameter settings

[0223] (a5) Loss function

[0224] Use the cross-entropy loss function to measure the difference between the model prediction result and the true label:

[0225]

[0226] Among them, is the true label of the i-th sample (using one-hot encoding). is the model prediction probability distribution of the i-th input

[0227] (b5) Optimization algorithm

[0228] Use the Adam optimizer to update the model parameters:

[0229] Initial learning rate α = 0.001; β1 = 0.9, β2 = 0.999

[0230] Among them, β1 controls the calculation of the first-order momentum (accumulation of gradients) for accelerating gradient descent. β2 controls the calculation of the second-order momentum (accumulation of squared gradients) for adjusting the adaptive change of the learning rate.

[0231] Learning rate decay: The learning rate decays by 0.1 every 10 epochs to ensure fast convergence in the initial stage and more refined search for the optimal solution in the later stage.

[0232] (c5) Training parameters

[0233] Batch size: 64

[0234] Number of training epochs: 50

[0235] Dropout: 0.5 (used for LSTM layers to prevent overfitting)

[0236] Regularization: L2 regularization coefficient is 0.001

[0237] (d5) Dataset division

[0238] Training set: 80% of the data is used to train the model.

[0239] Validation set: 10% of the data is used to adjust the model hyperparameters.

[0240] Test set: 10% of the data is used to evaluate the model performance.

[0241] (e5) Data augmentation and preprocessing

[0242] The sensor data is standardized so that its mean is 0 and variance is 1.

[0243]

[0244] Among them, μ x and σ x are the mean and standard deviation of the sensor data respectively. In addition, the data is augmented by adding noise, random shearing, or time reversal to improve the generalization ability of the model.

[0245] (f5) Final output behavior labels

[0246] After data processing and model training, the probability distribution obtained by the softmax function after inputting the data:

[0247]

[0248] Final output behavior labels:

[0249]

[0250] In the formula, is the predicted probability distribution of the behavior category; W o is the weight matrix of the linear classifier, with a dimension of K×D, where K is the total number of behavior categories and D is the input feature vector; c represents the feature representation vector obtained after data processing and model training; b o represents the bias vector of the linear classifier, with a dimension of K.

[0251] In this embodiment, the specific process of the fusion judgment of the user behavior and the Bluetooth signal strength RSSI is as follows:

[0252] (a6) Weighted judgment alarm method

[0253] By assigning different weights to the user behavior recognition result y pred and the Bluetooth signal strength RSSI distance d bt (t), a comprehensive score S(t) is calculated for mobile phone anti-theft judgment. If the comprehensive score S(t) exceeds a preset threshold S threshold , the anti-theft alarm is triggered:

[0254] S(t) = w1·f behavior (y pred ) + w2·f distance (d bt (t))

[0255] where f behavior is the risk score corresponding to the user behavior recognition result. f distance is the risk score corresponding to the Bluetooth RSSI distance. w1 and w2 are the weights of the behavior recognition result and the RSSI distance, reflecting their respective influences on the comprehensive score.

[0256] (b6) Mapping of risk scores

[0257] To enhance the flexibility of the system, the risk scores are more finely divided according to different behavior categories and RSSI distances:

[0258] Behavior recognition risk score f behavior (y pred ):

[0259] High-risk behavior: Behavior category y pred = 6 (the mobile phone is forcibly moved or stolen), and the score is 1.0.

[0260] Behavior category y pred = 5 (the mobile phone is put into a pocket or a bag), and the score is 0.7. This behavior has a certain risk but is not necessarily theft.

[0261] Low-risk behavior: Behavior category ypred {0,1,2,3,,4} (stationary, light operation, picking up to check, rotating the phone), with a score of 0.5.

[0262] RSSI distance risk score f distance (d bt (t)):

[0263] High risk distance: When RSSI distance d bt (t)>d threshold , the phone is too far from the display stand and the score is 1.0.

[0264] Medium risk distance: When the RSSI distance is close to the threshold d bt (t)∈[d threshold -δ,d threshold ], indicating that the phone may be taken away from the display stand, with a score of 0.7.

[0265] Low risk distance: When RSSI distance d bt (t)≤d threshold -δ, indicating that the phone is still near the display stand, with a score of 0.3.

[0266] (c6) Dynamic weight adjustment mechanism

[0267] In order to enhance the intelligence of the system, a dynamic adjustment mechanism is adopted to change the weights w1 and w2 of the behavior recognition results and RSSI distance based on the following factors:

[0268] Time Factor:

[0269] During peak hours of mobile phone display (e.g., busy hours in a store), the weight w1 of user behavior should be increased because user behavior in the display area is more frequent and diverse. During off-peak hours, the weight w2 of Bluetooth RSSI should be higher because there is less activity in the display area and the risk of mobile phones being forcibly moved or stolen is higher.

[0270] Mobile phone trends:

[0271] If the phone continuously detects high-risk behaviors in a short period of time, the system should increase the weight w1 of behavior recognition, because the risk of user behavior is higher in this case. Conversely, if the phone maintains low-risk behavior (such as stillness or light operation) in a short period of time, the weight w2 of RSSI should be increased.

[0272] In this embodiment, the specific process of user usage behavior and preference analysis is as follows:

[0273] Analysis of users' behaviors and preferences in the mobile phone display area can help stores deeply understand how customers interact, their points of interest, and potential purchase intentions. This information can not only optimize the layout of mobile phone displays but also provide data support for personalized recommendations and marketing campaign design. Based on the result y of user behavior recognition pred and sensor data, combined with multi-dimensional data analysis methods, in-depth analysis of users' usage behaviors and preferences will be carried out.

[0274] (a7) Recording and storing of user behavior data

[0275] In the display area, each interaction of users will generate a series of sensor data and behavior recognition results. To analyze users' behaviors and preferences, the system needs to record and store this data.

[0276] Behavior recording: Each interaction behavior of a user with the displayed mobile phone will be recognized and stored as a behavior sequence. Specifically, it includes: behavior category, timestamp of behavior occurrence, behavior duration, and location where the behavior occurs.

[0277] (b7) Dimensions of user behavior analysis

[0278] Analyze the interaction frequency of users with the displayed mobile phones to help merchants understand which users are more interested in the mobile phone display area and which behaviors are the most frequent. The specific dimensions are as follows:

[0279] Behavior distribution: Statistically analyze the distribution of various behaviors of users in the display area, such as the proportions of behaviors like standing still, light operations, taking photos, rotating the mobile phone, etc.

[0280]

[0281] Among them, N(y) is the number of occurrences of behavior category y. N total is the total number of all behaviors.

[0282] Analysis of behavior duration: Behavior duration can reflect the degree of users' interest in certain interactions.

[0283] Generally, the longer the user stays, the greater their interest in a certain mobile phone or a certain function.

[0284]

[0285] Among them, is the duration of behavior y.

[0286] Behavior Sequence and Pattern Analysis: The sequence and pattern of user behavior can reveal the typical interaction process of users. For example, users may first pick up the phone to check its appearance, then perform light operations, and finally raise the phone to take pictures. By constructing a behavior transition matrix M, analyze the typical behavior path of users in the display area:

[0287]

[0288] where N(y i →y i ) is the number of times a user transfers from behavior y i to y i . N(y i ) is the total number of times behavior y i occurs.

[0289] Through the behavior transition matrix, we can identify the typical interaction patterns of users. For example, users may be more inclined to raise the phone to take pictures immediately after light operations, which means they may have a high interest in the operation experience and camera function of the phone.

[0290] (c7) User Preference Analysis Model

[0291] Based on the user behavior analysis, further establish a user preference analysis model to predict the potential interest points and purchase intentions of users. The embodiment of the present invention adopts collaborative filtering technology to mine user preferences. The collaborative filtering model can predict their potential interest in other mobile phone models or functions based on the historical behavior data of users.

[0292] Specifically, construct a user - mobile phone interaction matrix R, where the element R ui represents the interaction frequency or stay time between user u and mobile phone i:

[0293]

[0294] where, is the duration of this interaction behavior.

[0295] Interest Prediction: Based on the user - mobile phone interaction matrix, use collaborative filtering algorithms (such as matrix factorization) to predict the potential interest of users in other mobile phones. Through the prediction matrix mobile phone models that users may be interested in can be recommended to users.

[0296] In summary, the present invention collects Bluetooth signal strength data, acceleration and angular velocity data, geomagnetic sensor data, and ambient light sensor data; performs Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; preprocesses the collected acceleration and angular velocity data using a zero-phase filter; preprocesses the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; preprocesses the collected ambient light sensor data using an exponentially weighted moving average filter; uses the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, and maps the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp; defines user behavior through the acceleration, angular velocity data, the geomagnetic sensor data, and the ambient light sensor data; constructs a user behavior database according to the defined user behavior; constructs a CNN-LSTM hybrid recognition model, where the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; trains the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtains a user behavior recognition result by using the real-time collected multi-modal sensor data through the trained CNN-LSTM hybrid recognition model; calculates a comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; determines whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, triggers an anti-theft alarm. By fusing the Bluetooth signal strength RSSI data, the acceleration and angular velocity data of the IMU, the geomagnetic sensor, and the ambient light sensor data, the present invention can accurately judge the relative movement and position change of the mobile phone in the exhibition hall, avoiding the problems of single sensor being vulnerable to interference and high false alarm rate; especially through the weighted judgment method, combining the user behavior recognition result with the Bluetooth signal strength RSSI data, enabling the anti-theft system to intelligently analyze the user behavior and the physical position of the mobile phone, significantly improving the accuracy of anti-theft judgment; being able to flexibly adjust the judgment strategy according to different behaviors and Bluetooth signal strengths, thereby reducing false alarms and missed alarms. The present invention proposes a dynamic weight adjustment and threshold adaptive mechanism, which can adjust the weights and thresholds of anti-theft judgment according to the activity level, time period of the exhibition area, and the behavior trend of the mobile phone; can adaptively adjust the anti-theft sensitivity to ensure a quick response to potential theft behaviors during peak hours, while maintaining a low false alarm rate during off-peak hours; through time-dependent analysis, can identify the behavior trend within a short period of time, thus more quickly identifying the situation where the mobile phone is stolen or abnormally moved. With the help of the CNN-LSTM model and the attention mechanism, the present invention can real-time identify various behaviors of users in the exhibition area, including picking up the mobile phone to view, mild operation, taking pictures, rotating the mobile phone, etc. This behavior recognition not only helps with anti-theft, but also helps the store understand the behavior patterns and preferences of users.The behavioral recognition results can help the store optimize the layout of the display area and improve the user experience. For example, it can identify the mobile phone models or functions that users use more frequently, and help adjust the display focus. Through the recording and analysis of user behaviors, the present invention can obtain information such as user behavior frequency, stay duration, and behavior sequence, providing data support for user preference prediction. It can accurately predict users' potential interest points and make personalized recommendations based on user behavior data. The store can provide customized promotion plans and product recommendations for different user groups based on these analysis results, improving users' purchase intention and conversion rate. The present invention can adapt to different display environments and device types, can adjust and optimize the model according to different display area requirements, has high scalability, can be easily integrated into the existing anti-theft system for the display area, and realizes function expansion through software upgrade without additional hardware modification, reducing the implementation cost.

[0297] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In such a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiments of the present disclosure, and these multiple devices will interact with each other to complete the described method.

[0298] It should be noted that some embodiments of the present disclosure are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order from those in the above embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0299] Embodiment 2

[0300] See Figure 2 , Embodiment 2 of the present invention further provides a user behavior analysis device for anti-theft of mobile devices in an exhibition hall, including:

[0301] A multimodal sensor data acquisition module 001, configured to acquire Bluetooth signal strength data, acceleration and angular velocity data, geomagnetic sensor data, and ambient light sensor data

[0302] The multi-modal sensor data preprocessing module 002 is used to perform Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; perform preprocessing on the collected acceleration and angular velocity data using a zero-phase filter; perform preprocessing on the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; perform preprocessing on the collected ambient light sensor data using an exponentially weighted moving average filter;

[0303] The multi-modal data time alignment and fusion module 003 is used to use the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, and map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp;

[0304] The user behavior recognition module 004 is used to define user behavior through the acceleration, angular velocity data, the geomagnetic sensor data, and the ambient light sensor data;

[0305] The user behavior database construction module 005 is used to construct a user behavior database according to the defined user behavior;

[0306] The hybrid recognition model construction and training module 006 is used to construct a CNN-LSTM hybrid recognition model, and the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain a user behavior recognition result by using the real-time collected multi-modal sensor data through the trained CNN-LSTM hybrid recognition model;

[0307] The user behavior-Bluetooth signal strength fusion judgment module 007 is used to calculate a comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; judge whether the comprehensive score exceeds a preset threshold, and if the comprehensive score exceeds the preset threshold, trigger an anti-theft alarm.

[0308] In this embodiment, in the multi-modal sensor data preprocessing module 002, the true value of the current Bluetooth signal strength data is estimated through a prediction step and an update step;

[0309] In the prediction step, predict the value of the Bluetooth signal strength data at the current moment and the error covariance P k|k-1 :

[0310]

[0311] P k|k-1 = A·P k-1|k-1 ·AT +Q

[0312] Wherein, A is the state transition matrix; Q is the process noise covariance;

[0313] In the update step, calculate the Kalman gain K k , according to the Kalman gain K k Update the value of the Bluetooth signal strength data and the error covariance P k|k-1 :

[0314]

[0315]

[0316] P k|k =(1 - K k )·P k|k-1

[0317] Wherein, z k is the currently measured Bluetooth signal strength data value, and R is the observation noise covariance.

[0318] In this embodiment, in the multi-modal sensor data preprocessing module 002, the formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is:

[0319]

[0320] bt (t) is the distance between the mobile device estimated by Bluetooth and the display stand; A` is the Bluetooth signal strength data value at the reference distance; n is the path loss exponent of signal propagation.

[0321] In this embodiment, in the multi-modal sensor data preprocessing module 002, two-way filtering is used, that is, a zero-phase filter is applied once in the forward and backward directions to cancel the phase shift;

[0322] The zero-phase filter uses a Butterworth filter or a Chebyshev filter;

[0323] In the multi-modal sensor data preprocessing module 002, the formula for preprocessing the collected geomagnetic sensor data with a third-order Savitzky-Golay filter is:

[0324]

[0325] Wherein, c k is the coefficient of the filter; K is the window size, and a suitable window size is selected according to the characteristics of the signal and the application scenario; m filtered(t) is the geomagnetic sensor data after preprocessing by a third-order Savitzky-Golay filter; m(t + k) is the geomagnetic sensor data before preprocessing by a third-order Savitzky-Golay filter.

[0326] In this embodiment, in the multimodal sensor data preprocessing module 002, the formula for preprocessing the collected ambient light sensor data by an exponentially weighted moving average filter is:

[0327] L filtered (t) = γ · L(t) + (1 - γ) · L filtered (t - 1)

[0328] In the formula, L(t) is the ambient light intensity at the current moment, and L filtered (t) is the filtered light intensity at the current moment; L filtered (t - 1) is the filtered light intensity at the previous moment; γ is the smoothing coefficient of the filter.

[0329] In this embodiment, in the multimodal data time alignment and fusion module 003, the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp by linear interpolation or nearest neighbor interpolation methods.

[0330] In this embodiment, in the user behavior database construction module 005, the input data of the user behavior database is the multimodal sensor data within a time window; the output data of the user behavior database represents the category of user behavior;

[0331] The data X within each time window and the corresponding behavior label y together form a sample and are added to the dataset D; the size N of the dataset D depends on the acquisition duration and the window sliding setting; the form of the dataset D is:

[0332]

[0333] In the formula, X (i) is the multimodal sensor data of the i-th sample, with a dimension of T w × d; y (i) is the behavior label of the i-th sample.

[0334] In this embodiment, in the user behavior - Bluetooth signal strength fusion and judgment module 007:

[0335] By assigning weights to the user behavior recognition result and the Bluetooth signal strength distance, the formula for calculating the comprehensive score is:

[0336] S(t) = w1 · f behavior (y pred) + w2·f distance (d bt (t))

[0337] Where f behavior is the risk score corresponding to the user behavior recognition result y pred ; f distance is the risk score corresponding to the Bluetooth signal strength distance d bt (t); w1 and w2 are the weights of the behavior recognition result y pred and the Bluetooth signal strength distance d bt (t).

[0338] In this embodiment, in the user behavior-Bluetooth signal strength fusion judgment module 007, the risk score is divided into a behavior recognition risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

[0339] This embodiment further includes a user usage behavior preference analysis module 008, which is used to record and store user behavior data, and identify and store the interaction behavior of each user with the display mobile device as a behavior sequence. The behavior sequence includes behavior category, timestamp of behavior occurrence, behavior duration, and location where the behavior occurs;

[0340] The user usage behavior preference analysis module 008 is further used to analyze the interaction frequency of the user with the display phone, count the behavior distribution of the user in the display area, and analyze the behavior duration of the user; by constructing a behavior transition matrix M, analyze the behavior path of the user in the display area;

[0341] The user usage behavior preference analysis module 008 is further used to establish a user preference analysis model, and the user preference analysis model uses a user-phone interaction matrix to predict the potential interest points and purchase intentions of the user.

[0342] It should be noted that the information interaction, execution process, etc. between the above device modules, since they are based on the same concept as the method embodiment in Embodiment 1 of this application, bring the same technical effects as the method embodiment of this application. For specific content, please refer to the description in the method embodiment shown above in this application, and details will not be repeated here.

[0343] Embodiment 3

[0344] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, and a program code for a user behavior analysis method for anti-theft of a display hall mobile device is stored in the computer-readable storage medium. The program code includes instructions for executing a user behavior analysis method for anti-theft of a display hall mobile device in Embodiment 1 or any possible implementation manner thereof.

[0345] A computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that incorporates one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0346] Embodiment 4

[0347] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor;

[0348] The processor and the memory complete communication with each other through a bus; the memory stores program instructions executable by the processor, and the processor can execute a user behavior analysis method for anti-theft of exhibition hall mobile devices according to Embodiment 1 or any possible implementation manner thereof by invoking the program instructions.

[0349] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor that is implemented by reading software code stored in the memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0350] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center by wire (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (e.g., infrared, wireless, microwave, etc.).

[0351] Obviously, those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented by a general-purpose computing system. They can be centralized on a single computing system or distributed across a network composed of multiple computing systems. Optionally, they can be implemented by program code executable by the computing system, so that they can be stored in the storage system and executed by the computing system. And in some cases, the steps shown or described can be executed in a different order from here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0352] Although the present invention has been described in detail with general descriptions and specific embodiments above, based on the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of the present invention claimed.

Claims

1. A user behavior analysis method for anti-theft of mobile devices in exhibition halls, characterized in that, Including: Multi-modal sensor data acquisition: Collect Bluetooth signal strength data, acceleration and angular velocity data, geomagnetic sensor data, and ambient light sensor data Multi-modal sensor data preprocessing: Perform Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data; Preprocess the collected acceleration and angular velocity data using a zero-phase filter; Preprocess the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; Preprocess the collected ambient light sensor data using an exponentially weighted moving average filter; Multi-modal data time alignment and fusion: Use the sampling rate of the preprocessed acceleration and angular velocity data as the reference time, and map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp; User behavior recognition: Define user behavior through the acceleration and angular velocity data, the geomagnetic sensor data, and the ambient light sensor data; User behavior database construction: Construct a user behavior database based on the defined user behavior; Hybrid recognition model construction and training: Construct a CNN-LSTM hybrid recognition model, where the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; Train the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtain the user behavior recognition result using the real-time collected multi-modal sensor data through the trained CNN-LSTM hybrid recognition model; User behavior and Bluetooth signal strength fusion judgment: Calculate the comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance; Judge whether the comprehensive score exceeds the preset threshold. If the comprehensive score exceeds the preset threshold, trigger an anti-theft alarm; The formula for calculating the comprehensive score by assigning weights to the user behavior recognition result and the Bluetooth signal strength distance is: S(t) = w1·f behavior (y pred ) + w2·f distance (d bt (t)) Where f behavior is the risk score corresponding to the user behavior recognition result y pred ; f distance is the risk score corresponding to the Bluetooth signal strength distance d bt (t); w1 and w2 are the weights of the behavior recognition result y pred and the Bluetooth signal strength distance d bt (t). It also includes dividing the risk score into a behavior recognition risk score and a Bluetooth signal strength distance risk score according to the specified behavior category and Bluetooth signal strength distance.

2. The user behavior analysis method for anti-theft of exhibition hall mobile devices according to claim 1, characterized in that, During the process of performing Kalman filtering on the collected Bluetooth signal strength data to smooth the Bluetooth signal strength data, estimate the true value of the current Bluetooth signal strength data through the prediction step and the update step; In the prediction step, predict the value of the Bluetooth signal strength data at the current moment and the error covariance P k|k-1 : P k|k-1 = A·P k-1|k-1 ·A T + Q In the formula, A is the state transition matrix; Q is the process noise covariance; In the update step, calculate the Kalman gain K k , and based on the Kalman gain K k update the value of the Bluetooth signal strength data and the error covariance P k|k-1 : P k|k = (1 - K k ) · P k|k-1 where z k is the currently measured Bluetooth signal strength data value, and R is the observation noise covariance; is the Bluetooth signal strength data value after filtering at the current time t; P k|k is the error covariance of the Bluetooth signal strength data after filtering at the current time t; The formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is: where d bt (t) is the distance between the mobile device estimated by Bluetooth and the display stand; A` is the Bluetooth signal strength data value at the reference distance; n is the path loss exponent of signal propagation.

3. The user behavior analysis method for anti-theft of exhibition hall mobile devices according to claim 1, characterized in that During the process of preprocessing the collected acceleration and angular velocity data using a zero-phase filter, perform two-way filtering, that is, apply the zero-phase filter once forward and once backward to cancel the phase shift; The zero-phase filter uses a Butterworth filter or a Chebyshev filter.

4. The user behavior analysis method for anti-theft of exhibition hall mobile devices according to claim 1, wherein, The formula for preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter is: where c k is the coefficient of the filter; K is the window size, and a suitable window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing by the third-order Savitzky-Golay filter; m(t + k) is the geomagnetic sensor data before preprocessing by the third-order Savitzky-Golay filter; The formula for preprocessing the collected ambient light sensor data using an exponentially weighted moving average filter is: L filtered L(t) = γ·L(t) + (1 - γ)·L filtered (t - 1) where L(t) is the ambient light intensity at the current moment, and L filtered (t) is the filtered light intensity at the current moment; L filtered (t - 1) is the filtered light intensity at the previous moment; γ is the smoothing coefficient of the filter; Map the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp using linear interpolation or nearest neighbor interpolation methods.

5. A user behavior analysis method for anti-theft of exhibition hall mobile devices according to claim 1, characterized in that, The input data of the user behavior database is multi-modal sensor data within a time window; the output data of the user behavior database represents the categories of user behaviors. The data h within each time window and the corresponding behavior label y together form a sample, which is added to the dataset ; the dataset size N depends on the acquisition duration and the window sliding setting; the dataset is in the form of: where X (i) is the multimodal sensor data of the i-th sample, with dimension T w ×d; y (i) is the behavior label of the i-th sample.

6. A user behavior analysis device for anti-theft of mobile devices in an exhibition hall, characterized in that, It includes: A multi-modal sensor data acquisition module for collecting Bluetooth signal strength data, acceleration, angular velocity data, geomagnetic sensor data, and ambient light sensor data. A multi-modal sensor data preprocessing module for smoothing the Bluetooth signal strength data by Kalman filtering for the collected Bluetooth signal strength data; preprocessing the collected acceleration and angular velocity data using a zero-phase filter; preprocessing the collected geomagnetic sensor data using a third-order Savitzky-Golay filter; preprocessing the collected ambient light sensor data using an exponentially weighted moving average filter. A multi-modal data time alignment and fusion module for mapping the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data to the same timestamp with the sampling rate of the preprocessed acceleration and angular velocity data as the reference time. A user behavior recognition module for defining user behaviors through the acceleration, angular velocity data, the geomagnetic sensor data, and the ambient light sensor data. A user behavior database construction module for constructing a user behavior database based on the defined user behaviors. A hybrid recognition model construction and training module for constructing a CNN-LSTM hybrid recognition model, where the CNN-LSTM hybrid recognition model includes CNN module parameters, LSTM module parameters, attention mechanism parameters, a fully connected layer, and an output layer; training the constructed CNN-LSTM hybrid recognition model according to the user behavior database, and obtaining user behavior recognition results using the real-time collected multi-modal sensor data through the trained CNN-LSTM hybrid recognition model. A user behavior - Bluetooth signal strength fusion judgment module for calculating a comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance; determining whether the comprehensive score exceeds a preset threshold, and triggering an anti-theft alarm if the comprehensive score exceeds the preset threshold. In the user behavior - Bluetooth signal strength fusion judgment module: The formula for calculating the comprehensive score by assigning weights to the user behavior recognition results and the Bluetooth signal strength distance is: S(t) = w1·f behavior (y pred ) + w2·f distance (d bt (t)) where f behavior is the risk score corresponding to the user behavior recognition result y pred ; f distance is the risk score corresponding to the Bluetooth signal strength distance d bt (t); w1 and w2 are the weights of the behavior recognition result y pred and the Bluetooth signal strength distance d bt (t). In the user behavior - Bluetooth signal strength fusion judgment module, the risk scores are divided into behavior recognition risk scores and Bluetooth signal strength distance risk scores according to the specified behavior categories and Bluetooth signal strength distances.

7. The user behavior analysis device for anti-theft of exhibition hall mobile devices according to claim 6, wherein In the multi-modal sensor data preprocessing module, estimate the true value of the current Bluetooth signal strength data through prediction steps and update steps. In the prediction step, predict the value of the Bluetooth signal strength data at the current moment and the error covariance P k|k-1 : P k|k-1 = A·P k-1|k-1 ·A T + Q Where A is the state transition matrix; Q is the process noise covariance. In the update step, calculate the Kalman gain K k , and based on the Kalman gain K k update the value of the Bluetooth signal strength data and the error covariance P k|k-1 : P k|k = (1 - K k ) · P k|k-1 where z k is the currently measured Bluetooth signal strength data value, and R is the observation noise covariance; is the filtered Bluetooth signal strength data value at the current moment t; P k|k is the error covariance of the filtered Bluetooth signal strength data at the current moment t; In the multi-modal sensor data preprocessing module, the formula for distance estimation using the Bluetooth signal strength data after Kalman filtering is: where d bt (t) is the distance between the mobile device estimated by Bluetooth and the display stand; A` is the Bluetooth signal strength data value at the reference distance; n is the path loss exponent of signal propagation; In the multi-modal sensor data preprocessing module, bidirectional filtering is adopted, that is, a zero-phase filter is applied once in the forward and backward directions to cancel the phase shift; The zero-phase filter adopts a Butterworth filter or a Chebyshev filter; In the multi-modal sensor data preprocessing module, the formula for preprocessing the collected geomagnetic sensor data with a third-order Savitzky-Golay filter is: where c k is the coefficient of the filter; K is the window size, and a suitable window size is selected according to the characteristics of the signal and the application scenario; m filtered (t) is the geomagnetic sensor data after preprocessing by the third-order Savitzky-Golay filter; m(t + k) is the geomagnetic sensor data before preprocessing by the third-order Savitzky-Golay filter; In the multi-modal sensor data preprocessing module, the formula for preprocessing the collected ambient light sensor data with an exponentially weighted moving average filter is: L filtered L(t) = γ·L(t) + (1 - γ)·L filtered (t - 1) Where, L(t) is the ambient light intensity at the current moment, and L filtered (t) is the filtered light intensity at the current moment; L filtered (t - 1) is the filtered light intensity at the previous moment; γ is the smoothing coefficient of filtering; In the multi-modal data time alignment and fusion module, the preprocessed Bluetooth signal strength data, the geomagnetic sensor data, and the ambient light sensor data are mapped to the same timestamp by linear interpolation or nearest neighbor interpolation methods; In the user behavior database construction module, the input data of the user behavior database is multi-modal sensor data within a time window; The output data of the user behavior database represents the category of user behavior; The data X within each time window and the corresponding behavior label y together form a sample, which is added to the dataset ; the dataset size N depends on the acquisition duration and the window sliding setting; the dataset is in the form of: where X (i) is the multi-modal sensor data of the i-th sample, with a dimension of T w ×d; y (i) is the behavior label of the i-th sample.

8. The user behavior analysis device for anti-theft of exhibition hall mobile devices according to claim 6, characterized in that, It further includes a user usage behavior preference analysis module, which is used to record and store user behavior data, identify and store the interaction behavior of each user with the displayed mobile device as a behavior sequence, and the behavior sequence includes the behavior category, the timestamp when the behavior occurs, the behavior duration, and the location where the behavior occurs.

Images