A touch-aware eavesdropping detection method for mobile applications

Abstract

The present invention discloses a method and system for detecting eavesdropping of mobile applications based on touch perception. The method comprises: first, conducting a detailed study on the normal working logic of the user-authorized mobile application to access the microphone, and finding that normal microphone access is related to the screen focus; second, in order to characterize the microphone access mode of the relationship between microphone access and screen focus, the user touching the voice icon is regarded as a request event, and the application occupying the microphone is regarded as a response event, and a request-response time model is established; finally, numerical analysis proves that the request response time of each application under user authorization obeys a normal distribution. By comparing the models, we can accurately determine whether the current microphone access belongs to the user's normal operation of the application or the application's illegal eavesdropping behavior. This solves the problem that malicious eavesdropping of mobile applications is difficult to detect, and protects the safety and privacy of users.

Description

Technical Field

[0001] The present invention relates to the field of mobile device security, and in particular to a touch-sensing-based eavesdropping detection method and system for mobile applications. Background Art

[0002] Millions of mobile applications have been developed and installed on mobile devices such as smartphones, tablets, and wearables, providing users with a wealth of functionality. To enhance market competitiveness, many applications, such as those from Baidu and Google, offer voice-based human-computer interfaces (HMIs) to facilitate user-application interaction. Once users trigger an application-specific voice button and authorize the application to access the microphone, they can conveniently operate the voice application and execute corresponding functions through voice perception.

[0003] While voice apps have created new operational capabilities, there are growing concerns about whether these apps are secretly recording conversations without user authorization, a practice known as unauthorized eavesdropping. Mobile devices can be transformed into microphones for eavesdropping on surrounding conversations. Malware activates their microphones, continuously recording surrounding sounds. Recently, there have been reports of users talking within earshot of their mobile devices and subsequently being recommended for advertisements in apps. This can lead users to suspect that they are being secretly eavesdropped on by apps.

[0004] To explore the eavesdropping risks of applications, researchers have proposed various schemes.

[0005] Since eavesdropping attacks may occur because malicious applications contain hidden behaviors that have not been disclosed, such as backdoors, many researchers have proposed to determine whether an application is eavesdropping by checking for backdoors. The author proposed identifying malicious code or bugs in the application source code. However, in the face of applications protected by security reinforcement, this simple method is not feasible. Zhao et al. proposed an input verification behavior tool INPUTSCOPE to detect hidden functions in applications. TriggerScop and IntelliDroid use symbolic execution to generate external inputs for malware detection. ICCDetector uses feature-based algorithms to detect hidden malicious behaviors in Android applications. Unfortunately, unlike hidden functions, eavesdropping is likely to be masked by normal behavior, and these tools will show low efficiency in this case. More importantly, eavesdropping may be caused by the developer's incorrect logic, which is more difficult to detect through backdoor analysis.

[0006] While the authors claim they can effectively detect eavesdropping, the problem they address is that physical signals can be eavesdropped on due to interactions between sensors. For example, eavesdropping on the accelerometer and gyroscope while tapping the screen can effectively recover the user's typing. However, these solutions are ineffective when it comes to eavesdropping on an application using the microphone.

[0007] There are also many solutions designed to detect eavesdropping on smart home devices. These exploit the fact that smart home devices need to transmit voice signals to a server in a timely manner and detect eavesdropping attacks by analyzing network traffic. Mitevet.al proposed a LeakyPick architecture that periodically "probes" the environment and monitors subsequent network traffic statistical patterns, indicating the transmission of audio content, making eavesdropping detection possible. However, these methods are only applicable to smart home devices with a single traffic pattern. They are no longer effective for mobile devices with complex network traffic, especially those with multiple voice-enabled applications installed. Summary of the Invention

[0008] The present invention provides a touch-aware eavesdropping detection method for mobile applications, which is used to overcome the challenge of difficult-to-detect eavesdropping behaviors that leak user security and privacy in mobile applications, and to implement a lightweight and effective eavesdropping detection solution.

[0009] To achieve the above purpose, the technical solution of the present invention is as follows:

[0010] A touch-aware eavesdropping detection method for a mobile application comprises the following steps:

[0011] Step 1: Obtain the response time range of each mobile application for activating the microphone by touching the screen, which is used as the normal response time range of each mobile application, and record the time when the user last touches the screen on the mobile device in real time;

[0012] Step 2: When the microphone is detected to be in use, the package name of the application accessing the microphone and the package name of the application in the current focus window are detected; if the package name of the application accessing the microphone is inconsistent with the package name of the application in the current focus window, it is determined that microphone eavesdropping has occurred, otherwise proceed to the next step;

[0013] Step 3: Detect the time T1 when the microphone is occupied and the time T2 when the user last touches the screen on the mobile device before the microphone is occupied, and obtain the request response time RRT current =T1-T2; if RRT current If the response time is within the normal range of the application program accessing the microphone, it is determined that the microphone is started normally; otherwise, it is determined that microphone eavesdropping has occurred.

[0014] A further improvement is to issue an alarm when it is determined that microphone eavesdropping has occurred.

[0015] As a further improvement, the alarm method includes activating a buzzer and displaying a prompt floating window on a display screen.

[0016] As a further improvement, the display prompt floating window is provided in a semi-transparent and non-clickable manner, and the prompt floating window is located in the center of the screen when displayed.

[0017] As a further improvement, the floating window will display the microphone status and the microphone calling program to the user in real time.

[0018] As a further improvement, in step 1, the method for determining the normal response time range of each mobile application is as follows:

[0019] Multiple experiments are conducted to obtain multiple response times of normal responses of the mobile application. If the multiple response times conform to the normal distribution, the mean μ and variance σ of the corresponding normal distribution are calculated, and the normal response time range is (μ-3σ, μ+3σ).

[0020] A touch-sensing-based eavesdropping detection system for mobile applications is characterized by comprising a memory and a processor, wherein the memory stores an eavesdropping detection program of the application, and when the processor runs the eavesdropping detection program, the steps of the above method are executed.

[0021] The present invention provides a touch-aware eavesdropping detection method and system for mobile applications. By exploring the normal mode of microphone occupation by enabling the voice function of an application, it is found that normal microphone access is related to screen focus. In order to characterize the microphone access mode in terms of the relationship between microphone access and screen focus, a request-response time model is established by taking the user touching the voice icon as a request event and the application occupying the microphone as a response event. Numerical analysis proves that the request response time of each application under user authorization follows a normal distribution. Through model comparison, we can accurately determine whether the current microphone access is a normal operation of the user on the application or an illegal eavesdropping behavior of the application. This solves the problem of difficult detection of malicious eavesdropping by mobile applications and protects the safety and privacy of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0023] Figure 1 Diagram of the eavesdropping detection framework provided by an embodiment of the present invention;

[0024] Figure 2a Request-response time distribution graph for 8 normal applications under user authorization;

[0025] Figure 2b This is a graph showing the authorization and request-response time distribution for native attacks.

[0026] Figure 2c This is a diagram showing the authorization and request-response time distribution for injection attacks.

[0027] Figure 3 Probability distribution of response time for user request to authorize microphone occupation;

[0028] Figure 4a Native attack eavesdropping detection under different resource utilization on Xiaomi 10;

[0029] Figure 4b Injection attack eavesdropping detection under different resource utilization on Xiaomi 10

[0030] Figure 5a For native attack eavesdropping detection under different resource utilization on Redmi K30;

[0031] Figure 5b Injection attack eavesdropping detection under different resource utilization on Redmi K30

[0032] Figure 6a Detect native eavesdropping attacks in different network environments on Xiaomi 10;

[0033] Figure 6b Detect injection attacks and eavesdropping in different network environments on Xiaomi 10;

[0034] Figure 7a For native attack eavesdropping detection in different network environments on Redmi K30;

[0035] Figure 7b Injection attack eavesdropping detection in different network environments on Redmi K30.

[0036] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0038] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the various components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0039] In addition, the terms "first," "second," and so on, used in this disclosure are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referenced. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this disclosure, "plurality" means at least two, such as two or three, unless otherwise specifically defined.

[0040] In the present invention, unless otherwise specified or limited, the terms "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can mean fixed connection, detachable connection, or integration; it can mean mechanical connection, electrical connection, physical connection, or wireless communication connection; it can mean direct connection or indirect connection through an intermediate medium; it can mean internal communication between two elements or interaction between two elements, unless otherwise specified. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0041] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in this field can implement it. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0042] Example 1

[0043] As attached Figure 1-7b As shown, an embodiment of the present invention provides a method for detecting eavesdropping of a mobile application based on touch perception, which is characterized by comprising the following steps:

[0044] A touch-aware eavesdropping detection method for a mobile application comprises the following steps:

[0045] Step 1: We conducted a detailed study of the normal working logic of users authorizing mobile applications to access the microphone. We found that normal microphone access is related to screen focus.

[0046] Step 2: To characterize the microphone access pattern in relation to screen focus, a request-response time model was established, taking the user touching the voice icon as a request event and the application occupying the microphone as a response event.

[0047] Step 3: Through numerical analysis, it is proved that the request response time of each application under user authorization follows a normal distribution;

[0048] Step 4: Through model comparison, we can accurately determine whether the current microphone access is a normal operation of the user on the application or an illegal eavesdropping behavior of the application.

[0049] Most mobile apps accessing the microphone in step 1 require users to tap a specific button for authorization, such as clicking the microphone icon or long-pressing the "Hold to Speak" button. Therefore, when an app is using the microphone, any user tapping the screen to authorize access is considered normal; otherwise, it is considered eavesdropping.

[0050] There are three steps for an application to access the microphone: requesting an API, calling a library function, and connecting to the microphone hardware. Therefore, to gain access to the microphone, we need to hook the API functions requested during microphone access. By installing EdXposed on the system, we can hook all Java processes on the Android system, allowing us to obtain microphone access and current screen window focus information in a timely manner.

[0051] The AudioService process and WindowManager process are used to manage audio access and screen windows respectively. They are also interfaces connecting upper-level applications and low-level library functions.

[0052] In the AudioService process, EdXposed can intercept calls to the updateSapshot() function in the RecordingActivityMonitor class through code insertion. This function calls the RecordingEvent class constructor whenever the microphone status is updated. This function can then obtain information about the application accessing the microphone, including mClientPackageName, mClientSessionId, and mClientUid.

[0053] For the WindowManager process, by injecting code into the updateFocusedWindowLocked() function of the DisplayContent class, we can obtain the package name of the application running in the foreground. When the focused window changes, the return value of the updateFocusedWindowLocked() function is true, thus obtaining the application package name of the currently focused window, that is, the package name of the foreground application.

[0054] Based on the package name of the application accessing the microphone and the package name of the application that obtains the currently focused window, we can make a preliminary eavesdropping judgment. If the two package names are inconsistent, meaning the app is in the background when accessing the microphone, it is considered eavesdropping. If the two package names are consistent, meaning the app is in the foreground when accessing the microphone, it is difficult to determine whether this is normal user behavior or an eavesdropping attack. To address this issue, we developed a request-response time model for eavesdropping detection.

[0055] Step 2 is as follows: We consider touching the voice icon as a request and the microphone being engaged as a response. By determining whether a response occurs alongside normal request behavior, we can identify eavesdropping. While this approach appears simple, technical challenges remain. Smartphones are touchscreen devices, and identifying the user's touch of the voice icon from multiple screen operations is a difficult task.

[0056] To address this challenge and achieve effective eavesdropping detection, we analyzed several voice applications. First, by analyzing the developer documentation Android and the application source code, we found that the design principle of the application's voice function is to respond immediately after the user request. Second, we randomly selected 8 voice applications for tracking analysis. We used the getevent() function to capture the user's actions on the screen and recorded the Unix timestamps of the request and response. The results showed that touching the voice icon was the last time the finger appeared on the screen before the microphone was occupied. Based on this key result, we used the time of the last touch on the screen before occupying the microphone as the request event, that is, the action of touching the voice icon.

[0057] Step 3 is as follows:

[0058] To explore correlations between request-response times for the same application, we performed 100 operations on eight voice applications. We also explored the request-response times for exploitation and injection attacks. We typically used features provided by two attack applications, with actions including tapping, holding, and swiping. We found that while request-response times for the same application were often consistent, eavesdropping request response times were irregular and chaotic.

[0059] On this basis, we use the normal probability density function method to further study the request response time. The normal probability density estimation based on the request response time is

[0060]

[0061] where {y1, ..., y n} is the collected request response time, w is the kernel (symmetric probability density function), and h>0 is a smoothing parameter (ie, bandwidth).

[0062] Based on the large amount of request response time data we collected, we performed probability density estimation for different numbers of speech samples. We found that the request response time of each application seems to follow a normal distribution. To further verify this normal distribution, we used the Kolmogorov-Smirnov (KS) test, which is a useful non-parametric hypothesis test. Assume that the collected RRT dataset is T = {t, ..., t n}, we first calculate the cumulative distribution function (CDF)

[0063] F CDF (t i )=P(T≤t)

[0064] Where n is the number of RRTs in T, i≤n. Next, we sort the RRTs in T and calculate the empirical distribution function (EDF)

[0065]

[0066] Again, we propose the hypothesis test H0 and the alternative hypothesis H1

[0067] H0:F CDF (t) = F EDF (t)

[0068] H1:F CDF (t)≠F EDF (t)

[0069] Where H0 means RRT follows normal distribution, and H1 means RRT does not follow normal distribution. CDF (t) and F EDF (t), we calculate

[0070] D=max|F CDF (t)-F EDF (t)|

[0071] If D < D na , assuming H0 is established, where D nais the critical value for the sample size n and significance level a. We empirically set a = 0.05. The hypothesis test results show that the RRTs of the eight randomly selected normal applications and the normal behavior of the attacking application all follow independent normal distributions.

[0072] Eavesdropping detection is performed based on the microphone status and whether the user touches the microphone icon. The triggering of the microphone icon is modeled as a request event P, and the microphone being occupied is regarded as a response event Q. We consider the currently used microphone to be eavesdropping. If P∧Q, we detect eavesdropping based on the constructed RRT model.

[0073] If the microphone is occupied by app1, but it is not known whether it is a normal request, based on the time the captured application occupied the microphone t1 and the time the screen was last touched t2, we can get the current request response time RRT current =|T1-T2|, if RRT current If the RRT distribution belongs to the verified app1, it is considered normal behavior; otherwise, it is considered eavesdropping. This decision logic can be formally expressed as follows:

[0074]

[0075] We believe that the behavior of occupying the microphone is caused by eavesdropping. N is the normal distribution of app1 that we verified earlier, μ and σ 2 are the mean and variance of the normal distribution respectively.

[0076] This solution provides a touch-aware eavesdropping detection method for mobile applications. By exploring the normal pattern of microphone occupation when an application enables voice functionality, it finds that normal microphone access is related to screen focus. To characterize the microphone access pattern in relation to screen focus, a request-response time model is established, using the user touching the voice icon as a request event and the application occupying the microphone as a response event. Numerical analysis demonstrates that the request-response time for each application under user authorization follows a normal distribution. By comparing the models, we can accurately determine whether the current microphone access is a normal user operation of the application or an illegal eavesdropping behavior. This solves the problem of difficult detection of malicious eavesdropping by mobile applications and protects user security and privacy.

[0077] To allow users to promptly determine whether eavesdropping is occurring, we propose an app eavesdropping detection method based on microphone status visualization. This is achieved by visually displaying the microphone status and access status to the user in real time. This visualization method uses a floating window, which allows users to understand the microphone status in real time while addressing the limited resources of the status bar. The floating window is located at the top center of the screen, is semi-transparent and non-clickable, and does not affect the user's normal use. The floating window has a level of TYPE_SYSTEM_OVERLAY and can be displayed above the lock screen or notification bar without being easily obscured. When the microphone is idle, the floating window is transparent and invisible.

[0078] Step 5: By displaying the microphone status and call status to the user in real time, the illegal calls of the microphone can be monitored to the greatest extent possible.

[0079] Example 2

[0080] Corresponding to the above-mentioned embodiment 1, the present invention also provides a mobile application eavesdropping detection system, including a memory and a processor, wherein the memory stores a network traffic anomaly detection program, and when the processor runs the network traffic anomaly detection program, the steps of any method of the above-mentioned embodiment 1 are executed.

[0081] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for detecting eavesdropping on mobile applications based on touch perception, characterized in that: The following steps are involved: Step 1: Obtain the response time range of each mobile application for activating the microphone by touching the screen, which is used as the normal response time range of each mobile application, and record the time when the user last touches the screen on the mobile device in real time; Step 2: When the microphone is detected to be in use, the package name of the application accessing the microphone and the package name of the application in the current focus window are detected; if the package name of the application accessing the microphone is inconsistent with the package name of the application in the current focus window, it is determined that microphone eavesdropping has occurred, otherwise proceed to the next step; Step 3: Detect the time T1 when the microphone is occupied and the time T2 when the user last touches the screen on the mobile device before the microphone is occupied, and obtain the request response time RRT current =T1-T2; if RRT current If the response time is within the normal range of the application program accessing the microphone, it is determined that the microphone is started normally; otherwise, it is determined that microphone eavesdropping has occurred.

2. The eavesdropping detection method for mobile applications based on touch perception according to claim 1, characterized in that: When it is determined that microphone eavesdropping has occurred, an alarm is issued.

3. The eavesdropping detection method for mobile applications based on touch perception according to claim 2, characterized in that: The alarm method includes starting a buzzer and displaying a prompt floating window on a display screen.

4. The eavesdropping detection method for mobile applications based on touch perception according to claim 3, characterized in that: The display prompt floating window is provided in a semi-transparent and non-clickable manner, and the prompt floating window is located in the center of the screen when displayed.

5. The eavesdropping detection method for mobile applications based on touch perception as claimed in claim 3, characterized in that: When prompted, the floating window displays the microphone status and the microphone calling program to the user in real time.

6. The eavesdropping detection method for mobile applications based on touch perception according to claim 1, characterized in that: In step one, the method for determining the normal response time range of each mobile application is as follows: multiple experiments are performed to obtain multiple response times of the normal response of the mobile application. If the multiple response times conform to the normal distribution, the mean μ and variance σ of the corresponding normal distribution are calculated, and the normal response time range is (μ-3σ, μ+3σ).

7. A touch-sensing mobile application eavesdropping detection system, characterized in that: The invention comprises a memory and a processor, wherein the memory stores a wiretap detection program of an application program, and when the processor runs the wiretap detection program, the steps of the method according to any one of claims 1 to 6 are executed.

Images