Method and system for identifying photographing behavior through operation record

Abstract

The invention discloses a method and a system for identifying a photographing behavior through an operation record. The method comprises the following steps: acquiring a file operation record of a user in a period of time; after the file operation records are sorted according to time points, preprocessing is carried out to remove noise data, and a plurality of events to be judged are obtained after processing; mouse and keyboard operation records in the time interval corresponding to the events to be judged are captured; and analyzing whether the mouse and keyboard operation records have periodicity in time or not, and if yes, judging that the user has a suspected photographing behavior. Whether the user has the behavior or not is predicted through the operation log of the user on the terminal, and compared with a traditional method, the performance overhead can be greatly reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data leakage prevention, and in particular to a method and system for identifying photographing behavior through operation records. Background Art

[0002] Data leakage prevention technology usually involves installing a monitoring program on the user's terminal. The program performs a series of checks and restrictions on the user's operations to prevent data leakage. For example, it prohibits users from copying files to external devices, prohibits users' private devices from accessing the corporate network, prohibits outgoing emails, etc. These restrictions are easier to implement from a software perspective.

[0003] However, with the popularity of smartphones equipped with cameras, it is difficult for companies to require employees not to bring smartphones into the office area or to completely control employees from taking photos. Checking employees' personal mobile phones is itself an invasion of privacy. Therefore, if employees use their mobile phones to take photos of computer screens to obtain corporate data, it is relatively complicated to detect this behavior through technical means.

[0004] There are also some methods to detect whether the user has taken a photo of the computer screen. The usual method is to equip each terminal with a camera. The camera will take photos periodically, and the system will analyze the photos to detect whether there is any photo-taking behavior in the photos.

[0005] Using image recognition to detect the behavior of taking pictures of the screen to obtain data first requires that the terminal device must be equipped with a camera, and it will greatly increase the performance overhead compared to just processing logs. In addition, the recognition false alarm rate for behaviors such as taking pictures that do not have particularly typical characteristics is relatively high, which means that the accuracy rate cannot be significantly improved under the condition of high performance consumption. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to address the deficiencies in the prior art and provide a method for identifying photo-taking behavior through operation records. The method predicts whether the user has such behavior through the user's operation log on the terminal, which will greatly reduce performance overhead compared to traditional methods.

[0007] Technical solution: The method for identifying photographing behavior through operation records of the present invention comprises the following steps:

[0008] S1: Obtain the user's file operation records within a period of time;

[0009] S2: After sorting the file operation records according to time points, preprocessing is performed to remove noise data, and several events to be determined are obtained after processing;

[0010] S3: Capturing the mouse and keyboard operation records within the time interval corresponding to the above-mentioned events to be determined;

[0011] S4: Analyze whether the mouse and keyboard operation records are periodic in time. If they are periodic, it is determined that the user has suspected photo-taking behavior.

[0012] To further improve the above technical solution, the file operation record includes the file type, operation event type, and time point.

[0013] Furthermore, the preprocessing includes merging the file operation records that overlap in time and discarding the file operation records whose operation time is less than a set time.

[0014] Furthermore, each of the events to be determined includes an event number, a file path, an opening timestamp, a closing timestamp, and an opening duration.

[0015] Furthermore, the mouse and keyboard operation records include mouse operation events and keyboard operation events. The mouse operation events include mouse operation type, mouse operation time point, and screen coordinates of the operation point. The keyboard operation events include keyboard operation type and keyboard operation time point.

[0016] Furthermore, the S4 adopts an autocorrelation function type algorithm or a periodogram type algorithm to perform periodic analysis; the autocorrelation function type algorithm includes: converting the mouse and keyboard operation records into a time series, calculating the autocorrelation functions of the mouse and keyboard time series respectively, drawing the corresponding autocorrelation function graph, observing the autocorrelation function graph, and judging whether there is a periodic peak within a limited time, and if so, determining that it has periodicity; the periodogram type algorithm includes: converting the mouse and keyboard operation records into a time series, performing discrete Fourier transform on the mouse and keyboard time series respectively, calculating the power spectral density, drawing the corresponding periodogram, observing the energy distribution at a specific frequency, and judging whether there is a periodic peak within a limited time, and if so, determining that it has periodicity.

[0017] Furthermore, in the S4, the autocorrelation function graph of the keyboard is observed, and if the keyboard operation event has an obvious peak within 1 second to 10 seconds, it is determined that it has periodicity. At the same time, the autocorrelation function graph of the mouse is observed, and if the mouse operation event has an obvious peak within 1 second to 10 seconds and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior; in the S4, the period graph of the keyboard is observed, and if the keyboard operation event has an obvious peak within 1 second to 10 seconds, it is determined that it has periodicity. At the same time, the period graph of the mouse is observed, and if the mouse operation event has an obvious peak within 1 second to 10 seconds and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior.

[0018] Furthermore, the method further includes S5: obtaining files corresponding to suspected photo-taking behaviors, and pushing the files to manual analysis.

[0019] The system for implementing the above-mentioned method of identifying photographing behavior through operation records includes:

[0020] The file operation record acquisition module is used to obtain the user's file operation records within a specified time;

[0021] A file operation record processing module, used to remove noise data and obtain a number of events to be determined, wherein the noise data removal includes merging file operation records with overlapping time and discarding file operation records with operation time less than a set time, and each of the events to be determined includes an event number, a file path, an opening timestamp, a closing timestamp, and an opening duration;

[0022] A mouse and keyboard operation record capture module is used to obtain the mouse and keyboard operation records within the time interval corresponding to the event to be determined;

[0023] The analysis module is used to analyze whether the mouse and keyboard operation records have periodicity. If so, it is determined that the user has suspected photo-taking behavior.

[0024] Furthermore, the analysis module uses an autocorrelation function type algorithm or a periodogram type algorithm to perform periodic analysis; the autocorrelation function type algorithm includes: converting the mouse and keyboard operation records into time series, calculating the autocorrelation functions of the mouse and keyboard time series respectively, drawing the corresponding autocorrelation function graph, observing the autocorrelation function graph of the keyboard, if the keyboard operation event has an obvious peak within the limited time, it is determined to have periodicity, and at the same time observing the autocorrelation function graph of the mouse, if the mouse operation event has an obvious peak within the limited time and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior; the periodogram type algorithm includes: converting the mouse and keyboard operation records into time series, performing discrete Fourier transform on the mouse and keyboard time series respectively, calculating the power spectral density, drawing the corresponding periodogram, observing the periodogram of the keyboard, if the keyboard operation event has an obvious peak within the limited time, it is determined to have periodicity, and at the same time observing the periodogram of the mouse, if the mouse operation event has an obvious peak within the limited time and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior.

[0025] Beneficial effects: Compared with the prior art, the advantages of the present invention are: the present invention accurately locates the user's file operation behavior by acquiring and processing file operation records, combining time and event type; and comprehensively analyzes the user's specific operations and identifies the possibility of photo-taking behavior by capturing mouse and keyboard operation records.

[0026] Noise removal and record merging in the preprocessing step ensure the accuracy and validity of the data and improve the precision of the analysis; the autocorrelation function and periodogram algorithm can effectively identify the periodic features in the operation records and ensure the reliability of the recognition results; periodic judgment under specific conditions (such as obvious peaks and mouse displacements within the time window) further improves the accuracy of the judgment; the analysis results of multiple operation records are combined to provide a reliable preliminary judgment; suspected photo-taking behaviors are manually reviewed to ensure the accuracy and reliability of the final judgment.

[0027] The present invention effectively improves the accuracy and reliability of user photo-taking behavior recognition through multi-step, multi-dimensional data collection and analysis combined with periodic feature recognition. At the same time, the pre-processing and noise removal steps improve the validity of the data, and the comprehensive judgment and manual review ensure the accuracy of the final judgment, forming a complete recognition and verification system.

[0028] Compared with the traditional detection method of image recognition, the present invention predicts whether the user has such behavior through the user's operation log on the terminal, which can greatly reduce the performance overhead compared with the traditional method. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0031] Example 1: Principle explanation: If a user wants to obtain data by taking photos, he usually has a fixed operation mode: first open the file to be photographed, then take a photo with the mobile phone, then turn to the next page and continue taking photos, and repeat this process. By analyzing the operation records and finding the operation track that conforms to this mode, it can be considered that the user is suspected of taking photos to obtain data.

[0032] like Figure 1 The method for identifying the photo-taking behavior through operation records shown in the figure has the following specific steps:

[0033] S1: Obtain the user's file operation records within a period of time;

[0034] S2: After sorting the file operation records according to time points, preprocessing is performed to remove noise data, and several events to be determined are obtained after processing;

[0035] S3: Capturing the mouse and keyboard operation records within the time interval corresponding to the above-mentioned events to be determined;

[0036] S4: analyzing whether the mouse and keyboard operation records have periodicity in time. If so, it is determined that the user has suspected photo-taking behavior;

[0037] S5: Obtain files corresponding to suspected photo-taking behaviors, and push the files to manual analysis.

[0038] Specifically, the file operation records in S2 include file type, operation event type, and time point; preprocessing includes merging file operation records that overlap in time and discarding file operation records whose operation time is less than the set time; each event to be determined includes event number, file path, open timestamp, close timestamp, and open duration.

[0039] Specifically, the mouse and keyboard operation records in S3 include mouse operation events and keyboard operation events. The mouse operation events include the mouse operation type, the mouse operation time point, and the screen coordinates of the operation point, among which the mouse operation type is a left-button click event. The keyboard operation events include the keyboard operation type and the keyboard operation time point, among which the keyboard operation type is a page turning operation.

[0040] Specifically, S4 uses an autocorrelation function type algorithm or a periodogram type algorithm to perform periodicity analysis; the autocorrelation function type algorithm includes: converting the mouse and keyboard operation records into a time series, calculating the autocorrelation functions of the mouse and keyboard time series respectively, drawing the corresponding autocorrelation function graph, observing the autocorrelation function graph of the keyboard, if the keyboard operation event has an obvious peak within the limited time, it is determined to have periodicity, and at the same time observing the autocorrelation function graph of the mouse, if the mouse operation event has an obvious peak within the limited time and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior; the periodogram type algorithm includes: converting the mouse and keyboard operation records into a time series, performing discrete Fourier transform on the mouse and keyboard time series respectively, calculating the power spectral density, drawing the corresponding periodogram, observing the periodogram of the keyboard, if the keyboard operation event has an obvious peak within the limited time, it is determined to have periodicity, and at the same time observing the periodogram of the mouse, if the mouse operation event has an obvious peak within the limited time and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior.

[0041] Embodiment 2: A system for implementing the method in Embodiment 1, comprising:

[0042] The file operation record acquisition module is used to obtain the user's file operation records within a specified time;

[0043] A file operation record processing module, used to remove noise data and obtain a number of events to be determined, wherein the noise data removal includes merging file operation records with overlapping time and discarding file operation records with operation time less than a set time, and each of the events to be determined includes an event number, a file path, an opening timestamp, a closing timestamp, and an opening duration;

[0044] A mouse and keyboard operation record capture module is used to obtain the mouse and keyboard operation records within the time interval corresponding to the event to be determined;

[0045] The analysis module is used to analyze whether the mouse and keyboard operation records have periodicity. If so, it is determined that the user has suspected photo-taking behavior.

[0046] Example 3: A specific description of the photo-taking behavior using the method in Example 1 is provided to capture the operation records of users opening and closing text files on the terminal within a period of time (e.g., 1 day). The file types usually concerned are word, excel, ppt, pdf, etc. The time points of opening and closing of each file are recorded separately. The collection process and method are not within the scope of the present invention.

[0047] The time points of opening and closing the above files are sorted in the order of present and latter, and the operation records with overlapping time are merged. The records with short opening time (for example, less than 30 seconds) can be directly discarded (it is considered unlikely that there is photo taking behavior), and finally multiple determined time ranges are generated. The examples generated by the above three steps are as follows:

[0048]

[0049] Capture the mouse and keyboard operation records within the above multiple time ranges respectively. For mouse operation, record: the time point of the left-click event and the screen coordinates of the left-click event. For keyboard operation, record: the time point of page-turning related operations (such as pageup, pagedown, up and down, ctrl+up and down, etc.). The example is as follows:

[0050] Occurrence timestamp Associated event number Operation Type Operational Events 1695104454500 1 mouse Left-click stand-alone (1978, 997) 1695104453400 1 mouse Left click stand-alone (1964,990) 1695104336000 3 keyboard PageDown 1695104456040 3 keyboard PageDown

[0051] Analyze the above mouse events and keyboard events separately to see if they are periodic in time. The limit period should be between 1 second and 10 seconds. For mouse events, additional analysis is required to check if the coordinates of each click do not have a large displacement (it is recommended that the displacement does not exceed 30 pixels). There are many methods for periodicity detection, for example:

[0052] 1. Autocorrelation function (ACF) algorithm

[0053] Autocorrelation, also called serial correlation, is the correlation between a signal and itself at different time points, that is, the similarity between two observations as a function of the time difference between them. It is a mathematical tool for finding repetitive patterns (such as periodic signals masked by noise) or identifying the fundamental frequency that disappears in the harmonic frequency of the signal. It is often used in signal processing to analyze functions or a series of values, such as time domain signals. The autocorrelation function ACF(k) measures the correlation between the same event in two different periods. Figuratively speaking, it measures the impact of one's past behavior on one's present.

[0054] In this embodiment: the above-mentioned mouse and keyboard operation records are sorted according to the time series, and for each time series, Z t With Z t-k The autocorrelation coefficient ACF(k) with an interval of k is mathematically expressed as:

[0055]

[0056] in, is the time series Z t The mean of .

[0057] Plot the autocorrelation function graph and check how the autocorrelation coefficient changes with time lag k. If there is an obvious peak in the autocorrelation coefficient at a certain time lag kk and the periodicity is obvious (the period should be between 1 second and 10 seconds), it indicates that there is periodic behavior.

[0058] 2. Periodogram Algorithms

[0059] According to Fourier theory, any signal in the time domain can be composed of one or more sine waves with appropriate frequency, amplitude and phase. In other words, any time domain signal can be transformed into a corresponding frequency domain signal, and the energy value of the signal at a specific frequency can be obtained through frequency domain measurement. Spectral analysis is a method of analyzing time series in the frequency domain. It uses Fourier analysis to convert time domain signals to frequency domain and find out the changing rules of signal spectrum from the frequency domain.

[0060] Then the periodogram method is a signal power spectrum density estimation method. The basic periodogram method can improve the computational efficiency and does not require the calculation of the autocorrelation function, but the spectral resolution is low. The basic principle of the basic periodogram is to directly perform Fourier transform on the observed data, and then take the square of the modulus to obtain the power spectrum.

[0061] In this embodiment: the above-mentioned mouse and keyboard operation records are sorted according to the time series. For each time series, a finite number of observation points x(0), x(1), ...x(n-1) of the stationary random signal x(n) are taken, and the Fourier transform of X(n) is:

[0062]

[0063] Since the discrete Fourier transform X(e jw ) is periodic, so this power spectrum It is also periodic, called a periodogram:

[0064]

[0065] The periodogram is a biased estimate of the signal power spectrum. It looks at how the power spectrum density changes with frequency. If the power spectrum density has an obvious peak at a certain frequency and the period is between 1 second and 10 seconds, it indicates periodic behavior. It is believed that the user is suspected of taking photos to obtain data. Further information (such as file name, file content, whether the behavior occurs multiple times, etc.) is needed to manually confirm whether to include him / her in the target object of information security investigation and audit.

[0066] The present invention does not require image analysis and can predict whether the user has taken photos through log records. It does not rely on the camera and can greatly reduce resource overhead, but the success rate of recognition will not be significantly reduced.

[0067] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for identifying photographing behavior through operation records, characterized in that: The steps include: S1: Obtain the user's file operation records within a period of time; S2: After sorting the file operation records according to time points, preprocessing is performed to remove noise data, and several events to be determined are obtained after processing; S3: Capturing the mouse and keyboard operation records within the time interval corresponding to the above-mentioned events to be determined; S4: Analyze whether the mouse and keyboard operation records are periodic in time. If they are periodic, it is determined that the user has suspected photo-taking behavior.

2. The method for identifying photographing behavior through operation records according to claim 1, characterized in that: The file operation record includes the file type, operation event type, and time point.

3. The method for identifying photographing behavior through operation records according to claim 2, characterized in that: The preprocessing includes merging the file operation records that overlap in time and discarding the file operation records whose operation time is less than the set time.

4. The method for identifying photographing behavior through operation records according to claim 1, characterized in that: Each of the events to be determined includes an event number, a file path, an opening timestamp, a closing timestamp, and an opening duration.

5. The method for identifying photographing behavior through operation records according to claim 1, characterized in that: The mouse and keyboard operation records include mouse operation events and keyboard operation events. The mouse operation events include mouse operation type, mouse operation time point, and screen coordinates of the operation point. The keyboard operation events include keyboard operation type and keyboard operation time point.

6. The method for identifying photographing behavior through operation records according to claim 1, characterized in that: The S4 uses an autocorrelation function type algorithm or a periodogram type algorithm to perform periodicity analysis; The autocorrelation function type algorithm includes: converting the mouse and keyboard operation records into time series, calculating the autocorrelation functions of the mouse and keyboard time series respectively, drawing the corresponding autocorrelation function graphs, observing the autocorrelation function graphs, judging whether there is a periodic peak within a limited time, and if so, determining that it has periodicity; The periodogram algorithm includes: converting mouse and keyboard operation records into time series, performing discrete Fourier transform on the mouse and keyboard time series respectively, calculating power spectrum density, drawing corresponding periodograms, observing energy distribution at specific frequencies, and judging whether there is a periodic peak within a limited time. If so, it is determined to have periodicity.

7. The method for identifying photographing behavior through operation records according to claim 6, characterized in that: In S4, the autocorrelation function graph of the keyboard is observed. If the keyboard operation event has an obvious peak value within 1 second to 10 seconds, it is determined that there is periodicity. At the same time, the autocorrelation function graph of the mouse is observed. If the mouse operation event has an obvious peak value within 1 second to 10 seconds and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is a suspected photo-taking behavior; In S4, the keyboard cycle diagram is observed. If the keyboard operation event has an obvious peak value within 1 second to 10 seconds, it is determined to have periodicity. At the same time, the mouse cycle diagram is observed. If the mouse operation event has an obvious peak value within 1 second to 10 seconds and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior.

8. The method for identifying photographing behavior through operation records according to claim 1, characterized in that: It also includes S5: obtaining files corresponding to suspected photo-taking behaviors, and pushing the files to manual analysis.

9. A system for implementing the method of identifying photographing behavior through operation records as described in claim 1, characterized in that: include: The file operation record acquisition module is used to obtain the user's file operation records within a specified time; A file operation record processing module, used to remove noise data and obtain a number of events to be determined, wherein the noise data removal includes merging file operation records with overlapping time and discarding file operation records with operation time less than a set time, and each of the events to be determined includes an event number, a file path, an opening timestamp, a closing timestamp, and an opening duration; A mouse and keyboard operation record capture module is used to obtain the mouse and keyboard operation records within the time interval corresponding to the event to be determined; The analysis module is used to analyze whether the mouse and keyboard operation records have periodicity. If so, it is determined that the user has suspected photo-taking behavior.

10. The system according to claim 9, characterized in that: The analysis module uses an autocorrelation function type algorithm or a periodogram type algorithm to perform periodicity analysis; The autocorrelation function type algorithm includes: converting the mouse and keyboard operation records into time series, calculating the autocorrelation functions of the mouse and keyboard time series respectively, drawing corresponding autocorrelation function graphs, observing the keyboard autocorrelation function graph, and if the keyboard operation event has an obvious peak within a limited time, it is determined that it has periodicity, and observing the mouse autocorrelation function graph at the same time, if the mouse operation event has an obvious peak within a limited time and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is a suspected photo-taking behavior; The periodogram algorithm includes: converting mouse and keyboard operation records into time series, performing discrete Fourier transform on the mouse and keyboard time series respectively, calculating power spectral density, drawing corresponding periodograms, observing the keyboard periodogram, and if the keyboard operation event has an obvious peak within a limited time, it is determined to have periodicity. At the same time, observing the mouse periodogram, if the mouse operation event has an obvious peak within a limited time and the screen coordinate displacement of the operation point is less than 30 pixels, it is determined that there is suspected photo-taking behavior.

Images