Data processing method and apparatus

Abstract

Embodiments of the present specification provide a data processing method and device, wherein the data processing method comprises: collecting historical access traffic data of a server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data, clustering the user identifiers of the access users according to an association relationship between the user identifiers and the data identifiers, generating clustering results corresponding to a plurality of data identifiers respectively, and performing data category labeling on the access data according to the clustering results.

Description

[0001] This application is a divisional application of application No. 202110423768.X, titled "Data processing method and device", filed on April 20, 2021. TECHNICAL FIELD

[0002] Embodiments of the present specification relate to the technical field of computer technology, and in particular to a data processing method. One or more embodiments of the present specification also relate to a data processing device, a computing device, and a computer-readable storage medium. BACKGROUND

[0003] A web application is an application based on a browser / server architecture, and is an application type generated with the development of network technology. The web application contains some static pages, and has not only an information display function, but also can execute corresponding processing operations on data by calling different business logic interfaces in the page. However, like traditional computer applications, the web application has a certain amount of vulnerabilities due to defects in security policies during development. Among them, the privilege escalation vulnerability is one of the common business logic vulnerabilities in testing the web application. The reason for its formation is that the server side trusts the data operation request of the client-side web application too much and ignores the judgment of the operation permission.

[0004] Due to the defects in the design of the web program, the guessability of the input parameters through the URL, and by changing the input parameter values, it is possible to cause horizontal privilege escalation access, resulting in the leakage of user privacy information. And the current detection method of the privilege escalation vulnerability is mainly to replace the user identity authentication information to achieve this method, which not only has low efficiency, but also has high false positive rate, and the detection result is not accurate enough. SUMMARY

[0005] Therefore, embodiments of the present specification provide a data processing method. One or more embodiments of the present specification also relate to a data processing device, a computing device, and a computer-readable storage medium to solve the technical defects in the prior art.

[0006] According to a first aspect of embodiments of the present specification, a data processing method is provided, comprising:

[0007] Collecting historical access traffic data of a server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifier being used to index the access data;

[0008] According to the association relationship between the user identifier and the data identifier, the user identifiers of the access users are clustered to generate clustering results corresponding to a plurality of data identifiers respectively;

[0009] According to the clustering result, data categories of the access data are labeled.

[0010] Optionally, the historical access traffic data of the collection server further comprises:

[0011] The historical access traffic data is split;

[0012] A hit result of the access data contained in the split result to at least one preset data screening rule is determined;

[0013] According to the hit result, the access data is screened, and the access data contained in the screening result is labeled by using a corresponding target character of the data identifier.

[0014] Optionally, the labeling of the access data according to the clustering result comprises:

[0015] A target data identifier corresponding to a user identifier in the clustering result is de-duplicated, and a quantity of target user identifiers contained in a de-duplication result is compared with a first preset quantity threshold;

[0016] If it is determined that the quantity of target user identifiers contained in the de-duplication result is less than or equal to the first preset quantity threshold, it is determined that the access data associated with the target data identifier is private data; or

[0017] If it is determined that the quantity of target user identifiers contained in the de-duplication result is greater than the first preset quantity threshold, it is determined that the access data associated with the target data identifier is public data; wherein the target data identifier is one of the plurality of data identifiers.

[0018] Optionally, the data processing method further comprises:

[0019] According to a data category labeling result, a mapping relationship table between data identifiers of the access data and data categories is established.

[0020] Optionally, the data processing method further comprises:

[0021] Access traffic data to be detected is obtained;

[0022] According to a data identifier of access data to be accessed contained in the access traffic data and the mapping relationship table, a data category of the access data to be accessed is determined.

[0023] Optionally, the data processing method further comprises:

[0024] According to the data category of the access data to be accessed, it is detected whether the server has an overreach vulnerability.

[0025] Optionally, the data category labeling of the access data according to the clustering result comprises:

[0026] The user identifiers in the clustering result corresponding to the target data identifier are de-duplicated, and the number of target user identifiers contained in the de-duplication result is determined, the target data identifier being one of the plurality of data identifiers;

[0027] The clustering result is filtered according to the number, and a target clustering result is obtained;

[0028] The data category labeling of the data identifier corresponding to the target clustering result is performed.

[0029] Optionally, the data processing method further comprises:

[0030] The access data contained in the labeling result and the user identifier associated with the access data are input into the data category labeling model to be trained for training, and the data category labeling model is obtained.

[0031] Optionally, the data processing method further comprises:

[0032] Obtain the access flow data to be detected;

[0033] The data category labeling of the access flow data contained in the access flow data is performed by inputting the data category labeling model, and the data category labeling result of the access flow data is generated.

[0034] Optionally, the data category labeling of the access data according to the clustering result comprises:

[0035] The user identifiers in the clustering result corresponding to the target data identifier are de-duplicated, and the number of target user identifiers contained in the de-duplication result is determined, the target data identifier being one of the plurality of data identifiers;

[0036] The access interface of the access data corresponding to the target data identifier is determined;

[0037] The inverse of the number of target user identifiers in the de-duplication result corresponding to different data identifiers under the access interface is averaged;

[0038] The data category labeling of the access data is performed according to the average value, wherein the target data identifier is one of the plurality of data identifiers.

[0039] According to a second aspect of the embodiments of the present specification, a data processing apparatus is provided, comprising:

[0040] The collection module is configured to collect historical access traffic data of the server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data.

[0041] The clustering module is configured to cluster the user identifiers of the access users according to an association relationship between the user identifiers and the data identifiers, and generate clustering results corresponding to a plurality of data identifiers respectively.

[0042] The labeling module is configured to label the access data according to the clustering results.

[0043] According to a third aspect of an embodiment of the present specification, a computing device is provided, comprising:

[0044] a memory and a processor;

[0045] The memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions:

[0046] collecting historical access traffic data of the server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data;

[0047] clustering the user identifiers of the access users according to an association relationship between the user identifiers and the data identifiers, and generating clustering results corresponding to a plurality of data identifiers respectively;

[0048] labeling the access data according to the clustering results.

[0049] According to a fourth aspect of an embodiment of the present specification, a computer readable storage medium is provided, which stores computer executable instructions, the instructions being executed by a processor to implement the steps of the data processing method.

[0050] One embodiment of the present specification collects historical access traffic data of the server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data, clusters the user identifiers of the access users according to an association relationship between the user identifiers and the data identifiers, generates clustering results corresponding to a plurality of data identifiers respectively, and labels the access data according to the clustering results.

[0051] The embodiment of the present specification divides the access data into data categories, analyzes and identifies which access data is private data and which access data is public data from the historical access traffic data, so as to facilitate accurate unauthorized access detection on the to-be-accessed data based on the identification result, improve the accuracy of horizontal unauthorized access detection, and improve the detection efficiency of unauthorized access vulnerabilities. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 is a processing flow diagram of a data processing method provided by an embodiment of the present specification;

[0053] FIG. 2(a) is a schematic diagram of a private data access mode provided by an embodiment of the present specification;

[0054] FIG. 2(b) is a schematic diagram of a public data access mode provided by an embodiment of the present specification;

[0055] Figure 3 is a processing flow diagram of a data processing method provided by an embodiment of the present specification;

[0056] Figure 4 is a schematic diagram of a data processing apparatus provided by an embodiment of the present specification;

[0057] Figure 5 is a structural block diagram of a computing device provided by an embodiment of the present specification. DETAILED DESCRIPTION

[0058] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present specification. However, the present specification can be practiced without the specific details, other than those described herein, and it is understood that the scope of the present specification is not limited to the details below.

[0059] The terms used in one or more embodiments of the present specification are merely for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of the present specification. The singular forms "a", "an" and "the" used in one or more embodiments of the present specification and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present specification means and includes any or all possible combinations of one or more associated listed items.

[0060] It should be understood that, although the terms first, second, etc. can be employed in describing various information in one or more embodiments of the present specification, the information should not be limited to such terms. These terms are only used to differentiate one piece of information from another piece of information of the same type. For example, without departing from the scope of one or more embodiments of the present specification, first can also be referred to as second, and similarly, second can also be referred to as first. Depending on the context, the word "if' as used herein can be interpreted as meaning "when" or "upon" or "in response to determining".

[0061] Firstly, the noun terms related to one or more embodiments of the present specification are explained.

[0062] Private data: data that only belongs to a specified user on a business system and only the specified user can operate or access is referred to as private data in the present specification.

[0063] Private data interface: the interface designed by the business system to access private data is referred to as a private data interface in the present specification.

[0064] Horizontal privilege breach: a security breach through which a user can access or operate other users' private data through a private data interface is referred to as a horizontal privilege breach.

[0065] Data unique identifier: a value marking a piece of data, which is relatively broad, and in most cases is a pure numerical value such as id = 1, but there can also be other cases such as a random string.

[0066] Public data: in contrast to private data, data that is accessible to all users in the system design.

[0067] Public data interface: in contrast to the private data interface, an interface designed in the system design to access public data.

[0068] In the present specification, a data processing method is provided, and the present specification also relates to a data processing device, a computing device, and a computer readable storage medium, which are described in detail one by one in the following embodiments.

[0069] Figure 1 A processing flowchart of a data processing method according to one embodiment of the present specification is shown, including steps 102 to 106.

[0070] Step 102: Collecting historical access traffic data of a server.

[0071] The historical access traffic data contains data identifiers of access data and user identifiers of access users, and the data identifier is used to index the access data.

[0072] Specifically, the historical access traffic data can be access traffic data generated by a user accessing a target application through a server; the target application can be a Web application based on a browser / server architecture, and the Web application not only includes some static pages with data display functions, but also has different business logic interfaces capable of processing operations such as querying, modifying, adding, and deleting data.

[0073] In actual applications, a user can perform operation processing on data in a target application through a server, for example, adding, deleting, modifying, or querying, and the like. However, the data in the target application can be divided into private data and public data. For the private data, if a part of the private data belongs to a user U1, only the user U1 can call an interface to perform adding, deleting, modifying, or querying on the part of the private data, and other users do not have the permission to perform any operation processing on the part of the private data. Similarly, the user U1 also does not have the permission to perform operation processing on private data of other users.

[0074] However, due to negligence of a background developer, the user identity is not determined during the adding, deleting, modifying, or querying on the data, or there is an error in the determination result of determining the user identity, that is, the server side excessively trusts a data operation request of the user U1 accessing the target application and ignores the determination of the operation permission of the user U1, so that the user U1 can perform adding, deleting, modifying, or querying on private data belonging to other users, thereby causing the target application to have an overreach vulnerability.

[0075] The data processing method provided by the embodiments of the present specification can be applied to a gateway at an outlet / entrance of a computer room or implemented in a WAF (Web Application Firewall) application protection system, so as to ensure that the obtained historical access traffic data is as complete as possible, thereby being capable of accurately detecting whether a page or an interface of a target application accessed by a user has an overreach vulnerability.

[0076] In addition, common network monitoring modes can be divided into two types: one is a bypass monitoring mode, and the other is a series monitoring mode. The bypass monitoring mode generally refers to monitoring through a "port mirroring" function of a network device such as a switch, and in this mode, the monitoring device only needs to be connected to a specified mirroring port of the switch, so it is called "bypass monitoring". In the case of using the bypass monitoring mode, the historical access traffic data can be bypass mirroring traffic. The series mode generally monitors through a gateway, a network bridge, or a proxy server. Since the monitoring device is connected in series as a gateway or a network bridge in the network, it is called "series monitoring mode". In the case of using the series monitoring mode, the historical access traffic data can be series traffic.

[0077] The specific network monitoring mode can be determined according to actual needs, and is not limited herein.

[0078] In the embodiments of the present specification, the historical access traffic data contains data identifiers of access data and user identifiers of access users, the user identifier is used to represent the identity of the user accessing the target application, and the data identifier is used to index the access data. If it is detected whether the target application has an unauthorized vulnerability, it is necessary to determine whether the user has the permission to access the access data, and the user identifier of the user is used to determine the user.

[0079] In practical applications, the session ID in the cookie can be used to identify the user identity accessing the access data, wherein the cookie refers to the data stored in the local terminal of the user for identifying the user identity and performing session tracking, is a small text file saved on the user browser by the Web server, can contain information about the user, is one of the main places for users to obtain, exchange and transfer information, and the Web site can access the cookie information whenever the user links to the server.

[0080] However, the session ID of the cookie is randomly generated when the user logs in, and the session ID is reset when the user logs in again or the session time expires. In this case, it will appear on the historical access traffic data that one data is accessed by multiple users, but actually the different session IDs accessing the one data all point to the same user, only the session ID changes, which will lead to inaccurate results obtained by analyzing the public and private data of the historical traffic.

[0081] In the embodiments of the present specification, in order to overcome the above problems and improve the accuracy of the unauthorized vulnerability detection results, while ensuring that the private data of the user is not leaked, the encrypted user information is stored in the cookie to avoid the problem of displaying multiple users caused by the session ID.

[0082] Specifically, when the user links to the server to request access to the access data, the Web site can access the cookie information to decrypt the encrypted user information stored in the cookie, and then use the decryption result as the user identifier accessing the access data. Since the encrypted user information stored in the cookie does not change, the user identifier after decryption always points to the same user.

[0083] By encrypting the user information in the cookie and obtaining the encrypted user information and decrypting it after receiving the data access request of the user, the user information is avoided from being tampered, and the user information statistics caused by session expiration or re-login is also avoided from being inaccurate.

[0084] In addition, the historical access traffic data is obtained from the bypass mirror traffic in the gateway, which is beneficial to ensure the integrity of the historical access traffic data collection, thereby being beneficial to ensure the accuracy of the data category labeling result.

[0085] In actual implementation, after collecting the historical access traffic data of the server, the user identifiers in the historical access traffic data need to be clustered, and the access data is divided into data categories according to the clustering result. However, in order to ensure the accuracy of the data category division result, the historical access traffic data needs to be filtered and screened. The filtering and screening can be realized by the following methods:

[0086] The historical access traffic data is cleaned to generate a corresponding data cleaning result.

[0087] The historical access traffic data contained in the data cleaning result is split to generate a plurality of historical access traffic sub-data.

[0088] The user identifiers and data identifiers in the plurality of historical access traffic sub-data are labeled by using target characters.

[0089] The hit result of the plurality of historical access traffic sub-data on at least one preset data screening rule in the labeling result is determined.

[0090] The plurality of historical access traffic sub-data is screened according to the hit result.

[0091] Specifically, since the real access traffic data in the actual application scenario is relatively complex, the historical access traffic data generated by the user by accessing the data in the target application program may contain some attack traffic data, crawler traffic data, invalid access traffic data and the like from the outside. Therefore, in order to ensure the data processing efficiency, after collecting the historical access traffic data, the historical access traffic data can be cleaned first, that is, the attack traffic data, the crawler traffic data and the invalid access traffic data are removed, and then the remaining historical access traffic is split, labeled, screened, clustered and the like.

[0092] Since the historical access traffic data contains characteristic parameters of various types of access data, in order to avoid the interference between different characteristic parameters, different characteristic parameters in a piece of historical access traffic data are respectively generated into a piece of data. For example, the historical access traffic data contains / ccc?phone=13000000000&info_id=333, which needs to be split to generate two pieces of data (historical access traffic sub-data): / ccc?phone=13000000000 and / ccc?info_id=333, and then the two pieces of data are labeled or subjected to subsequent screening, clustering and data category labeling process.

[0093] In actual application, the user identifier and the data identifier in the plurality of historical access traffic sub-data can be marked according to a preset marking rule. The marking rule can be defined according to actual needs. For example, the marking rule can be that the parameter name is the character ID at the end, and the parameter value is the ID value (here, the ID value is relatively broad, and can be a value uniquely identifying various formats of identification data). Specifically, the user identifier and the data identifier in the plurality of historical access traffic sub-data are marked by using the target character (ID).

[0094] After marking, a whitelist mode can be used. That is, the historical access traffic sub-data that meets the rule can be extracted by setting the rule. For example, the rule is that the historical access traffic sub-data ends with ID, and contains the data identifier of the access data, the user identifier of the access user, and the access time (the generation time of the historical traffic access data). If the user identifier and the data identifier are to be filtered, the user identifier and the data identifier in the historical access traffic data are marked by using the target character (ID). The marked user identifier and data identifier can meet the rule, and then the sub-data in the historical access traffic data can be filtered according to the meeting result.

[0095] The historical access traffic data is cleaned to remove interference traffic, which is beneficial to avoid the influence of interference data on subsequent statistics. In addition, the marked data that meets the rule is used, and multiple data can be generated by one traffic meeting multiple times, that is, the historical access traffic data is split, which is beneficial to avoid interference between feature parameters, and is also beneficial to avoid waste of computing resources.

[0096] In step 104, the user identifier of the access user is clustered according to the association relationship between the user identifier and the data identifier, and a clustering result corresponding to each data identifier is generated.

[0097] Specifically, after the historical access traffic data is collected, the user identifier of the access user can be clustered according to the association relationship between the user identifier and the data identifier, and a clustering result corresponding to each data identifier is generated, because the historical access traffic data contains the data identifier of the access data and the user identifier of the access user.

[0098] For example, if it is determined according to the historical access traffic data that the user U1, the user U2, and the user U3 have an association relationship with the data D1, the user U1, the user U2, and the user U3 are clustered, and a clustering result corresponding to the data D1 is generated.

[0099] In step 106, the access data is annotated according to the clustering result.

[0100] In a specific implementation, the access data is labeled according to the clustering result, and the labeling can be implemented in the following manner:

[0101] The target data identifier is subjected to deduplication processing, and the number of target user identifiers contained in the deduplication result is compared with a first preset number threshold.

[0102] If it is determined that the number of target user identifiers contained in the deduplication result is less than or equal to the first preset number threshold, it is determined that the access data associated with the target data identifier is private data; or

[0103] If it is determined that the number of target user identifiers contained in the deduplication result is greater than the first preset number threshold, it is determined that the access data associated with the target data identifier is public data; wherein the target data identifier is one of the plurality of data identifiers.

[0104] Specifically, since the association relationship between the user identifier and the data identifier in the historical access traffic data can be used to represent which users have accessed which data, the clustering result corresponding to the data identifier generated according to the association relationship can be used to represent which users have accessed the access data corresponding to the data identifier.

[0105] In the horizontal override scenario, if each data identifier (ID value) under a certain interface corresponds to only one user, it is determined that the interface is an interface for accessing private data of a user, and the access data corresponding to the data identifier is the private data of the user; if each data identifier (ID value) under a certain interface corresponds to multiple users, it is determined that the interface is an interface for accessing public data of a user, and the access data corresponding to the data identifier is public data. Therefore, after the clustering result corresponding to each data identifier is generated, the user identifiers contained in the clustering result can be subjected to deduplication processing, for example, the clustering result contains user U1, user U2, user U1, and user U3, and after deduplication, the clustering result contains user U1, user U2, and user U3.

[0106] After obtaining the deduplication result, the number of target user identifiers contained in the deduplication result is compared with a first preset number threshold, if it is determined that the number of target user identifiers contained in the deduplication result is less than or equal to the first preset number threshold, it is determined that the access data associated with the target data identifier is private data; or, if it is determined that the number of target user identifiers contained in the deduplication result is greater than the first preset number threshold, it is determined that the access data associated with the target data identifier is public data.

[0107] The schematic diagram of the access mode of the private data in the horizontal privilege-escalation scenario provided by the embodiments of the present specification is shown in FIG. 2(a). In FIG. 2(a), the data D1 has only one access user, i.e., the user U1, the data D2 has only one access user, i.e., the user U2, and the data D3 has only one access user, i.e., the user U3. Therefore, the data D1, the data D2 and the data D3 are private data.

[0108] The schematic diagram of the access mode of the public data in the horizontal privilege-escalation scenario provided by the embodiments of the present specification is shown in FIG. 2(b). In FIG. 2(b), the data D1 has three access users, i.e., the user U1, the user U2 and the user U3, and the data D2 has two access users, i.e., the user U1 and the user U3. Therefore, the data D1 and the data D2 are public data.

[0109] Further, after the data category labeling of the access data according to the clustering result, a mapping relationship table between the data identifiers of the access data and the data categories can be established according to the data category labeling result.

[0110] Moreover, the mapping relationship table can be used to determine whether the to-be-accessed data in the newly generated data access traffic is private data, so as to perform privilege-escalation detection. Specifically, the determination can be achieved in the following manner:

[0111] Obtaining the access traffic data to be detected;

[0112] Determining the data category of the to-be-accessed data in the access traffic data according to the data identifier of the to-be-accessed data and the mapping relationship table;

[0113] Detecting whether the server has a privilege-escalation vulnerability according to the data category of the to-be-accessed data.

[0114] Specifically, after the clustering result corresponding to each data identifier is generated, the access data corresponding to each data identifier is labeled according to the number of user types contained in the clustering result. That is, the data accessed by only one user is marked as private data, and the interface accessing the private data is marked as a private data interface. The data accessed by multiple users is marked as public data, and the interface accessing the public data is marked as a public data interface.

[0115] These data labeling results can be used as the basis for judging whether there is privilege escalation in the privilege-escalation detection process. In actual applications, the scanner can be used to perform privilege-escalation detection, or other methods capable of achieving privilege-escalation detection can be used in addition to the scanner. The specific method can be determined according to actual needs, which is not limited herein.

[0116] In addition, the data category labeling of the access data according to the clustering result can also be achieved in the following manner:

[0117] The target data identifier is one of the plurality of data identifiers, and the target user identifier is determined by deduplication processing of user identifiers in a corresponding clustering result of the target data identifier, and the number of target user identifiers contained in the deduplication processing result is determined.

[0118] The clustering result is filtered according to the number to obtain a target clustering result.

[0119] The data identifier corresponding to the target clustering result is annotated with a data category.

[0120] Specifically, since the actual business use case is relatively complex, there may be some interface users that rarely access in a target application, and if these cases are ignored and only the ideal state is calculated according to the case that each interface will be frequently accessed, the data generated by the interface rarely accessed by the user is not statistically meaningful, and therefore the result obtained by this way is not accurate enough.

[0121] Therefore, after generating the clustering result corresponding to each data identifier, the embodiment of the present specification can perform deduplication processing on the user identifiers in the clustering result corresponding to the target data identifier, and determine the number of target user identifiers contained in the deduplication processing result, the target data identifier being one of the plurality of data identifiers, the number of target user identifiers being used to represent the number of access users accessing the access data corresponding to the target data identifier.

[0122] After determining the number of target data identifiers in the deduplication processing result, the access interface of the access data corresponding to the target data identifier can be determined, and the deduplication processing results of the target data identifiers of the plurality of access data under the same access interface are integrated and deduplicated to obtain the number of target user identifiers corresponding to the access interface, the number of target user identifiers corresponding to the access interface being used to represent the number of access users accessing the access interface, so as to determine whether the historical access traffic data of the access interface has statistical significance according to the number of access users accessing the access interface.

[0123] If the number of access users accessing the access interface is lower than a second preset number threshold, it indicates that the historical access traffic data of the access interface is not statistically significant, and on the contrary, if the number of access users accessing the access interface is higher than the second preset number threshold, it indicates that the historical access traffic data of the access interface is statistically significant. Therefore, the clustering result is filtered according to the number to obtain a target clustering result, specifically, in the case that the number of access users accessing the access interface is higher than the second preset number threshold, the clustering result corresponding to the target data identifier of the access data under the access interface is taken as the target clustering result, and finally the data identifier corresponding to the target clustering result is marked with a data category.

[0124] Since the number of user identifiers included in the clustering result can be used to represent the number of times that a user accesses the access data, by filtering the clustering result, the interface traffic with too few access users and without statistical significance is removed, which is conducive to improving the accuracy of the data category marking result.

[0125] In addition, according to the clustering result, the access data can also be marked with a category in the following manner:

[0126] The user identifiers in the clustering result corresponding to the target data identifier are de-duplicated, and the number of target user identifiers included in the de-duplication result is determined.

[0127] The access interface of the access data corresponding to the target data identifier is determined.

[0128] The inverse of the number of target user identifiers in the de-duplication result corresponding to different data identifiers under the access interface is averaged.

[0129] The access data is marked with a category according to the average value. The target data identifier is one of the plurality of data identifiers.

[0130] Specifically, since the association relationship between the user identifier and the data identifier in the historical access traffic data can be used to represent which users access which data, after the user identifier is clustered according to the association relationship, the user identifier included in the clustering result is de-duplicated to represent the number of access users accessing the access data corresponding to the target data identifier according to the number of target user identifiers included in the de-duplication result.

[0131] Further, the embodiment of the present specification labels the access data under the same access interface according to the average of the reciprocal of the number of access users of different access data, so that after the user identifiers in the clustering results corresponding to different target data identifiers are de-duplicated, the access interface of the access data corresponding to different target data identifiers is determined, the number of access users contained in the de-duplicated results corresponding to the target data identifiers of different access data under the same access interface is determined, then the reciprocal of the number of access users contained in the de-duplicated results corresponding to the target data identifiers of different access data under the same access interface is obtained, and the average of the reciprocal of the number of access users of different access data under the same access interface is obtained, and the access data is labeled according to the average.

[0132] In an ideal state, if the access data under a certain access interface is private data, the calculation result obtained by the foregoing average calculation process should be equal to 1, representing that each access data under the access interface has only one user accessing; if the access data under a certain access interface is public data, the calculation result obtained by the foregoing average calculation process should be less than 1, representing that multiple users have accessed.

[0133] However, in actual application, the collected historical access flow data may contain some interference flow data such as other external attack flow data, crawler flow data, invalid access flow data, etc. Although the historical access flow data is cleaned before clustering, it may not be able to guarantee that all the interference flow data can be removed, so the calculation result obtained by the average calculation may have errors. In order to ensure the accuracy of the labeling result generated by labeling the access data according to the average calculation result, the embodiment of the present specification can set a numerical range, such as [0.95, 1], if the average calculation result falls within this numerical range, the access interface can be labeled as a private interface, and the access data under the access interface can be labeled as private data.

[0134] For example, a user can access data D1, data D2 and data D3 through access interface A, the number of target user identifiers contained in the de-duplicated result corresponding to data D1 is 1, the number of target user identifiers contained in the de-duplicated result corresponding to data D2 is 2, and the number of target user identifiers contained in the de-duplicated result corresponding to data D3 is 1. The average of the reciprocal of the number of access users of data D1, data D2 and data D3 under access interface A is obtained, and the average calculation result is (1 / 1+1 / 2+1 / 1) / 3=0.83. This average calculation result does not fall within the foregoing numerical range, so the access interface A can be labeled as a public interface, and data D1, data D2 and data D3 under access interface A can be labeled as public data.

[0135] In addition, after tagging the access data contained in the filtering results using the target characters corresponding to the data identifiers, the access data contained in the tagging results and the user identifiers that are related to the access data can be input into the data category labeling model to be trained for training, thereby obtaining the data category labeling model.

[0136] After obtaining the newly generated data access traffic to be detected, the data to be accessed contained in the access traffic data can be input into the data category labeling model for data category labeling, and the data category labeling result of the data to be accessed can be generated.

[0137] Specifically, the data access traffic to be detected is input into a pre-trained data category labeling model for data category labeling. The model output results can be used to determine whether the access data in the data access traffic is private data. If the access data is determined to be private data, the user identifier of the user to which the access data belongs is compared with the user identifier of the accessing user to perform unauthorized access detection.

[0138] The embodiments in this specification obtain public and private data models of application interfaces and corresponding data identifiers by aggregating and analyzing the historical traffic of the application, thereby determining whether the interface and data are private interfaces and private data, which helps to improve the accuracy of unauthorized access detection.

[0139] One embodiment of this specification collects historical access traffic data from a server. The historical access traffic data includes data identifiers of access data and user identifiers of accessing users. The data identifiers are used to index the access data. The user identifiers of accessing users are clustered according to the association between the user identifiers and the data identifiers to generate clustering results corresponding to multiple data identifiers. The access data is then labeled with data categories based on the clustering results.

[0140] This specification's embodiments categorize access data to identify which access data is private and which is public from historical access traffic data. This facilitates accurate privilege escalation detection based on the identification results, improves the accuracy of horizontal privilege escalation detection, and enhances the detection efficiency of privilege escalation vulnerabilities.

[0141] The following is in conjunction with the appendix Figure 3 Taking the application of the data processing method provided in this specification in an unauthorized access detection scenario as an example, the data processing method will be further explained. Among other things, Figure 3 A flowchart of a data processing method provided in one embodiment of this specification is shown, with specific steps including steps 302 to 328.

[0142] Step 302, collecting historical access traffic data of a server.

[0143] The historical access traffic data contains data identifiers of access data, and the data identifiers are used to index the access data.

[0144] Step 304, splitting the historical access traffic data.

[0145] Specifically, before splitting the historical access traffic data, the historical access traffic data can be cleaned to remove part of external malicious attack traffic, crawler traffic, invalid access traffic and the like, and the remaining historical access traffic data can be split.

[0146] In addition, after cleaning the historical access traffic data to obtain a first screening result, the historical access traffic data contained in the first screening result can be further screened. Specifically, cookie information associated with each historical access traffic data in the first screening result can be obtained, and encrypted user information stored in the cookie information can be decrypted. The access data contained in the first screening result can be screened according to the decryption result to generate a second screening result.

[0147] Specifically, when a user links to a server to request access to the access data, the Web site can access the cookie information to decrypt the encrypted user information stored in the cookie, and then use the decryption result as the user identifier for accessing the access data. In the embodiment of the application, after the encrypted user information stored in the cookie is decrypted to generate a corresponding decryption result, the historical access traffic data in the first screening result can be screened according to the decryption result. Specifically, it can be determined whether the user identifier is contained in the decryption result. If not, the historical access traffic data associated with the cookie information can be deleted from the first screening result to generate a second screening result.

[0148] Further, the historical access traffic data is split, i.e., the historical access traffic data contained in the second screening result is split.

[0149] Step 306, marking the historical access traffic sub-data in the splitting result.

[0150] Step 308, determining a hit result of the historical access traffic sub-data contained in the marking result on at least one preset data screening rule.

[0151] Step 310, screening the historical access traffic sub-data according to the hit result.

[0152] Step 312, according to the association relationship between the user identifier and the data identifier in the screening result, the user identifier of the access user is clustered, and the clustering result corresponding to the plurality of data identifiers is generated.

[0153] Step 314, the user identifier in the clustering result corresponding to the target data identifier is de-duplicated, and the number of target user identifiers contained in the de-duplicated result is determined, the target data identifier is one of the plurality of data identifiers.

[0154] Step 316, according to the number, the clustering result is screened, and the target clustering result containing the target user identifier is obtained, the number is greater than the second preset number threshold.

[0155] Step 318, the user identifier in the target clustering result is de-duplicated, and the user identifier contained in the de-duplicated result is compared with the first preset number threshold.

[0156] Step 320, according to the comparison result, the access data is data category labeled.

[0157] Specifically, the user identifier in the clustering result corresponding to the target data identifier is de-duplicated, and the number of target user identifiers contained in the de-duplicated result is compared with the first preset number threshold;

[0158] If it is determined that the number of target user identifiers contained in the de-duplicated result is less than or equal to the first preset number threshold, it is determined that the access data associated with the target data identifier is private data; or,

[0159] If it is determined that the number of target user identifiers contained in the de-duplicated result is greater than the first preset number threshold, it is determined that the access data associated with the target data identifier is public data; wherein the target data identifier is one of the plurality of data identifiers.

[0160] Step 322, according to the data category labeling result, the mapping relationship table between the data identifier and the data category of the access data is established.

[0161] Step 324, the access flow data to be detected is obtained.

[0162] Step 326, according to the data identifier of the to-be-accessed data contained in the access flow data and the mapping relationship table, the data category of the to-be-accessed data is determined.

[0163] Step 328, according to the data category of the to-be-accessed data, whether the server has an over-reach vulnerability is detected.

[0164] Specifically, the scanner and the data category of the to-be-accessed data can be used to detect whether the server has an over-reach vulnerability.

[0165] The embodiment of the present specification divides the access data into data categories, analyzes and identifies which access data is private data and which access data is public data from the historical access traffic data, thereby facilitating accurate unauthorized access detection on the to-be-accessed data based on the identification result, improving the accuracy of horizontal unauthorized access detection, and improving the detection efficiency of unauthorized access vulnerabilities.

[0166] Corresponding to the method embodiments described above, the present specification also provides data processing device embodiments, Figure 4 A schematic diagram of a data processing device provided by one embodiment of the present specification is shown. As shown in the figure, Figure 4 The device comprises:

[0167] The collection module 402 is configured to collect historical access traffic data of a server, wherein the historical access traffic data contains data identifiers of access data and user identifiers of access users, and the data identifiers are used to index the access data;

[0168] The clustering module 404 is configured to cluster the user identifiers of the access users according to the association relationship between the user identifiers and the data identifiers, and generate clustering results corresponding to a plurality of data identifiers respectively;

[0169] The labeling module 406 is configured to label the access data according to the clustering results.

[0170] Optionally, the data processing device further comprises:

[0171] The splitting module is configured to split the historical access traffic data;

[0172] The determination module is configured to determine a hit result of the access data contained in the splitting result on at least one preset data screening rule;

[0173] The screening module is configured to screen the access data according to the hit result, and mark the access data contained in the screening result with a target character corresponding to the data identifier.

[0174] Optionally, the labeling module 406 comprises:

[0175] The comparison submodule is configured to perform deduplication processing on the user identifiers in the clustering result corresponding to the target data identifier, and compare the number of target user identifiers contained in the deduplication processing result with a first preset number threshold;

[0176] The first determining sub-module is configured to determine that the access data associated with the target data identifier is private data if it is determined that the number of target user identifiers contained in the deduplication result is less than or equal to the first preset number threshold.

[0177] The second determining sub-module is configured to determine that the access data associated with the target data identifier is public data if it is determined that the number of target user identifiers contained in the deduplication result is greater than the first preset number threshold. The target data identifier is one of the plurality of data identifiers.

[0178] Optionally, the data processing apparatus further comprises:

[0179] The establishing module is configured to establish a mapping relationship table between the data identifier of the access data and the data category according to the data category annotation result.

[0180] Optionally, the data processing apparatus further comprises:

[0181] The first data obtaining module is configured to obtain access traffic data to be detected.

[0182] The first data category determining module is configured to determine the data category of the to-be-accessed data according to the data identifier of the to-be-accessed data contained in the access traffic data and the mapping relationship table.

[0183] Optionally, the data processing apparatus further comprises:

[0184] The detecting module is configured to detect whether the server has an overreach vulnerability according to the data category of the to-be-accessed data.

[0185] Optionally, the annotation module 406 comprises:

[0186] The number determining sub-module is configured to perform deduplication processing on the user identifiers in the clustering result corresponding to a target data identifier, and determine the number of target user identifiers contained in the deduplication processing result. The target data identifier is one of the plurality of data identifiers.

[0187] The clustering result screening sub-module is configured to screen the clustering result according to the number to obtain a target clustering result.

[0188] The annotation sub-module is configured to perform data category annotation on the data identifier corresponding to the target clustering result.

[0189] Optionally, the data processing apparatus further comprises:

[0190] The training module is configured to input access data contained in the marking result and a user identifier associated with the access data into a data category labeling model to be trained to obtain the data category labeling model.

[0191] Optionally, the data processing apparatus further comprises:

[0192] The second data acquisition module is configured to acquire access flow data to be detected.

[0193] The generation module is configured to input to-be-accessed data contained in the access flow data into the data category labeling model for data category labeling to generate a data category labeling result of the to-be-accessed data.

[0194] Optionally, the labeling module 406 comprises:

[0195] The deduplication processing submodule is configured to perform deduplication processing on the user identifiers in the clustering result corresponding to the target data identifier, and determine a number of target user identifiers contained in a deduplication processing result.

[0196] The access interface determination submodule is configured to determine an access interface of the access data corresponding to the target data identifier.

[0197] The calculation submodule is configured to take a mean value of reciprocals of the number of target user identifiers in the deduplication processing results corresponding to different data identifiers under the access interface.

[0198] The category labeling submodule is configured to label the access data according to the mean value; and the target data identifier is one of the plurality of data identifiers.

[0199] The embodiments of the present specification are beneficial to accurately detecting to-be-accessed data based on the identification result, improving the accuracy of horizontal privilege detection, and improving the detection efficiency of privilege leakage.

[0200] The above is a schematic scheme of the data processing apparatus of the present embodiment. It should be noted that the technical scheme of the data processing apparatus and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the data processing apparatus that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0201] Figure 5A structural block diagram of a computing device 500 according to one embodiment of the present specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected with the memory 510 through a bus 530, and a database 550 is used to save data.

[0202] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 can include one or more of any type of network interface (e.g., a network interface card (NIC)) such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like, either wired or wireless.

[0203] In one embodiment of the present specification, the above-mentioned components of the computing device 500 and other components not shown in the above-mentioned components can be connected with each other, for example, through a bus. It should be understood that Figure 5 the computing device structure block diagram shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced as needed by those skilled in the art. Figure 5 the computing device structure block diagram shown is only for the purpose of example, and is not a limitation on the scope of the present specification. Other components can be added or replaced as needed by those skilled in the art.

[0204] The computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other type of mobile device, or a stationary computing device such as a desktop computer or a PC. The computing device 500 can also be a mobile or stationary server.

[0205] The memory 510 is configured to store computer-executable instructions, and the processor 520 is configured to execute the computer-executable instructions as follows:

[0206] Collect historical access traffic data of a server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data;

[0207] Cluster the user identifiers of the access users according to an association relationship between the user identifiers and the data identifiers, to generate clustering results corresponding to a plurality of data identifiers respectively;

[0208] According to the clustering result, data category labeling is performed on the access data.

[0209] The above is a schematic scheme of the computing device of the embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the computing device that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0210] An embodiment of the present specification also provides a computer readable storage medium storing computer instructions, which are executed by a processor to implement the steps of the data processing method.

[0211] The above is a schematic scheme of the computer readable storage medium of the embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the data processing method described above belong to the same concept, and the details of the technical scheme of the storage medium that are not described in detail can be referred to the description of the technical scheme of the data processing method.

[0212] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims can be performed in a different order than the order in which they are recited and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0213] The computer instructions include computer program code, which can be in the form of source code, object code, executable code, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0214] It should be noted that, for the aforementioned method embodiments, the sequences of the described actions are not necessarily required to implement the present application, and certain actions can be performed in other sequences, or even at the same time, in accordance with the present application. Furthermore, certain actions can not be required to implement the present application. Additionally, the described embodiments are not necessarily the only possible implementation of the present application.

[0215] In the above embodiments, the description of each embodiment is focused on a certain aspect, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0216] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details, nor limit the present application to the specific embodiments described. Obviously, according to the content of the present application, many modifications and changes can be made. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and use the present application. The present application is limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: collecting historical access traffic data of a server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data; clustering the user identifiers; performing data category labeling on the access data according to the clustering result, the data category including private data and public data; establishing a mapping relationship table between the data identifiers and the data category of the access data according to the data category labeling result; obtaining access traffic data to be detected; determining the data category of access data to be accessed according to the data identifier of the access data to be accessed contained in the access traffic data and the mapping relationship table; and detecting whether the server has an unauthorized vulnerability according to the data category of the access data to be accessed. 2.The data processing method of claim 1, after the collecting historical access traffic data of a server, further comprising: splitting the historical access traffic data; determining a hit result of the access data contained in the splitting result on at least one preset data screening rule; screening the access data according to the hit result, and marking the access data contained in the screening result with a target character corresponding to the data identifier. 3.The data processing method of claim 1 or 2, wherein the performing data category labeling on the access data comprises: performing deduplication processing on the user identifiers in the clustering result corresponding to a target data identifier, and comparing the number of target user identifiers contained in the deduplication processing result with a first preset number threshold; if it is determined that the number of target user identifiers contained in the deduplication result is less than or equal to the first preset number threshold, determining that the access data associated with the target data identifier is private data; or if it is determined that the number of target user identifiers contained in the deduplication result is greater than the first preset number threshold, determining that the access data associated with the target data identifier is public data; wherein the target data identifier is one of the plurality of data identifiers. 4.The data processing method of claim 1, wherein the performing data category labeling on the access data comprises: performing deduplication processing on the user identifiers in the clustering result corresponding to a target data identifier, and determining the number of target user identifiers contained in the deduplication processing result, the target data identifier being one of the plurality of data identifiers; screening the clustering result according to the number to obtain a target clustering result; and performing data category labeling on the data identifiers corresponding to the target clustering result. 5.The data processing method of claim 2, further comprising: inputting the access data contained in the marking result and the user identifiers having an association relationship with the access data into a data category labeling model to be trained to train the data category labeling model. 6.The data processing method of claim 5, further comprising: obtaining access traffic data to be detected. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The to-be-accessed data contained in the access traffic data is input into the data category labeling model for data category labeling, and a data category labeling result of the to-be-accessed data is generated.

7. The data processing method of claim 1 or 2, wherein the category labeling of the access data comprises: performing deduplication processing on the target data identifier in the corresponding clustering result, and determining a number of target user identifiers contained in a deduplication processing result; determining an access interface of the access data corresponding to the target data identifier; averaging inverses of the number of target user identifiers in the deduplication processing results corresponding to different data identifiers under the access interface; category labeling the access data according to the average; and wherein the target data identifier is one of the plurality of data identifiers.

8. A data processing apparatus comprising: a collection module configured to collect historical access traffic data of a server, the historical access traffic data containing data identifiers of access data and user identifiers of access users, the data identifiers being used to index the access data; a clustering module configured to cluster the user identifiers; a labeling module configured to data category label the access data according to a clustering result, the data categories including private data and public data; the data processing apparatus further comprises an establishing module configured to establish a mapping relationship table between data identifiers of the access data and data categories according to a data category labeling result; a first data acquisition module configured to acquire to-be-detected access traffic data; a first data category determination module configured to determine a data category of the to-be-accessed data according to a data identifier of to-be-accessed data contained in the access traffic data and the mapping relationship table; a detection module configured to detect whether the server has an unauthorized vulnerability according to the data category of the to-be-accessed data.

9. A computing device comprising: a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to implement the steps of the data processing method of any one of claims 1 to 7 when executing the computer executable instructions.

10. A computer readable storage medium storing computer instructions, the instructions being executed by a processor to implement the steps of the data processing method of any one of claims 1 to 7.

11. A computer program product, characterised in that, computer instructions, the computer instructions being executed by a processor to implement the steps of the data processing method of any one of claims 1 to 7.

Images