Data detection method and device for application program and storage medium

Abstract

The embodiment of the invention provides a data detection method and device for an application program and a storage medium, and the method comprises the steps: collecting flow data flowing through an application program firewall, the flow data comprising first input data; and according to the data type corresponding to the first response data, converting the first response data into a JSON object so as to extract a key value pair contained in the first response data. The first response data is data which is extracted from the first input data, has a data packet size meeting a preset threshold value and does not belong to the simulation environment. And determining a classification label corresponding to the key value pair according to the data label tree corresponding to the application program. And performing sensitive data detection on the key value pair according to the classification label so as to determine whether the first response data comprises sensitive data or not. The classification labels corresponding to the key value pairs in the response data are automatically marked through the data label tree, whether the response data contain the sensitive data or not is further determined in combination with the classification labels, and the detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of security protection technology, and in particular to a data detection method, device and storage medium for an application program. Background Art

[0002] With the development of mobile Internet and the popularization of smart devices, various applications have emerged. As the carrier of network information, applications are responsible for disseminating information to the outside world. If important data such as user privacy data, algorithm keys, etc. in the application are leaked to the Internet, it will cause serious losses to application security, user privacy protection, property safety, etc.

[0003] In order to effectively prevent the leakage of important data in applications, existing data detection solutions basically use the method of detecting whether there is potential exposure risk in the code corresponding to the application. However, this static detection method cannot detect sensitive data dynamically generated during the operation of the application, which may cause some user privacy data to be leaked, resulting in losses to the interests of users.

[0004] Therefore, how to accurately detect whether sensitive data leakage occurs in an application has become an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present invention provide a method, apparatus, device and storage medium for detecting data of an application program, so as to improve the accuracy of sensitive data detection results.

[0006] In a first aspect, an embodiment of the present invention provides a method for detecting data of an application program, the method comprising:

[0007] Collecting traffic data flowing through an application firewall, where the application firewall is deployed at the front end of the application, the traffic data including first input data;

[0008] According to a data type corresponding to the first response data, convert the first response data into a JSON object to extract key-value pairs contained in the first response data, wherein the first response data is data extracted from the first input data, the data packet size of which meets a preset threshold and does not belong to a simulation environment;

[0009] Determine the classification label corresponding to the key-value pair according to the data label tree corresponding to the application;

[0010] According to the classification label, sensitive data detection is performed on the key-value pair to determine whether the first response data includes sensitive data.

[0011] In a second aspect, an embodiment of the present invention provides a data detection device for an application program, the device comprising:

[0012] A collection module, used for collecting traffic data flowing through an application firewall, the application firewall being deployed at the front end of the application, the traffic data including first input data;

[0013] a parsing module, configured to convert the first response data into a JSON object according to a data type corresponding to the first response data, so as to extract key-value pairs contained in the first response data, wherein the first response data is data extracted from the first input data, the data packet size of which meets a preset threshold and does not belong to a simulation environment;

[0014] A determination module, used to determine the classification label corresponding to the key-value pair according to the data label tree corresponding to the application program;

[0015] A detection module is used to perform sensitive data detection on the key-value pair according to the classification label to determine whether the first response data includes sensitive data.

[0016] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the data detection method of the application as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of a transmission device, the processor can at least implement the data detection method of the application as described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present invention provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, the processor can implement the data detection method of the application as described in the first aspect above.

[0019] In the data detection scheme for the application provided in the embodiment of the present invention, during the operation of the application, sensitive data detection is performed on the traffic data flowing through the application firewall. In this way, sensitive data detection can be performed on all response data entering and leaving the application, thereby achieving full coverage detection, improving the accuracy of the sensitive data detection results, and timely discovering sensitive data leakage in the application.

[0020] Specifically, when performing data detection on an application, first, traffic data flowing through the application firewall is collected, and the traffic data includes the first input data. Among them, the application firewall is deployed at the front end of the application. Then, the data packet size corresponding to the first input data and the data type corresponding to the first input data are determined to extract data whose data packet size meets a preset threshold and does not belong to the simulation environment from the first response data, and the extracted data is determined as the first response data. According to the data type corresponding to the first response data, the first response data is converted into a JSON object to extract the key-value pairs contained in the first response data. Then, according to the data label tree corresponding to the application, the classification label corresponding to the key-value pair is determined. Finally, according to the classification label corresponding to each key-value pair in the first response data, sensitive data detection is performed on the key-value pair to determine whether the first response data contains sensitive data.

[0021] In the above scheme, during the operation of the application, by collecting the first input data flowing through the application firewall and extracting the first response data from the first input data, sensitive data detection is performed on each key-value pair according to the classification label corresponding to each key-value pair in the first response data to determine whether the first response data includes sensitive data. In this way, sensitive data detection and processing can be performed on both the intranet and the extranet traffic data, which not only achieves full coverage detection and reduces the possibility of missed detection, but also improves detection efficiency. In addition, when performing sensitive data detection, invalid data is filtered out by first determining the data packet size corresponding to the first response data and the data type corresponding to the first response data, and then using the data label tree to automatically mark the classification label corresponding to each key-value pair in the response data, and combining the classification label of each key-value pair in the response data to further determine whether the response data contains sensitive data, which can improve the efficiency and accuracy of sensitive data detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A schematic diagram of a flow chart of a data detection method for an application program provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a data tag tree provided by an embodiment of the present invention;

[0025] Figure 3A flowchart of filtering collected first response data provided by an embodiment of the present invention;

[0026] Figure 4 An application schematic diagram of a data detection method for an application program provided by an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the structure of a data detection device for an application provided by an embodiment of the present invention;

[0028] Figure 6 For Figure 5 A schematic diagram of the structure of an electronic device corresponding to the data detection device for an application provided in the illustrated embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0032] In order to facilitate understanding of the technical solution of the present invention, the following is a brief description of the related technology:

[0033] Applications usually include important data such as user privacy data and algorithm keys. These data are closely related to application security, user privacy protection, and property safety. If these important data are leaked to the Internet, serious losses will be caused.

[0034] In order to effectively monitor whether important data in the application is leaked, sensitive data leakage detection is usually performed directly on the code corresponding to the application. Some sensitive data is encrypted and stored in the application, but it will be decrypted and used when the application is running, and data leakage may still occur. In addition, during the operation of the application, users may dynamically enter some sensitive data, such as transaction / login passwords. If they are not destroyed in time after use, leakage may occur, causing privacy leakage and property loss to users. However, the above method cannot detect the leakage of this part of sensitive data, and cannot timely discover the leakage of sensitive data in the application.

[0035] In order to solve the above technical problems, an embodiment of the present invention provides a new data detection method in an application. This technical solution, by using the data tag tree corresponding to the application, determines the classification tags corresponding to each key-value pair in the response data flowing through the application firewall, and then performs sensitive data detection on the key-value pairs according to the classification tags corresponding to each key-value pair in the response data, so as to determine whether the response data flowing through the application firewall contains sensitive data. In this way, not only can the traffic data from the intranet and the extranet be detected and processed for sensitive data, but also the accuracy of the sensitive data detection results can be improved, so as to timely discover the leakage of sensitive data in the application, and improve the security of the application.

[0036] In the following, some embodiments of the present invention are described in detail in conjunction with the accompanying drawings. In the case where there is no conflict between the embodiments, the following embodiments and features in the embodiments can be combined with each other.

[0037] Figure 1 A flow chart of a data detection method for an application program provided by an embodiment of the present invention; Figure 1 As shown, this embodiment provides a data detection method for an application program. The execution subject of the method may be a data detection device for an application program. Specifically, the method may include the following steps:

[0038] 101. Collect traffic data flowing through an application firewall, where the application firewall is deployed at the front end of the application, and the traffic data includes first input data.

[0039] 102. According to a data type corresponding to the first response data, convert the first response data into a JSON object to extract key-value pairs contained in the first response data, wherein the first response data is data extracted from the first input data, the data packet size meets a preset threshold and does not belong to a simulation environment.

[0040] 103. Determine the classification label corresponding to the key-value pair based on the data label tree corresponding to the application.

[0041] 104. Perform sensitive data detection on the key-value pairs according to the classification labels to determine whether the first response data includes sensitive data.

[0042] The application data detection scheme provided in the embodiment of the present invention can be used to detect sensitive data leakage of any application with security protection requirements. Among them, the application can be any application installed on a computer device (such as a computer, a mobile phone), such as financial application software, instant messaging application software, shopping application software, etc., and the present invention does not limit this. Sensitive data can be important data such as user privacy data and algorithm keys.

[0043] In actual applications, the API interface in the application provides a convenient way to access data. In the design and development process of the API interface, there is a situation where the focus is on functionality and security is ignored, resulting in excessive exposure of sensitive data. Some unauthorized users or attackers may access sensitive data through the API interface, resulting in potential data leakage risks.

[0044] In order to ensure that sensitive data in the application is not leaked, API sensitive data leakage detection can be performed on the entire traffic at the Internet boundary application firewall WAF exit. This can not only detect and process sensitive data on traffic data from both the intranet and the extranet to fully cover the data leakage surface and reduce the possibility of missed detection, but also can bypass the sensitive data of the entire traffic passing through the WAF exit without affecting the availability of the web server.

[0045] In a specific implementation, traffic data flowing through an application firewall may be collected, wherein the application firewall is deployed at the front end of the application, and the traffic data includes first input data. It should be noted that the first input data here is equivalent to response data corresponding to the request data. Moreover, the number of input data included in the collected traffic data is not limited here.

[0046] In addition, the request data flowing through the application firewall can also be collected at the same time. Since the input data does not carry API interface information, when the first input data is detected to contain sensitive data, the API interface with security risks can be determined based on the API interface information carried in the first request data corresponding to the first input data. That is, API data and API response data from the Internet and the intranet are collected to ensure that all traffic data in and out of the application is fully covered.

[0047] Among them, the Openresty plug-in can be used to collect API request data and API input data from the Internet and the intranet that flow through the application firewall. The Openresty plug-in is a high-performance Web platform based on Nginx and Lua, which integrates a large number of sophisticated Lua libraries, third-party modules, and most dependencies. Nginx refers to an open source high-performance HTTP and reverse proxy web server, and Lua is a lightweight, embeddable scripting language.

[0048] In addition, in order to timely discover sensitive data leaked in the application, the flow data flowing through the application firewall can be collected in real time. However, in actual applications, the amount of data flowing through the application firewall is very large. In order to ensure the efficiency of sensitive data detection, in an optional embodiment, after the flow data flowing through the application firewall is collected in real time, the collected flow data can be sampled and processed, and then sensitive data detection can be performed on the sampled flow data.

[0049] Specifically, the sampling ratio can be set according to actual needs, and a corresponding amount of flow data can be extracted from the collected flow data in sequence according to the preset sampling ratio. For example, assuming that the sampling ratio is percentage sampling, 1 input data is extracted from every 100 input data.

[0050] It should be noted that: when collecting traffic data, API request data and API input data corresponding to the API request data are collected at the same time. When performing sensitive data detection, there is a possibility that the API input data will leak sensitive data to the outside, so sensitive data detection can be performed only on the API input data. Therefore, when performing sampling processing, the API request data and the API input data corresponding to the API request data can be regarded as a data group, and the traffic data can be sampled in the form of data groups.

[0051] In another optional embodiment, the Monte Carlo method can also be used to statistically analyze the corresponding flow data from the large amount of collected flow data. Specifically, a mathematical model for generating random samples can be constructed, and the mathematical model can be used to perform sampling processing on the large amount of collected flow data to obtain sampled flow data.

[0052] Among them, when the collected traffic data is sampled, if there are new functions in the application, then while sampling the collected traffic data, the specific server cluster corresponding to the newly launched function can also be collected for full traffic collection and processing. This can ensure detection efficiency while ensuring that certain security risks are missed.

[0053] After obtaining the first input data flowing through the application firewall and the first request data corresponding to the first input data, it is then possible to determine whether the data packet size corresponding to the first input data meets a preset threshold and whether the first input data is simulated environment data, so as to extract data whose data packet size meets the preset threshold and does not belong to a simulated environment from the first input data, and determine the extracted data as the first response data.

[0054] Next, the first response data is parsed to extract the key-value pairs contained in the first response data. Before parsing the first response data, the data packet size corresponding to the first input data and the data type corresponding to the first input data are first determined to filter out invalid data and improve data detection efficiency.

[0055] When parsing the first response data, the first response data may be converted into a JSON object according to a data type corresponding to the first response data, so as to extract key-value pairs contained in the first response data.

[0056] In an optional embodiment, according to the data type corresponding to the first response data, the first response data is converted into a JSON object to extract the key-value pairs contained in the first response data. The specific implementation process may be: according to the data type corresponding to the first response data, the first response data is converted into a JSON string; using a JSON parsing library, the JSON string is parsed into a JSON object; and the key-value pairs contained in the JSON object are extracted.

[0057] In web applications, various data are often transmitted in the form of key-value pairs. For example, in web applications, form data is often transmitted in the form of key-value pairs, where the key is the name of the form field and the value is the data entered by the user.

[0058] When performing sensitive data detection on the response data, sensitive data detection can be performed on the structured data transmitted in the form of key-value pairs in the response data. In the key-value pair structure, each data item consists of a unique key and a value associated therewith. Therefore, in the embodiment of the present invention, it can be detected whether the value in each key-value pair in the response data contains sensitive data.

[0059] Next, the classification label corresponding to the key-value pair in the first response data is determined according to the data label tree corresponding to the application. The data label tree organizes the relevant labels according to a hierarchical relationship. Before classifying the values ​​in the key-value pair, multiple category labels for classifying the data can be determined first.

[0060] In an optional embodiment, classification labels for data classification can be pre-constructed according to the classification scenario of the current data classification, and these classification labels have multiple category levels, and each classification label corresponds to a preset data security level. Among them, the preset data security level corresponding to each classification label can be determined according to the sensitivity of the data, the importance of the data, or the degree of loss caused by data leakage. The higher the sensitivity, the higher the corresponding data security level, the higher the importance of the data, the higher the corresponding data security level, and the higher the degree of loss caused by data leakage, the higher the corresponding data security level.

[0061] After constructing these classification labels, they can be sorted in order according to the hierarchical relationship to form a data label tree corresponding to the application. In this way, when classifying the values ​​in the key-value pairs, the data label tree can be used to automatically classify and label the values ​​in the key-value pairs to more accurately determine the classification labels corresponding to the values ​​in the key-value pairs.

[0062] For ease of understanding, please refer to the attached Figure 2 The data label tree is illustrated with an example. Figure 2 As shown, there are three levels of classification labels for classifying data, namely, primary classification labels, secondary classification labels, and tertiary classification labels, wherein a primary classification label contains multiple secondary classification labels, and a secondary classification label contains multiple tertiary classification labels, and the multi-level classification labels are in a tree structure. Figure 2 As shown, the first-level category tags include: user information, business data, security operation and maintenance, and business management. The user information category in the first-level category tag includes the following second-level category tags: personal basic information, personal identity information, personal device information, personal identity authentication information, network identity identification, personal Internet access records, personal property information, personal physiological health information, etc. The second-level category tag personal basic information includes the following third-level category tags: gender, ethnicity, nationality, education, birthday, marital status, user name, etc.

[0063] To facilitate the safe management of user data, Figure 2 In the illustrated scenario, the corresponding data security level is set from the dimension of the secondary category label, for example: personal basic information is set to C3 level, personal identity information is set to C4 level, personal device information is set to C3 level, personal identity authentication information is set to C4 level, network identity identification is set to C4 level, personal Internet records are set to C3 level, personal property information is set to C4 level, personal physiological health information is set to C3 level, etc.

[0064] After constructing the data label tree corresponding to the application, you can use the data label tree corresponding to the application to label the value in the key-value pair to determine the classification label corresponding to the value.

[0065] Among them, in order to improve the accuracy of sensitive data detection results, after determining the classification label corresponding to each value in the first response data, then, based on the classification label corresponding to the value, sensitive data detection is performed on the value to determine whether the first response data includes sensitive data.

[0066] Among them, a pre-trained sensitive data detection model can be used to perform sensitive data detection on the value according to the classification label corresponding to the value to determine whether the first response data includes sensitive data. Among them, a large number of sensitive data samples can be used to learn and train the neural network model to obtain a trained sensitive data detection model. After obtaining the trained sensitive data detection model, the sensitive data detection model can be used to perform sensitive data detection on the value in the key-value pair, which can improve the accuracy of the obtained detection results.

[0067] In an embodiment of the present invention, during the operation of an application, the first input data flowing through the application firewall is collected, and data whose data packet size meets a preset threshold and does not belong to a simulation environment is extracted from the first input data to be determined as the first response data, and sensitive data detection is performed on each key-value pair according to the classification label corresponding to each key-value pair in the first response data to determine whether the response data includes sensitive data. In this way, sensitive data detection and processing can be performed on both the intranet and the extranet traffic data, which not only achieves full coverage detection and reduces the possibility of missed detection, thereby improving the security of the application, but also can detect only valid response data, thereby improving data detection efficiency. In addition, when performing sensitive data detection, by using a data label tree to automatically mark the classification label corresponding to each key-value pair in the response data, and combining the classification label of each key-value pair in the response data, it is further determined whether the response data contains sensitive data, which can improve the efficiency and accuracy of sensitive data detection.

[0068] The above embodiment describes parsing the first response data flowing through the application firewall, extracting the key-value pairs in the first response data, and performing sensitive data detection processing on the value values ​​in the key-value pairs to determine whether the first response data contains sensitive data.

[0069] In actual applications, the first input data flowing through the application firewall includes not only the actual response data corresponding to the user request data, but also the response data corresponding to the test environment or honeypot environment. If a large amount of traffic data corresponding to the test environment is collected during real-time detection of traffic data, the detection processing burden will increase and the detection efficiency will be reduced. In addition, since the traffic data in the test environment often contains a large amount of simulated data or temporary data, these data may not contain sensitive information in the actual functions of the application, but their existence may interfere with the normal operation of sensitive data detection, resulting in false positives or negatives. Therefore, in the specific implementation, these response data need to be filtered out to improve the efficiency and accuracy of the detection results.

[0070] In order to more clearly understand the above filtering process, its implementation process is exemplarily described in conjunction with the following embodiments.

[0071] Figure 3 A flowchart of filtering collected first input data is provided in an embodiment of the present invention; Figure 3 As shown, the method comprises the following steps:

[0072] 301. Determine a data packet size corresponding to first input data.

[0073] 302. Determine whether the first input data is response data generated by an automated testing tool according to an identifier corresponding to the first input data or an IP address corresponding to the first input data.

[0074] 303. Determine whether the first input data is honeypot environment data according to an identifier corresponding to the first input data or an IP address corresponding to the first input data.

[0075] 304. If the data packet size is smaller than a preset threshold, the first input data is not test environment data, and the first input data is not honeypot environment data, then the first input data is determined as first response data.

[0076] 305. If the data packet size is greater than a preset threshold or the first response data is test environment data or the first response data is honeypot environment data, then delete the first input data.

[0077] Among them, the flow data that does not meet the requirements can be filtered out to avoid the influence of invalid or redundant data on the detection statistical results. For example, irregular large packet data, test environment flow data, and honeypot environment flow data can be filtered out.

[0078] In a specific implementation, the size of a data packet corresponding to the first input data is determined. If the size of a data packet corresponding to the first input data is larger than a preset threshold, the first input data is filtered out and no sensitive data detection processing is performed on the first input data.

[0079] If it is determined that the data packet size corresponding to the first input data is less than a preset threshold, it can also be determined whether the first input data is test environment traffic data. Specifically, according to the identifier corresponding to the first input data or the IP address corresponding to the first input data, it is determined whether the first input data is response data generated by an automated testing tool. If the first input data is response data generated by an automated testing tool, the first input data is filtered out and no sensitive data detection processing is performed on the first input data.

[0080] If it is determined that the data packet size corresponding to the first input data is smaller than a preset threshold and the first input data is not response data generated by an automated testing tool, it can also be determined whether the first input data is honeypot environment traffic data.

[0081] Specifically, based on the identifier corresponding to the first input data or the IP address corresponding to the first input data, determine whether the first input data is honeypot environment data; if the first input data is response data corresponding to the honeypot environment, filter out the first input data and no longer perform sensitive data detection processing on the first input data.

[0082] If the data packet size corresponding to the first input data is less than a preset threshold, the first input data is not test environment data, and the first input data is not honeypot environment data, the first input data is extracted and determined as the first response data to distinguish invalid data in the first input data. After the first response data is extracted, the first response data is parsed to extract the key-value pairs contained in the first response data, so as to perform sensitive data detection processing on the values ​​in the key-value pairs.

[0083] From the above description, it can be seen that before performing sensitive data detection processing on the collected first input data, the collected first input data can be screened first. This can not only improve the efficiency of sensitive data detection, but also avoid the influence of the response data corresponding to the test environment or honeypot environment on the test results, so as to improve the accuracy of the sensitive data detection results.

[0084] After filtering out invalid data or redundant data in the collected traffic data, the filtered traffic data is parsed to obtain the key-value pairs corresponding to each response data. Then, the data label tree corresponding to the application is used to determine the classification label corresponding to the key-value pair.

[0085] In an optional embodiment, in order to more accurately match the corresponding classification label for each key-value pair in the response data, when creating the data label tree, the keywords corresponding to each category label in the data label tree can also be determined. In this way, in actual applications, the classification label matched by the key-value pair can be determined based on the keywords corresponding to each category label.

[0086] In a specific implementation, where the key-value pair includes a field name and a field value, multiple category labels for classifying the field values ​​can be determined first, and keywords corresponding to each of the multiple category labels can be determined respectively. Based on the multiple category labels and the keywords corresponding to the multiple category labels, a data label tree corresponding to the application can be determined.

[0087] Next, the field values ​​in each key-value pair in the first response data are converted into vector representations to obtain the first word vectors corresponding to each field value. The keywords corresponding to each of the multiple category labels are converted into vector representations to obtain the second word vectors corresponding to each keyword, and the similarity between the first word vector and the second word vector corresponding to each keyword is determined. Based on the similarity between the first word vector and each second word vector, the keywords matching the field value are determined. Based on the keywords matching the field value, the classification label corresponding to the field value is determined.

[0088] Among them, the second word vectors corresponding to the keywords of each category label in the data label tree can be determined in advance and stored in a preset database. When sensitive data detection processing is performed on the application, the second word vectors corresponding to each keyword can be read from the preset database.

[0089] In addition, in an optional embodiment, the matching results of the field value and the keywords corresponding to each of the multiple category tags in the data tag tree can be determined based on a preset regular expression, and the category tag corresponding to the field value can be determined based on the matching results.

[0090] Alternatively, in another optional embodiment, a large language model may be used to determine the classification label corresponding to the field value. Specifically, a preset prompt word template may be used to generate a target prompt word according to the field value and the task description, and the generated target prompt word is input into the large language model to guide the large language model to output the classification label corresponding to the field value.

[0091] In the above-mentioned embodiments, the data detection process of the application is described in detail by taking the first response data as an example. However, in actual applications, the application will generate a large amount of traffic data in real time, and many of the response data come from the same API interface. If there is a risk of sensitive data leakage in a certain API interface corresponding to the application, then many of the response data collected from the API interface contain sensitive data. If an alarm message is generated for each response data, the workflow will be very cumbersome for the developer, and it will be difficult to timely manage the risk of leakage. Therefore, in an embodiment of the present invention, when the traffic data includes a large amount of response data, multiple response data from the same API interface can be aggregated to determine the same API interface corresponding to the multiple response data, and then an alarm message is generated based on the aggregated information and the API interface information corresponding to the multiple response data from the same API interface.

[0092] This process is described in detail below in conjunction with the following embodiments.

[0093] Figure 3 A flowchart of data detection of another application provided by an embodiment of the present invention; Figure 3 As shown, the traffic data includes a plurality of response data, and the plurality of response data includes the first response data, and the method further includes the following steps:

[0094] 301. Filter out a plurality of second response data containing sensitive data from a plurality of response data.

[0095] 302. Determine in sequence the cleaned uniform resource identifiers corresponding to the plurality of second response data respectively, where the cleaned uniform resource identifier is a uniform resource identifier with redundant information removed.

[0096] 303. Determine the API interfaces corresponding to the plurality of second response data respectively according to the cleaned uniform resource identifiers corresponding to the plurality of second response data respectively.

[0097] 304. Generate warning information based on the API interfaces corresponding to the multiple second response data and the sensitive data corresponding to the multiple second response data.

[0098] After sensitive data detection is performed on the multiple response data collected respectively, multiple second response data containing sensitive data are screened out from the multiple response data, and then the API interfaces corresponding to the multiple second response data are determined in turn, and the API interfaces corresponding to the multiple second response data are aggregated, and then an alarm message is generated based on the API interface information obtained after aggregation.

[0099] Since the specific implementation process of determining the API interface by each second response data is roughly the same, any one of the second response data is taken as an example here to describe in detail the specific implementation process of determining the API interface.

[0100] First, determine the cleaned uniform resource identifier URI corresponding to the second response data. The cleaned uniform resource identifier URI is a uniform resource identifier with redundant information removed. Then, determine the API interface corresponding to the second response data according to the cleaned uniform resource identifier corresponding to the second response data.

[0101] In an optional embodiment, the specific implementation method of determining the cleaned uniform resource identifier corresponding to the second response data may be: obtaining the uniform resource locator corresponding to the second response data; parsing the uniform resource locator to extract the domain name and uniform resource identifier from the uniform resource locator; and generating the cleaned uniform resource identifier corresponding to the second response data based on the domain name and the uniform resource identifier.

[0102] Among them, since the second response data usually does not carry the corresponding API interface information, in order to obtain the uniform resource locator corresponding to the second response data, the second request data corresponding to the second response data can be determined, and the uniform resource locator corresponding to the second response data can be determined according to the uniform resource locator corresponding to the associated second request data.

[0103] In order to accurately obtain the second request data corresponding to the second response data, when collecting traffic data, a unique Access ID can be set for each response data and request data to facilitate subsequent data tracking. In this way, when processing the second response data, the second request data corresponding to the second response data can be matched through the Access ID.

[0104] After obtaining the uniform resource locator corresponding to the second response data, parse the uniform resource locator to extract the domain name and uniform resource identifier from the uniform resource locator. In specific implementation, the traffic data can be streamed based on the Flink component to extract the domain name host and uniform resource identifier URI in the network address link URL corresponding to the second response data, where the URI is a uniform resource identifier without query parameters and other information. Among them, the API interface corresponding to the second response data can be judged according to the domain name and uniform resource identifier URI to which specific interface in which server the API interface corresponds to.

[0105] In actual applications, when a user accesses a certain functional module in an application, different display pages will be automatically displayed according to the region and performance category. For example, for the second-hand housing query function, the second-hand housing query function is provided through API interface 1. When the user triggers the second-hand housing query function, for different users, the corresponding page will be displayed according to the user's location. Assuming that user A is located in city A, the display page corresponding to the second-hand housing in city A will be displayed, and user B is located in city B, then the display page corresponding to the second-hand housing in city B will be displayed. However, it is essentially provided by a unified second-hand housing query interface, so the API interfaces corresponding to these traffic can be aggregated.

[0106] Then, in another optional embodiment, the specific implementation method of determining the cleaned uniform resource identifier corresponding to the second response data can be: obtaining the uniform resource locator corresponding to the second response data; extracting the attribution class field and the presentation class field from the uniform resource locator; and generating the cleaned uniform resource identifier corresponding to the second response data based on the attribution class field and the presentation class field.

[0107] After the cleaned uniform resource identifier corresponding to each response data is determined, the API interface corresponding to the second response data can be determined according to the cleaned uniform resource identifier corresponding to the second response data.

[0108] In an optional embodiment, the server cluster name associated with the cleaned uniform resource identifier corresponding to the second response data can be determined according to a preset mapping table, and the API interface corresponding to the second response data can be determined according to the server cluster name associated with the cleaned uniform resource identifier corresponding to the second response data.

[0109] After the API interfaces corresponding to the respective second response data are determined, the API interfaces are aggregated to aggregate the sensitive data in the plurality of second response data from the same API interface and generate warning information.

[0110] However, in actual applications, in order to improve processing efficiency, after determining the cleaned uniform resource identifier corresponding to each second response data, the cleaned uniform resource identifier corresponding to each second response data can be aggregated, and then the API interfaces corresponding to the multiple second response data can be determined based on the aggregated uniform resource identifier.

[0111] In specific implementation, an aggregation rule can be set according to each functional module in the application. After determining the cleaned uniform resource identifier corresponding to each second response data, the cleaned uniform resource identifier corresponding to each second response data is aggregated according to the aggregation rule to obtain an aggregated uniform resource identifier. Then, according to a preset mapping table, the server cluster name corresponding to the aggregated uniform resource identifier is determined, and according to the server cluster name, the API interface corresponding to the plurality of second response data is determined.

[0112] Through the above method, the detection results corresponding to multiple second response data from the same API interface can be aggregated together to generate an alarm message, which is sent to the developer's corresponding device, so that the developer can promptly rectify the API interface with the risk of sensitive data leakage according to the alarm information to improve the security of the application.

[0113] In addition, multi-level security alerts can be issued based on the test results and API call status, and the data security officer can be notified in time to carry out governance. When conducting governance, different governance plans can be adopted based on the security level corresponding to the sensitive data in the test results.

[0114] For example, after the alarm information is generated, the ownership of the API assets is identified through the server cluster information, and the data security officer is notified. By optimizing the detection rules, the sensitive data detection rules are further optimized and updated to reduce the false alarm rate. Alternatively, the response data of known normal requests can be whitened. For example: When performing sensitive data detection and processing on the personal real-name authentication module in the application, the special module can be removed and the special module can be whitened to ensure that the traffic data that meets the verification is not falsely reported. The network library signature verification function on the APP side ensures the integrity and authenticity of the data, effectively prevents replay attacks, and improves the security and protection threshold of data during transmission. For API interfaces that are determined to have security risks, remove access to sensitive data or increase access control or take the API interface offline.

[0115] Reverse recall of the confirmed rectified data to ensure the effectiveness of the rectification of similar problems. Specifically, extract the key data fields that have been rectified. Recall the confirmed rectified data through the filtering conditions of the key fields to ensure the effectiveness of the rectification of similar problems.

[0116] The detailed execution process in the embodiment of the present invention can refer to the relevant descriptions in the aforementioned other embodiments and will not be repeated here.

[0117] In order to understand the specific implementation process of the above solution, Figure 4The above scheme is illustrated with examples. In specific implementation, before performing data detection processing on the application, you can first configure the sensitive data detection rules in the corresponding configuration domain, configure the priority corresponding to each sensitive data, configure the aggregation rules, configure the whitelist, and configure the tags used for data classification processing of key-value pairs.

[0118] When the configuration is complete, the traffic data flowing through the application firewall is collected, including request data from the public network and the corresponding response data, and request data from the intranet and the corresponding response data.

[0119] After the traffic data is collected, the collected traffic data can be put into the message queue in sequence according to the collection order. The collected traffic data is preprocessed using the Flink component. Specifically, the response data and request data in each collected traffic data can be associated so that the request data and the corresponding response data have unique identifiers, and the request data corresponding to each response data can be matched according to the corresponding identifier. Among them, the collected traffic data can also be filtered to filter out invalid or redundant data. In addition, the collected traffic data can also be standardized. Since the formats corresponding to the collected response data are different, in order to facilitate sensitive detection processing of the collected traffic data, the traffic data can be converted into standard data and then sensitive detection processing can be performed on the traffic data.

[0120] Next, the detection engine can be used to perform sensitive data detection on the pre-processed traffic data. Among them, the values ​​corresponding to each response data can be subjected to sensitivity detection according to the classification (user information category, business data category, security operation and maintenance category, business management category) and classification (security level), and the corresponding classification labels can be marked. Then, the values ​​in the key-value pairs are subjected to sensitive data detection using a variety of detection methods such as Luhn algorithm, feature matching, rule calculation and natural language processing (NLP) to ensure the accuracy of the detection results.

[0121] After the detection results corresponding to the various response data are obtained, the detection results can be aggregated and alarm information can be generated based on the aggregated detection results.

[0122] In addition, in the data detection schemes for applications provided in the above embodiments, all steps are completed in the data detection device for the application, and the data detection device can be integrated in the client including the application for execution. However, in actual applications, some detection steps in the security detection method for the application can also be executed in another server end, which can be a cloud end, etc.

[0123] As mentioned above, the data detection method of the application provided by the present invention can be executed in the cloud, and this service is provided in a server in the cloud. The cloud can provide the service by providing a service interface to the outside, and the staff can call the service interface to use the corresponding service. The service interface includes software development kits, application program interfaces, etc.

[0124] In view of the solution provided by the embodiment of the present invention, the cloud server can provide a service interface of the data detection service of the application program, and the staff can call the data detection service interface of the application program through the electronic device to trigger a request to the cloud server to call the data detection service interface of the application program. The cloud server responds to the request and executes the following steps:

[0125] Collecting traffic data flowing through an application firewall, where the application firewall is deployed at the front end of the application, the traffic data including first input data;

[0126] According to a data type corresponding to the first response data, convert the first response data into a JSON object to extract key-value pairs contained in the first response data, wherein the first response data is data extracted from the first input data, the data packet size of which meets a preset threshold and does not belong to a simulation environment;

[0127] Determine the classification label corresponding to the key-value pair according to the data label tree corresponding to the application;

[0128] According to the classification label, sensitive data detection is performed on the key-value pair to determine whether the first response data includes sensitive data.

[0129] The detailed execution process can be referred to the related descriptions in other aforementioned embodiments and will not be repeated here.

[0130] Figure 5 A schematic diagram of a data detection device for an application program provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device includes: a collection module 11, an analysis module 12, a determination module 13, and a detection module 14.

[0131] The collection module 11 is used to collect traffic data flowing through the application firewall, where the application firewall is deployed at the front end of the application, and the traffic data includes first response data.

[0132] The parsing module 12 is used to convert the first response data into a JSON object according to the data type corresponding to the first response data, so as to extract the key-value pairs contained in the first response data, wherein the first response data is data extracted from the first input data, the data packet size meets a preset threshold and does not belong to the simulation environment.

[0133] The determination module 13 is used to determine the classification label corresponding to the key-value pair according to the data label tree corresponding to the application.

[0134] The detection module 14 is used to perform sensitive data detection on the key-value pair according to the classification label to determine whether the first response data includes sensitive data.

[0135] Optionally, the traffic data includes multiple response data, and the multiple response data include the first response data. The device also includes a generation module, and the generation module is specifically used to: filter out second response data containing sensitive data from the multiple response data; determine a cleaned uniform resource identifier corresponding to the second response data, wherein the cleaned uniform resource identifier is a uniform resource identifier with redundant information removed; determine an API interface corresponding to the second response data based on the cleaned uniform resource identifier corresponding to the second response data; and generate an alarm message based on the API interface and the sensitive data corresponding to the second response data.

[0136] Optionally, the generation module is specifically used to: obtain a uniform resource locator corresponding to the second response data; parse the uniform resource locator to extract a domain name and a uniform resource identifier from the uniform resource locator; and generate a cleaned uniform resource identifier corresponding to the second response data based on the domain name and the uniform resource identifier.

[0137] Optionally, the generation module is specifically used to: obtain a uniform resource locator corresponding to the second response data; extract an attribution class field and a presentation class field from the uniform resource locator; and generate a cleaned uniform resource identifier corresponding to the second response data based on the attribution class field and the presentation class field.

[0138] Optionally, the generation module is specifically used to: determine, according to a preset mapping table, a server cluster name associated with the cleaned uniform resource identifier corresponding to the second response data; and determine, according to the server cluster name, an API interface corresponding to the second response data.

[0139] Optionally, before converting the first response data into a JSON object according to the data type corresponding to the first response data to extract the key-value pairs contained in the first response data, the parsing module 12 is also used to: determine the data packet size corresponding to the first input data; determine whether the first input data is response data generated by an automated testing tool according to the identifier corresponding to the first input data or the IP address corresponding to the first input data; determine whether the first input data is honeypot environment data according to the identifier corresponding to the first input data or the IP address corresponding to the first input data; if the data packet size is less than a preset threshold, the first input data is not test environment data and the first input data is not honeypot environment data, then determine the first input data as the first response data.

[0140] Optionally, the parsing module 12 is specifically used to: convert the first response data into a JSON string according to a data type corresponding to the first response data; parse the JSON string into a JSON object using a JSON parsing library; and extract key-value pairs contained in the JSON object.

[0141] Optionally, the determination module 13 is specifically used to: determine multiple category tags for data classification of the field value; respectively determine the keywords corresponding to each of the multiple category tags; and determine the data tag tree corresponding to the application based on the multiple category tags and the keywords corresponding to each of the multiple category tags.

[0142] Optionally, the determination module 13 is specifically used to: convert the field value into a vector representation to obtain a first word vector corresponding to the field value; convert the keywords corresponding to each of the multiple category labels into a vector representation to obtain a second word vector corresponding to each keyword; determine the similarity between the first word vector and the second word vector corresponding to each keyword; determine the keywords that match the field value based on the similarity; and determine the classification label corresponding to the field value based on the keywords that match the field value.

[0143] Figure 5 The device shown can execute the data detection method of the application provided in the aforementioned embodiment. The detailed execution process and technical effects are described in the aforementioned embodiment and will not be repeated here.

[0144] In one possible design, the above Figure 5 The structure of the data detection device of the application program shown can be implemented as an electronic device, such as Figure 6As shown, the electronic device may include: a processor 21, a memory 22, and a communication interface 23. The memory 22 stores executable codes, and when the executable codes are executed by the processor 21, the processor 21 can at least implement the data detection method of the application provided in the above-mentioned embodiment.

[0145] In addition, an embodiment of the present invention provides a non-temporary machine-readable storage medium, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor can at least implement the security detection method for the application provided in the aforementioned embodiment.

[0146] In addition, an embodiment of the present invention provides a computer program product having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the security detection method for an application program provided in the aforementioned embodiment.

[0147] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0148] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data detection method for an application, characterized in that: include: Collecting traffic data flowing through an application firewall, where the application firewall is deployed at the front end of the application, the traffic data including first input data; According to a data type corresponding to the first response data, convert the first response data into a JSON object to extract key-value pairs contained in the first response data, wherein the first response data is data extracted from the first input data, the data packet size of which meets a preset threshold and does not belong to a simulation environment; Determine the classification label corresponding to the key-value pair according to the data label tree corresponding to the application; According to the classification label, sensitive data detection is performed on the key-value pair to determine whether the first response data includes sensitive data.

2. The method according to claim 1, characterized in that The traffic data includes a plurality of response data, and the plurality of response data includes the first response data, and the method further includes: Filtering out second response data containing sensitive data from the plurality of response data; Determine a cleaned uniform resource identifier corresponding to the second response data, wherein the cleaned uniform resource identifier is a uniform resource identifier without redundant information; Determine the API interface corresponding to the second response data according to the cleaned uniform resource identifier corresponding to the second response data; Generate warning information based on the API interface and the sensitive data corresponding to the second response data.

3. The method according to claim 2, characterized in that The determining the cleaned uniform resource identifier corresponding to the second response data includes: Obtaining a uniform resource locator corresponding to the second response data; Parsing the uniform resource locator to extract a domain name and a uniform resource identifier from the uniform resource locator; Based on the domain name and the uniform resource identifier, a cleaned uniform resource identifier corresponding to the second response data is generated.

4. The method according to claim 2, characterized in that: The determining the cleaned uniform resource identifier corresponding to the second response data includes: Obtaining a uniform resource locator corresponding to the second response data; Extracting an attribution field and a presentation field from the uniform resource locator; Based on the attribution class field and the presentation class field, a cleaned uniform resource identifier corresponding to the second response data is generated.

5. The method according to claim 4, characterized in that The determining, according to the cleaned uniform resource identifier corresponding to the second response data, the API interface corresponding to the second response data includes: Determine, according to a preset mapping table, a server cluster name associated with the cleaned uniform resource identifier corresponding to the second response data; According to the server cluster name, an API interface corresponding to the second response data is determined.

6. The method according to claim 1, characterized in that Before converting the first response data into a JSON object according to a data type corresponding to the first response data to extract key-value pairs contained in the first response data, the method further includes: Determine a data packet size corresponding to the first input data; determining, according to an identifier corresponding to the first input data or an IP address corresponding to the first input data, whether the first input data is response data generated by an automated testing tool; determining whether the first input data is honeypot environment data according to an identifier corresponding to the first input data or an IP address corresponding to the first input data; If the data packet size is smaller than a preset threshold, the first input data is not test environment data, and the first input data is not honeypot environment data, the first input data is determined as the first response data.

7. The method according to claim 1, characterized in that The converting the first response data into a JSON object according to a data type corresponding to the first response data to extract key-value pairs contained in the first response data includes: According to the data type corresponding to the first response data, convert the first response data into a JSON string; Utilize the JSON parsing library to parse the JSON string into a JSON object; Extract the key-value pairs contained in the JSON object.

8. The method according to claim 1, characterized in that The key-value pair includes a field name and a field value. Before determining the classification label corresponding to the key-value pair according to the data label tree corresponding to the application, the method further includes: Determining a plurality of category labels for classifying data for the field value; Determine keywords corresponding to each of the multiple category tags respectively; Based on the multiple category tags and the keywords corresponding to each of the multiple category tags, a data tag tree corresponding to the application is determined.

9. The method according to claim 8, characterized in that Determining the classification label corresponding to the key-value pair according to the sensitive data label tree corresponding to the application program includes: Convert the field value into a vector representation to obtain a first word vector corresponding to the field value; Convert the keywords corresponding to the multiple category labels into vector representations to obtain a second word vector corresponding to each keyword; Determine the similarity between the first word vector and the second word vector corresponding to each keyword; Determining keywords matching the field value based on the similarity; According to the keywords that match the field value, a classification label corresponding to the field value is determined.

10. An electronic device, characterized in that: include: A memory, a processor, and a communication interface; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the data detection method of the application as described in any one of claims 1 to 9.

11. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the data detection method of the application program according to any one of claims 1 to 9.

Images