Data monitoring method, system, device and medium based on API traffic data

Abstract

The present application discloses a data monitoring method, system, device and medium based on API traffic data, which mainly relates to the field of data monitoring technology, and is used to solve the problem that the existing solutions cannot detect the risk of illegal and over-limit calls. It includes: regularly collecting business traffic; identifying the comparison API belonging to the data provider from the identified business traffic, and screening out the first traffic data belonging to the same comparison API; obtaining the first transmission data between the data access party and the data provider from the first traffic data, and obtaining the second transmission data between the data access party and the data accessor; performing a similarity comparison on the first transmission data and the second transmission data to obtain the similarity; obtaining the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold, and comparing it with the API access IP list and the authorized API list filed by the data provider to determine whether there is an illegal call or over-limit access.

Description

Technical Field

[0001] The present application relates to the field of data monitoring technology, and in particular to a data monitoring method, system, device and medium based on API traffic data. Background Art

[0002] As digital system transformation continues to deepen, system integration, sharing and exchange policies are introduced, and data security risks are highlighted in the use of key information facilities such as networks, cloud platforms, and information sharing platforms at all levels with data as the center. The current data sharing method is mainly open to users in the form of web API (Application Programming Interface). Therefore, API security has become one of the important directions of data security.

[0003] The existing data call scheme is: data provider A authorizes data access party B to access A's data through the API server. D is the access terminal. The data flow can be represented by the API call link. In order to avoid data leakage caused by data flow, the existing solutions are authorization whitelist mechanism and security key measures and identity authentication and authorization mechanism. However, in actual business scenarios. B may also have the role of data provider at the same time. Due to the needs of their respective businesses, they privately encapsulate the interface in the API authorized by B for access and provide it to C outside B. That is, B sends part or all of A's API response data to C through the interface response. This call will cause data security leakage problems in the API accessed by B. When C does not have the API access rights in B, it is an illegal call. If C has some of B's ​​API access rights, it is an over-limit call.

[0004] Therefore, there is an urgent need for a data monitoring method, system, device and medium based on API traffic data to solve the problem that existing solutions cannot detect violations and over-limit call risks. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present application provides a data monitoring method, system, device and medium based on API traffic data to solve the problem that the existing solutions are unable to detect violations and over-limit call risks.

[0006] In a first aspect, the present application provides a data monitoring method based on API traffic data, the method comprising: collecting business traffic at regular intervals; wherein the terminals corresponding to the business traffic include: a data provider, a data access party, and a data accessor; based on the REST API protocol specification, identifying the API, domain name field, IP address, destination port field, URL field (Uniform Resource Locator), request method field, and return data type field corresponding to the business traffic, and then identifying the comparison API belonging to the data provider from the identified business traffic, and filtering out the first traffic data belonging to the same comparison API from the business traffic; based on the IP addresses of the data provider and the data access party, obtaining the first transmission data between the data access party and the data provider, and obtaining the second transmission data between the data access party and the data accessor from the first traffic data; performing a similarity comparison on the first transmission data and the second transmission data to obtain a similarity; obtaining the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold, and comparing them with the API access IP list and the authorized API list filed by the data provider to determine whether there is an illegal call or excessive access.

[0007] Furthermore, the business traffic is collected regularly, specifically including: collecting the business traffic regularly through a configured reverse proxy or gateway.

[0008] Furthermore, the first transmission data includes the request message received by the data provider and the response message sent by the data provider; the second transmission data includes the request message received by the data access party and the response message sent by the data access party; wherein, the request message includes the request URL, request message data and the request data packet, and the response message includes the response URL, response message data and the data packet of the response message.

[0009] Further, a similarity comparison is performed on the first transmission data and the second transmission data to obtain a similarity, specifically including: calculating the similarity between the request URL received by the data provider and the request URL received by the data access party, denoted as S0; calculating the similarity between the request message data received by the data provider and the request message data received by the data access party, denoted as S1; calculating the ratio of the size of the request data packet received by the data provider to the size of the request data packet received by the request message data, denoted as S2; calculating the ratio of the number of request data packets received by the data provider to the number of request data packets received by B, denoted as S3; calculating The text similarity between the response URL sent by the data provider and the response URL sent by the data access party to the data accessor is recorded as S4; the text similarity between the response message data sent by the data provider and the response message data sent by the data access party to the data accessor is calculated, recorded as S5; the data packet size ratio of the response message sent by the data provider to the data packet size ratio of the response message sent by the data provider to the data packet size ratio of the response message sent by the data access party to the data accessor is calculated, recorded as S6; the data packet number ratio of the response message sent by the data provider to the data packet number ratio of the response message sent by the data access party to the data accessor is calculated, recorded as S7; according to the formula:

[0010] , calculate and obtain the similarity; wherein Wi is a preset similarity weight value;

[0011] and .

[0012] Furthermore, the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold are obtained, and compared with the API access IP list and authorized API list filed by the data provider to determine whether there is any illegal call or excessive access, specifically including: determining whether the access IP of the data accessor exists in the API access IP list. When the access IP of the data accessor is not in the API access IP list, it is determined that the data accessor has made illegal access and the data access party has provided an illegal call; when the access IP of the data accessor exists in the API access IP list, it is determined whether the API corresponding to the data accessor exists in the authorized API list. When the access IP of the data accessor is not in the authorized API list, it is determined that the data accessor has made excessive access.

[0013] Furthermore, after determining whether there is an illegal call or excessive access, the method also includes: when there is an illegal call or excessive access, the abnormal behavior is displayed on a preset interface.

[0014] In the second aspect, the present application provides a data monitoring system based on API traffic data, the system comprising: a collection module for periodically collecting business traffic; wherein the terminals corresponding to the business traffic include: a data provider, a data access party, and a data accessor; a screening module for identifying the API, domain name field, IP address, destination port field, URL field, request method field, and return data type field corresponding to the business traffic based on the REST API protocol specification, and then identifying the comparison API belonging to the data provider from the identified business traffic, and screening out the first traffic data belonging to the same comparison API from the business traffic; an acquisition module for acquiring the first transmission data between the data access party and the data provider and the second transmission data between the data access party and the data accessor from the first traffic data based on the IP addresses of the data provider and the data access party; an acquisition module for performing a similarity comparison on the first transmission data and the second transmission data to obtain a similarity; a determination module for acquiring the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold, and comparing them with the API access IP list and the authorized API list filed by the data provider to determine whether there is an illegal call or excessive access.

[0015] Furthermore, the determination module includes a determination unit for determining whether the data accessor's access IP exists in the API access IP list. When the data accessor's access IP does not exist in the API access IP list, it is determined that the data accessor has made illegal access and the data access party has provided an illegal call. When the data accessor's access IP exists in the API access IP list, it is determined whether the API corresponding to the data accessor exists in the authorized API list. When the data accessor's access IP does not exist in the authorized API list, it is determined that the data accessor has exceeded the limit of access.

[0016] In a third aspect, the present application provides a data monitoring device based on API traffic data, the device comprising: a processor; and a memory on which executable code is stored, and when the executable code is executed, the processor executes a data monitoring method based on API traffic data such as any one of the above items.

[0017] In a fourth aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, which when executed implement a data monitoring method based on API traffic data as described above.

[0018] Those skilled in the art can understand that the present application has at least the following beneficial effects:

[0019] This application detects whether the data access party sends the data provider's data to the data access party by comparing the similarity between the first transmission data between the data access party and the data provider and the second transmission data between the data access party and the data accessor. By obtaining the access IP and API of the data accessor corresponding to the second transmission data with a similarity greater than a preset threshold, and comparing it with the API access IP list and authorized API list filed by the data provider, it is possible to determine whether there is an illegal call or excessive access. This solves the problem that the existing solution cannot detect the risk of illegal and excessive calls. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Some embodiments of the present disclosure are described below with reference to the accompanying drawings, in which:

[0021] Figure 1 This is a flow chart of a data monitoring method based on API traffic data provided in an embodiment of the present application.

[0022] Figure 2 This is a schematic diagram of the internal structure of a data monitoring system based on API traffic data provided in an embodiment of the present application.

[0023] Figure 3 This is a schematic diagram of the internal structure of a data monitoring device based on API traffic data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not mean that the present disclosure can only be implemented through the preferred embodiments. The preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should still fall within the protection scope of the present disclosure.

[0025] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0026] The technical solution proposed in the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0027] The present application embodiment provides a data monitoring method based on API traffic data, such as Figure 1 As shown, the method provided in the embodiment of the present application mainly includes the following steps:

[0028] Step 110: Collect service traffic at regular intervals.

[0029] It should be noted that the terminals corresponding to the business traffic include: data providers, data access parties, and data accessors.

[0030] The periodic collection of business traffic may specifically include: periodic collection of business traffic through a configured reverse proxy or gateway.

[0031] Step 120: Based on the REST API protocol specification, identify the API, domain name field, IP address, destination port field, URL field, request method field, and return data type field corresponding to the business traffic, and then identify the comparison API belonging to the data provider from the identified business traffic, and filter out the first traffic data belonging to the same comparison API from the business traffic.

[0032] It should be noted that the scheme for identifying the API, domain name field, IP address, destination port field, URL field, request method field and return data type field corresponding to the business traffic can be implemented by the existing technology, and this application does not limit this.

[0033] Step 130: Based on the IP addresses of the data provider and the data accessor, first transmission data between the data accessor and the data provider and second transmission data between the data accessor and the data accessor are obtained from the first traffic data.

[0034] It should be noted that the first transmission data may include the request message received by the data provider and the response message sent by the data provider; the second transmission data may include the request message received by the data access party and the response message sent by the data access party; wherein the request message includes the request URL, request message data and request data packet, and the response message includes the response URL, response message data and the data packet of the response message.

[0035] Step 140: perform a similarity comparison on the first transmission data and the second transmission data to obtain a similarity.

[0036] This step can be specifically as follows:

[0037] The similarity between the request URL received by the data provider and the request URL received by the data access party is calculated and recorded as S0.

[0038] The similarity between the request message data received by the data provider and the request message data received by the data access party is calculated and recorded as S1.

[0039] Calculate the ratio of the size of the request data packet received by the data provider to the size of the request data packet received by the request message data, and record it as S2.

[0040] Calculate the ratio of the number of request data packets received by the data provider to the number of request data packets received by B, and record it as S3.

[0041] The text similarity between the response URL sent by the data provider and the response URL sent by the data access party to the data accessor is calculated, which is recorded as S4.

[0042] The text similarity between the response message data sent by the data provider and the response message data sent by the data access party to the data accessor is calculated, which is recorded as S5.

[0043] The ratio of the data packet size of the response message sent by the data provider to the data packet size of the response message sent by the data access party to the data accessor is calculated and recorded as S6.

[0044] The ratio of the number of data packets in the response message sent by the data provider to the number of data packets in the response message sent by the data accessor is calculated and recorded as S7.

[0045] According to the formula:

[0046] , calculate and obtain the similarity; wherein Wi is a preset similarity weight value.

[0047] and .

[0048] It should be noted that the text similarity calculation scheme can be implemented by existing technologies, and this application does not limit this.

[0049] Step 150: Obtain the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold, and compare them with the API access IP list and authorized API list filed by the data provider to determine whether there is any illegal call or excessive access.

[0050] This step can be specifically as follows: determine whether the data accessor's access IP exists in the API access IP list. When the data accessor's access IP does not exist in the API access IP list, determine that the data accessor has made illegal access and the data access party has provided illegal calls; when the data accessor's access IP exists in the API access IP list, determine whether the API corresponding to the data accessor exists in the authorized API list. When the data accessor's access IP does not exist in the authorized API list, determine that the data accessor has exceeded the limit of access.

[0051] In addition, after determining whether there is any illegal call or excessive access, the method further includes: when there is any illegal call or excessive access, displaying the abnormal behavior on a preset interface.

[0052] besides, Figure 2 A data monitoring system based on API traffic data is provided in an embodiment of the present application. Figure 2 As shown, the system provided in the embodiment of the present application mainly includes:

[0053] The collection module 210 is used to collect service traffic at regular intervals.

[0054] Among them, the terminals corresponding to the business traffic include: data providers, data access parties, and data accessors.

[0055] The screening module 220 is used to identify the API, domain name field, IP address, destination port field, URL field, request method field and return data type field corresponding to the business traffic based on the REST API protocol specification, and then identify the comparison API belonging to the data provider from the identified business traffic, and screen out the first traffic data belonging to the same comparison API from the business traffic.

[0056] The acquisition module 230 is used to acquire the first transmission data between the data access party and the data provider and the second transmission data between the data access party and the data accessor from the first traffic data based on the IP addresses of the data provider and the data access party.

[0057] The obtaining module 240 is used to perform a similarity comparison on the first transmission data and the second transmission data to obtain a similarity.

[0058] Determination module 250 is used to obtain the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold, and compare them with the API access IP list and authorized API list filed by the data provider to determine whether there is any illegal call or excessive access.

[0059] The determination module 250 includes a determination unit, which is used to determine whether the access IP of the data accessor exists in the API access IP list. When the access IP of the data accessor does not exist in the API access IP list, it is determined that the data accessor has made illegal access and the data access party has provided an illegal call; when the access IP of the data accessor exists in the API access IP list, it is determined whether the API corresponding to the data accessor exists in the authorized API list. When the access IP of the data accessor is not in the authorized API list, it is determined that the data accessor has exceeded the limit of access.

[0060] The above is a method embodiment of the present application. Based on the same inventive concept, the present application embodiment also provides a data monitoring device based on API traffic data. Figure 3 As shown, the device includes: a processor; and a memory on which executable codes are stored. When the executable codes are executed, the processor executes a data monitoring method based on API traffic data as described in the above embodiment.

[0061] Specifically, the server side collects business traffic at regular intervals; wherein, the terminals corresponding to the business traffic include: data providers, data access parties, and data accessors; based on the REST API protocol specification, the API, domain name field, IP address, destination port field, URL field, request method field, and return data type field corresponding to the business traffic are identified, and then the comparison API belonging to the data provider is identified from the identified business traffic, and the first traffic data belonging to the same comparison API is screened out from the business traffic; based on the IP addresses of the data provider and the data access party, the first transmission data between the data access party and the data provider is obtained from the first traffic data, and the second transmission data between the data access party and the data accessor is obtained; a similarity comparison is performed on the first transmission data and the second transmission data to obtain the similarity; the access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than a preset threshold are obtained, and compared with the API access IP list and the authorized API list filed by the data provider to determine whether there is any illegal call or excessive access.

[0062] In addition, an embodiment of the present application further provides a non-volatile computer storage medium on which executable instructions are stored. When the executable instructions are executed, a data monitoring method based on API traffic data as described above is implemented.

[0063] So far, the technical solutions of the present disclosure have been described in combination with the above multiple embodiments, but it is easy for those skilled in the art to understand that the protection scope of the present disclosure is not limited to these specific embodiments. Without departing from the technical principles of the present disclosure, those skilled in the art can split and combine the technical solutions in the above-mentioned various embodiments, and can also make equivalent changes or replacements to the relevant technical features. Any changes, equivalent replacements, improvements, etc. made within the technical concept and / or technical principle of the present disclosure will fall within the protection scope of the present disclosure.

Claims

1. A data monitoring method based on API traffic data, characterized in that: The method comprises: Collect business traffic regularly; the terminals corresponding to the business traffic include: data providers, data access parties, and data accessors; Based on the REST API protocol specification, the API, domain name field, IP address, destination port field, URL field, request method field, and return data type field corresponding to the business traffic are identified, and then the comparison API belonging to the data provider is identified from the identified business traffic, and the first traffic data belonging to the same comparison API is filtered out from the business traffic; Based on the IP addresses of the data provider and the data accessor, first transmission data between the data accessor and the data provider and second transmission data between the data accessor and the data accessor are obtained from the first traffic data; Performing a similarity comparison on the first transmission data and the second transmission data to obtain a similarity; specifically comprising: Calculate the similarity between the request URL received by the data provider and the request URL received by the data access party, denoted as S0; Calculate the similarity between the request message data received by the data provider and the request message data received by the data access party, denoted as S1; Calculate the ratio of the size of the request data packet received by the data provider to the size of the request data packet received by the data access party, denoted as S2; Calculate the ratio of the number of request data packets received by the data provider to the number of request data packets received by the data access party, denoted as S3; Calculate the text similarity between the response URL sent by the data provider and the response URL sent by the data access party to the data accessor, denoted as S4; Calculate the text similarity between the response message data sent by the data provider and the response message data sent by the data access party to the data accessor, denoted as S5; Calculate the ratio of the data packet size of the response message sent by the data provider to the data packet size of the response message sent by the data access party to the data accessor, denoted as S6; Calculate the ratio of the number of data packets in the response message sent by the data provider to the number of data packets in the response message sent by the data access party to the data accessor, and record it as S7; According to the formula: , calculate and obtain the similarity; Wherein, Wi is the preset similarity weight value; and ; The access IP and API of the data accessor corresponding to the second transmission data whose similarity is greater than the preset threshold are obtained, and compared with the API access IP list and authorized API list filed by the data provider to determine whether there is any illegal call or excessive access.

2. The data monitoring method based on API traffic data according to claim 1 is characterized in that: Regularly collect business traffic, including: Business traffic is collected regularly through the configured reverse proxy or gateway.

3. The data monitoring method based on API traffic data according to claim 1 is characterized in that: The first transmission data includes the request message received by the data provider and the response message sent by the data provider; the second transmission data includes the request message received by the data access party and the response message sent by the data access party; wherein, the request message includes the request URL, request message data and request data packet, and the response message includes the response URL, response message data and the data packet of the response message.

4. The data monitoring method based on API traffic data according to claim 1 is characterized in that: Obtain the access IP and API of the data accessor corresponding to the second transmission data with a similarity greater than a preset threshold, and compare them with the API access IP list and authorized API list filed by the data provider to determine whether there is any illegal call or excessive access, including: Determine whether the data accessor's access IP address is in the API access IP list. If the data accessor's access IP address is not in the API access IP list, determine that the data accessor has made an illegal access and the data access party has made an illegal call. When the access IP of the data accessor exists in the API access IP list, it is determined whether the API corresponding to the data accessor exists in the authorized API list. When the API corresponding to the data accessor is not in the authorized API list, it is determined that the data accessor has exceeded the access limit.

5. The data monitoring method based on API traffic data according to claim 1 is characterized in that: After determining whether there is an illegal call or excessive access, the method further includes: When there are illegal calls or excessive access, the abnormal behavior will be displayed on the preset interface.

6. A data monitoring system based on API traffic data, characterized in that: The system comprises: The collection module is used to collect business traffic at regular intervals; wherein the terminals corresponding to the business traffic include: data providers, data access parties, and data accessors; A screening module is used to identify the API, domain name field, IP address, destination port field, URL field, request method field and return data type field corresponding to the business traffic based on the REST API protocol specification, and then identify the comparison API belonging to the data provider from the identified business traffic, and screen out the first traffic data belonging to the same comparison API from the business traffic; An acquisition module, used to acquire first transmission data between the data access party and the data provider, and acquire second transmission data between the data access party and the data accessor from the first traffic data based on the IP addresses of the data provider and the data access party; The obtaining module is used to compare the first transmission data with the second transmission data to obtain the similarity; specifically comprising: Calculate the similarity between the request URL received by the data provider and the request URL received by the data access party, denoted as S0; Calculate the similarity between the request message data received by the data provider and the request message data received by the data access party, denoted as S1; Calculate the ratio of the size of the request data packet received by the data provider to the size of the request data packet received by the data access party, denoted as S2; Calculate the ratio of the number of request data packets received by the data provider to the number of request data packets received by the data access party, denoted as S3; Calculate the text similarity between the response URL sent by the data provider and the response URL sent by the data access party to the data accessor, denoted as S4; Calculate the text similarity between the response message data sent by the data provider and the response message data sent by the data access party to the data accessor, denoted as S5; Calculate the ratio of the data packet size of the response message sent by the data provider to the data packet size of the response message sent by the data access party to the data accessor, denoted as S6; Calculate the ratio of the number of data packets in the response message sent by the data provider to the number of data packets in the response message sent by the data access party to the data accessor, and record it as S7; According to the formula: , calculate and obtain the similarity; Wherein, Wi is the preset similarity weight value; and ; The determination module is used to obtain the access IP and API of the data visitor corresponding to the second transmission data whose similarity is greater than a preset threshold, and compare them with the API access IP list and authorized API list filed by the data provider to determine whether there is any illegal call or excessive access.

7. The data monitoring system based on API flow data according to claim 6 is characterized in that: The determination module comprises a determination unit, Used to determine whether the data accessor's access IP address is in the API access IP list. If the data accessor's access IP address is not in the API access IP list, the data accessor is determined to have made an illegal access and the data access party has provided an illegal call. When the access IP of the data accessor exists in the API access IP list, it is determined whether the API corresponding to the data accessor exists in the authorized API list. When the API corresponding to the data accessor is not in the authorized API list, it is determined that the data accessor has exceeded the access limit.

8. A data monitoring device based on API traffic data, characterized in that: The device comprises: processor; and a memory having executable codes stored thereon, which, when executed, enables the processor to execute a data monitoring method based on API traffic data as described in any one of claims 1-5.

9. A non-volatile computer storage medium, characterized in that: Computer instructions are stored thereon, and when the computer instructions are executed, the data monitoring method based on API traffic data as described in any one of claims 1-5 is implemented.

Images