A full-link data traceability method, device and electronic equipment

Abstract

The application discloses a full-link data tracing method and device and electronic equipment, and relates to the technical field of data processing, to improve the speed and accuracy of data tracing, the method comprises the following steps: dividing the obtained network flow data into a first network flow data segment and a second network flow data segment; then, extracting respective feature data of a first window in the first network flow data segment and a second window in the second network flow data segment; further, constructing a network link record based on the respective feature data in the first window and the second window, and saving the network link record to a data tracing database; finally, performing data tracing on sensitive data to be traced based on the data tracing database to obtain a data tracing result. Through the above method, the number of repeated link requests based on a rolling time window is detected by using multiple Bloom filters, and the method has the advantages of fast matching speed and small storage space.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a full-link data tracing method and device and electronic equipment. BACKGROUND

[0002] With the rapid development of information technology, data has become one of the most important assets of enterprises and organizations. However, security problems such as data leakage and illegal access follow. In these cases, it is crucial to efficiently and accurately trace data.

[0003] In related technologies, network traffic, log data, access time, operation statement features, and data content information are usually used for link association. However, in a micro-service architecture, a service request may cross multiple application layer network protocols. For example, user A accesses a web application through the HyperText Transfer Protocol (HTTP) protocol, and the web application accesses a backend mysql database through the mysql protocol. Requests and responses within the same protocol can be associated through the TCP layer, but different application layer protocols cannot be associated through the TCP layer. Therefore, if link association is performed based on time sequence and data content correlation, the accuracy of data matching and tracing is low in cross-protocol and high-concurrency scenarios. SUMMARY

[0004] Embodiments of the present application provide a full-link data tracing method and device and electronic equipment to perform data tracing and matching in high-concurrency and high-traffic scenarios, and improve the accuracy of data tracing and matching speed.

[0005] In a first aspect, the embodiments of the present application provide a full-link data tracing method, which includes:

[0006] The obtained network traffic data is divided into a first network traffic data segment and a second network traffic data segment, wherein the first network traffic data segment and the second network traffic data segment each have target identification information;

[0007] Each feature data of a first window in the first network traffic data segment and a second window in the second network traffic data segment is extracted, wherein the feature data is used to represent the operation behavior of the network traffic data segment;

[0008] Based on each feature data in the first window and the second window, a network link record is constructed, and the network link record is saved to a data tracing database, wherein the network link record represents the flow direction of the target network traffic data in the network node

[0009] Based on the data traceability database, data traceability is performed on sensitive data to be traced, and the data traceability results are obtained.

[0010] The above method utilizes multiple Bloom filters to detect the number of times network link records are repeated, achieving high source tracing accuracy and fast matching speed, thus avoiding the problem of low matching accuracy in high-concurrency scenarios of traditional traffic fitting.

[0011] In one optional implementation, a network link record is constructed based on the feature data in the first and second windows, and the network link record is saved to a data traceability database, including:

[0012] Based on the M feature data corresponding to the first window and the K feature data corresponding to the second window, network link record identifiers are generated by cross-referencing them pairwise.

[0013] Based on the feature similarity and / or time correlation of each network link record identifier, network link records are obtained and saved to the data traceability database.

[0014] By performing hash operations on the network link record identifiers using the above method, the occurrence count of network link records can be stored sequentially in each Bloom filter, thus realizing the storage of network link records and facilitating data traceability.

[0015] In one optional implementation, based on the data tracing database, data tracing is performed on the sensitive data to be traced to obtain data tracing results, including:

[0016] Based on the network link record identifier corresponding to the network traffic data to be traced, determine whether the occurrence frequency of the network link record identifier is recorded by the target Bloom filter; wherein, the target Bloom filter is determined based on the network link record matching probability;

[0017] If so, obtain the source tracing result corresponding to the network traffic data to be traced, and output the network link record corresponding to the network traffic data to be traced;

[0018] If not, then starting from the first Bloom filter, determine whether the occurrence count of the network link record identifier is recorded, until the sum of the occurrence counts of the network traffic data to be traced in each Bloom filter is greater than a preset threshold, and output the network link record corresponding to the network traffic data to be traced.

[0019] Using the above method, based on the matching probability of network link records, network link records can be directly searched from the Bloom filter at the corresponding position. If no network link record is found, the network link record can be saved.

[0020] In one optional implementation, after obtaining the tracing results corresponding to the network traffic data to be traced, the method further includes:

[0021] Extract sensitive data from the network traffic data to be traced to obtain the quantity of sensitive data;

[0022] Based on the number of sensitive data items and the number of different sensitive data items, the updated network link record matching probability is obtained.

[0023] By using the above method, network link records are matched based on sensitive data, and the matching probability of network link records is updated, which further improves the accuracy of data tracing.

[0024] In one optional implementation, before extracting sensitive data from the network traffic data to be traced and obtaining the quantity of sensitive data, the method further includes:

[0025] Multiple Bloom filters are created; each Bloom filter is used to manage different categories of sensitive data.

[0026] Multiple Bloom filters are used to record the data identifiers corresponding to different sensitive data.

[0027] By using the methods described above, different Bloom filters were established, enabling the storage of different types of sensitive data.

[0028] Secondly, this application provides a full-link data traceability device, the device comprising:

[0029] The processing module divides the acquired network traffic data into a first network traffic data segment and a second network traffic data segment; wherein the first network traffic data segment and the second network traffic data segment each have target identification information;

[0030] The extraction module extracts feature data from the first window in the first network traffic data segment and the second window in the second network traffic data segment, respectively; wherein, the feature data is used to characterize the operational behavior of the network traffic data segment;

[0031] The construction module is used to build network link records based on the feature data in the first and second windows, and save the network link records to the data traceability database. The network link records represent the flow direction of target network traffic data in network nodes.

[0032] The data tracing module is used to trace the source of sensitive data to be traced based on the data tracing database and obtain the data tracing results.

[0033] In one optional implementation, when constructing network link records based on the feature data in the first and second windows and saving the network link records to the data traceability database, the construction module is specifically used for:

[0034] Based on the M feature data corresponding to the first window and the K feature data corresponding to the second window, network link record identifiers are generated by cross-referencing them pairwise.

[0035] Based on the feature similarity and / or time correlation of each network link record identifier, network link records are obtained and saved to the data traceability database.

[0036] In one optional implementation, when performing data tracing on sensitive data to be traced based on the data tracing database to obtain data tracing results, the tracing module is specifically used for:

[0037] Based on the network link record identifier corresponding to the network traffic data to be traced, determine whether the occurrence frequency of the network link record identifier is recorded by the target Bloom filter; wherein, the target Bloom filter is determined based on the network link record matching probability;

[0038] If so, obtain the source tracing result corresponding to the network traffic data to be traced, and output the network link record corresponding to the network traffic data to be traced;

[0039] If not, then starting from the first Bloom filter, determine whether the occurrence count of the network link record identifier is recorded, until the sum of the occurrence counts of the network traffic data to be traced in each Bloom filter is greater than a preset threshold, and output the network link record corresponding to the network traffic data to be traced.

[0040] In one optional implementation, after obtaining the tracing result corresponding to the network traffic data to be traced, the tracing module is further configured to:

[0041] Extract sensitive data from the network traffic data to be traced to obtain the quantity of sensitive data;

[0042] Based on the number of sensitive data items and the number of different sensitive data items, the updated network link record matching probability is obtained.

[0043] In an optional implementation, before extracting sensitive data from the network traffic data to be traced and obtaining the quantity of sensitive data, the construction module is further configured to:

[0044] Multiple Bloom filters are created; each Bloom filter is used to manage different categories of sensitive data.

[0045] Multiple Bloom filters are used to record the data identifiers corresponding to different sensitive data.

[0046] Thirdly, this application provides an electronic device including a processor and a memory, wherein the memory stores program code that, when executed by the processor, causes the processor to perform the steps of the end-to-end data tracing method described in the first aspect.

[0047] Fourthly, this application provides a computer-readable storage medium including program code that, when run on an electronic device, causes the electronic device to perform the steps of the end-to-end data tracing method described in the first aspect.

[0048] Fifthly, this application provides a computer program product that, when invoked by a computer, causes the computer to execute the steps of the end-to-end data tracing method as described in the first aspect.

[0049] Furthermore, other features and advantages of this application will be set forth in the following description and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0051] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0052] Figure 2 A schematic diagram illustrating the implementation process of a full-link data tracing method provided in this application embodiment;

[0053] Figure 3 A schematic diagram of the structure of a full-link data traceability device provided in an embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of the technical solutions of this application, and not all embodiments. Based on the embodiments recorded in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the technical solutions of this application.

[0056] It should be noted that in the description of this application, "multiple" is understood as "at least two". "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. A connected to B can represent: A and B directly connected, or A and B connected through C. Furthermore, in the description of this application, terms such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0057] Furthermore, the data collection, dissemination, and use in the technical solution of this application all comply with the requirements of relevant national laws and regulations.

[0058] The design concept of the embodiments of this application is briefly introduced below:

[0059] With the rapid development of information technology, data has become one of the most important assets for enterprises and organizations. However, this has also brought with it security problems such as data breaches and unauthorized access. In these situations, efficient and accurate data traceability becomes crucial.

[0060] In related technologies, information such as access time, operation statement characteristics, and accessed data content from network traffic and log data is typically used to establish link associations. However, in a microservice architecture, a service request may cross multiple application layer network protocols. For example, user A accesses a web application via the Hypertext Transfer Protocol (HTTP), while the web application accesses a backend MySQL database via the MySQL protocol. While requests within the same protocol can be associated through the TCP request-response mechanism, requests between different application layer protocols cannot be associated through TCP. Therefore, if link associations are established based on time sequence and data content relevance, the accuracy of data matching and tracing is low in cross-protocol and high-concurrency scenarios.

[0061] In view of this, in order to achieve data tracing in high-concurrency, high-traffic scenarios, this application provides a full-link data tracing method. This method includes: first, using a rolling time window mechanism, dividing network traffic data into a first network traffic data segment and a second network traffic data segment; then, extracting various feature data from the first and second network traffic data segments respectively; further, constructing network link records based on the feature data from the first and second network traffic data segments, and saving the network links to the corresponding Bloom filters; finally, performing data tracing based on the preset Bloom filters to obtain the data tracing results. Through the above method, multiple Bloom filters are used to achieve link request repetition detection and link matching based on a rolling time window. Bloom filters have small storage space and can adapt to high-traffic, high-concurrency network environments.

[0062] The following description, in conjunction with the accompanying drawings, describes the end-to-end data tracing method provided by an exemplary embodiment of this application.

[0063] See Figure 1 The diagram shown is a possible application scenario diagram in the embodiments of this application. The application scenario diagram includes a server 11 and terminal devices 12 (including terminal devices 121, 122... 12n).

[0064] Server 11 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminal device 12 and server 11 can be directly or indirectly connected via wired or wireless communication; this application does not impose any restrictions on this.

[0065] Terminal devices 120 include, but are not limited to, mobile phones, tablets, laptops, desktop computers, e-readers, smart voice interaction devices, smart home appliances, in-vehicle terminals, and other devices; various software, such as applications and mini-programs, can be installed on terminal devices.

[0066] It should be noted that, Figure 1 The examples shown are merely illustrative; in reality, the number of terminal devices 12 and servers 11 is not limited and is not specifically limited in this embodiment.

[0067] See Figure 1 The diagram shown illustrates the implementation flow of a full-link data tracing method provided in this application embodiment. The specific implementation flow of this method is as follows:

[0068] S1: Divide the acquired network traffic data into the first network traffic data segment and the second network traffic data segment.

[0069] In this embodiment, the server can utilize network traffic monitoring tools, such as NetFlow, to monitor network traffic data in real time. These tools can be deployed on critical nodes such as routers and switches to obtain network traffic data flowing through these devices.

[0070] Furthermore, the acquired network traffic data can include network traffic data flowing through both sides of the server, namely: uplink data sent from the client to the server, and downlink data sent from the server to the client.

[0071] Furthermore, to prevent the impact of request-response cycles spanning time windows, a rolling time window mechanism can be used to acquire network traffic data, dividing the acquired network traffic data into a first network traffic data segment and a second network traffic data segment. Specifically, the overlap between windows can be controlled by a preset rolling step size to ensure data continuity and consistency, while reducing data loss or duplication caused by time window division.

[0072] For example, a first network traffic data segment is obtained using a first time period [T, T+△T], and a second network traffic data segment is obtained using a second time period [T+△S, T+△T+△S]. Here, △S is the time offset between time windows. Furthermore, if a request within the first time period takes longer than T+β△T, that request and any requests within the second time period that take longer than T+β△T+△S can be grouped into the next time window to prevent two consecutive requests from crossing windows and becoming unmatched. Here, β is a preset threshold, which can be set within the range of 0-1 according to the actual application scenario.

[0073] It should be noted that each segment of network traffic data has its own target identifier, which can be a Universally Unique Identifier (UUID). This target identifier ensures that all network traffic data segments have unique identification information. Of course, the target identifier can also be any other identifier that gives a network traffic data segment unique identification information.

[0074] By using a rolling time window mechanism, the acquired network traffic data is divided into two segments that are connected end-to-end and do not overlap, which facilitates subsequent link association matching.

[0075] S2: Extract the feature data of the first window in the first network traffic data segment and the second window in the second network traffic data segment respectively.

[0076] In this embodiment of the application, the server can extract various feature data of the first window in the first network traffic data segment and the second window in the second network traffic data segment.

[0077] It should be noted that this feature data can be used to identify the operational behavior of network traffic data segments.

[0078] Specifically, when extracting feature data from the first and second network traffic data segments, it is also necessary to determine the data request type corresponding to each network traffic data segment, such as a web request or a database request. For web requests, HTTP / HTTPS protocol data packets are typically used; therefore, HTTP usually includes a Uniform Resource Locator (URL), request method, request headers, and request body. For database requests, the corresponding Structured Query Language (SQL) can be obtained.

[0079] When extracting feature data, the protocols used by network traffic data can be parsed to identify feature data in the requests, such as the URL corresponding to an HTTP request, or the SQL statements and parameters in a MySQL protocol.

[0080] Therefore, different characteristic data can be obtained for different types of data requests. Specifically, this could be the URL of a web request, the SQL statement of a database request, etc.

[0081] S3: Based on the feature data in the first and second windows, construct network link records and save the network link records to the data traceability database.

[0082] In this embodiment, when extracting the feature data corresponding to the first window and the second window, it is also necessary to establish a network link record. This network link record represents the flow direction of the target network traffic data in the network nodes, such as node A sending data to node B, and node B sending data to node C.

[0083] In one optional implementation, the first window corresponds to M feature data and the second window corresponds to K feature data. Therefore, network link record identifiers can be generated by cross-referencing the M feature data in the first window and the K feature data in the second window pairwise, resulting in M ​​x K network link record identifiers. In other words, each feature data in the first window will generate a network link record identifier together with each feature data in the second window. For example, when generating network link record identifiers by cross-referencing pairwise, feature data A and feature data B can be concatenated, for example, the network link record identifier is A+B. This network link record identifier corresponds one-to-one with a network link record.

[0084] Then, based on the feature similarity and / or temporal correlation of each network link record, the network link record is obtained. Specifically, for example, if a certain data in the first window is close in time to a certain data in the second window, such as node A in the first window sending data A to node B at 9:20:06, while node B in the second window sends data B to node C at 9:20:07, it indicates that data A and data B have a high correlation. Therefore, the link relationship between nodes A, B, and C is obtained, and the corresponding network link record is saved to the data traceability database.

[0085] Furthermore, the Bloom filter maps elements into a bit array using multiple preset hash functions. By mapping elements to multiple different index positions and setting the values ​​at these index positions to 1, if the values ​​at the corresponding multiple index positions are all 1 during a query, then the element is considered to exist in the set.

[0086] Therefore, for the Nth Bloom filter: multiple preset hash functions corresponding to the Nth Bloom filter are used to perform hash operations on each network link record identifier, which can obtain multiple hash operation results corresponding to each network link record identifier. Furthermore, based on the obtained multiple hash operation results, that is, the hash operation result corresponding to the network link record identifier is recorded as 1 at the position corresponding to the Nth Bloom filter.

[0087] It should be noted that if the network link record identifier A+B is appearing for the first time, then the network link record identifier A+B is hashed using multiple hash functions corresponding to the first Bloom filter. Based on the results of these hash operations, the values ​​at the corresponding index positions in the first Bloom filter are set to 1. If the network link record identifier A+B is not appearing for the first time, then after hashing the network link record identifier A+B using multiple hash functions corresponding to the first Bloom filter, it can be determined that the value at the corresponding index position in the first Bloom filter has been set to 1, indicating that the network link record identifier A+B has already been recorded in the first Bloom filter. Further, the network link record identifier A+B is hashed using multiple hash functions corresponding to the second Bloom filter, and it is determined whether the network link record identifier A+B has already been saved in the second Bloom filter. If not, the network link record identifier is saved to the second Bloom filter; if so, this process continues until the third Bloom filter is used for further determination.

[0088] By using the above method and storing relevant network link records through multiple Bloom filters, it is possible to achieve judgment based on the repetition threshold, resulting in fast matching speed.

[0089] S4: Based on the data traceability database, perform data traceability on the sensitive data to be traced and obtain the data traceability results.

[0090] In this embodiment of the application, when tracing the network traffic data to be traced, the network link record identifier corresponding to the network traffic data to be traced is first obtained. Then, the target Bloom filter is determined based on the network link record matching probability.

[0091] For example, if the preset network link record matching probability is 50%, and there are 10 Bloom filters storing network link records, then based on the network link record matching probability and the number of Bloom filters, the target Bloom filter is determined to be the fifth Bloom filter. Further, multiple hash functions corresponding to the fifth Bloom filter are used to perform hash operations on the network link record identifiers corresponding to the network traffic data to be traced, to determine whether the network traffic data to be traced is stored in the fifth Bloom filter. If the fifth Bloom filter stores the network link record identifier corresponding to the network traffic data to be traced, then the tracing is successful and the network link record corresponding to the network traffic data to be traced is output. When a network link corresponding to the network traffic data to be traced is matched, the link relationships of each network node in the network data to be traced, as well as the target identifier information, are recorded and uploaded to the data tracing database.

[0092] If no source tracing result is found for the network traffic data to be traced in the fifth Bloom filter, the search continues starting from the first Bloom filter. Specifically, the method for searching for the source tracing result in the first Bloom filter is the same as the method for searching for the source tracing result from the target Bloom filter. If no source tracing result is found in the first Bloom filter, the network link record identifier corresponding to the network traffic data to be traced is hashed using the multiple preset hash functions corresponding to the first Bloom filter, and the value of the corresponding index position in the first Bloom filter is set to 1. If a source tracing result is found in the first Bloom filter, the search continues from the second Bloom filter. Similarly, if no source tracing result is found in the second Bloom filter, the network link record identifier corresponding to the network traffic data to be traced is hashed using the multiple preset hash functions corresponding to the second Bloom filter, and the value of the corresponding index position in the second Bloom filter is set to 1. If a source tracing result is found in the second Bloom filter, the search continues from the third Bloom filter. The process continues until the sum of the occurrences of the network traffic data to be traced in each Bloom filter exceeds a preset threshold. For example, if the network traffic data to be traced appears in the first through fifth Bloom filters, the corresponding tracing result is output.

[0093] Furthermore, in an optional implementation, multiple Bloom filters are typically set up separately to store sensitive data. By comparing the sensitive data, the accuracy of data tracing is further improved.

[0094] It's worth mentioning that multiple Bloom filters can be created, each used to store different categories of sensitive data. Then, when saving sensitive data, the data type is determined based on the data identifier corresponding to each type of sensitive data, and different types of sensitive data are saved to their respective Bloom filters. By using different Bloom filters, the classification and management of different types of sensitive data can be achieved.

[0095] Specifically, the collected sensitive data is first stored in a preset Bloom filter. When matching sensitive data, sensitive data is extracted from the network traffic data to be traced to obtain the number of sensitive data. Then, based on the number of sensitive data and the number of different sensitive data items, the updated network link matching probability is obtained.

[0096] For example, if a sensitive data item is "ID card number", the count of "ID card number" is 50. Other sensitive data items can be "name", "age", etc., and the count of different sensitive data items can be obtained. Then the updated network link matching probability is 50% * count of sensitive data / count of different sensitive data items + 50%.

[0097] Furthermore, by using a preset hash function, the value of each sensitive data point at the index position corresponding to the preset Bloom filter is calculated and saved.

[0098] By filtering and storing sensitive data, the accuracy of data tracing can be further improved.

[0099] For example, when performing data tracing, assume that there are M different feature data points from node A to node B within the first time window, and N different feature data points from node B to node C. The M feature data points and N feature data points are cross-referenced pairwise to generate M x N records, which are denoted as (Key...). A->B Key B->C )∈Smxn. Where S represents a string.

[0100] Then, for the set (Key) A->B Key B->C For each record in the list, concatenate the strings. Further, use the Nth Bloom filter to check if the string "Key" exists in the Nth Bloom filter. If it does, then concatenate (Key)... A->B Key B->C The record represents the link relationship from node A to node B to node C. If the link does not exist, the operation is performed in the first Bloom filter to check if the string Key exists. If it does not exist, the hash function corresponding to the first Bloom filter is used to calculate the index position of the string Key, and the corresponding position in the first Bloom filter is written as 1. This process continues until the (N-1)th Bloom filter.

[0101] Furthermore, based on the same technical concept, embodiments of this application provide a full-link data traceability device, which is used to implement the above-described method flow of embodiments of this application. See also... Figure 3 As shown, the device includes: a processing module 301, an extraction module 302, a construction module 303, and a traceability module 304, wherein...

[0102] The processing module 301 is used to divide the acquired network traffic data into a first network traffic data segment and a second network traffic data segment; wherein the first network traffic data segment and the second network traffic data segment each have target identification information;

[0103] The extraction module 302 is used to extract each feature data of the first window in the first network traffic data segment and the second window in the second network traffic data segment, respectively; wherein, the feature data is used to characterize the operation behavior of the network traffic data segment;

[0104] The construction module 303 is used to construct network link records based on the feature data in the first window and the second window, and save the network link records to the data traceability database. The network link records represent the flow direction of target network traffic data in network nodes.

[0105] The traceability module 304 is used to perform data traceability on sensitive data to be traced based on the data traceability database, and obtain data traceability results.

[0106] In an optional implementation, when constructing network link records based on the feature data in the first and second windows and saving the network link records to the data traceability database, the construction module 303 is specifically used for:

[0107] Based on the M feature data corresponding to the first window and the K feature data corresponding to the second window, network link record identifiers are generated by cross-referencing them pairwise.

[0108] Based on the feature similarity and / or time correlation of each network link record identifier, network link records are obtained and saved to the data traceability database.

[0109] In an optional implementation, when performing data tracing on sensitive data to be traced based on the data tracing database to obtain data tracing results, the tracing module 304 is specifically used for:

[0110] Based on the network link record identifier corresponding to the network traffic data to be traced, determine whether the occurrence frequency of the network link record identifier is recorded by the target Bloom filter; wherein, the target Bloom filter is determined based on the network link record matching probability;

[0111] If so, obtain the source tracing result corresponding to the network traffic data to be traced, and output the network link record corresponding to the network traffic data to be traced;

[0112] If not, then starting from the first Bloom filter, determine whether the occurrence count of the network link record identifier is recorded, until the sum of the occurrence counts of the network traffic data to be traced in each Bloom filter is greater than a preset threshold, and output the network link record corresponding to the network traffic data to be traced.

[0113] In an optional implementation, after obtaining the tracing result corresponding to the network traffic data to be traced, the tracing module 304 is further configured to:

[0114] Extract sensitive data from the network traffic data to be traced to obtain the quantity of sensitive data;

[0115] Based on the number of sensitive data items and the number of different sensitive data items, the updated network link record matching probability is obtained.

[0116] In an optional implementation, before extracting sensitive data from the network traffic data to be traced and obtaining the quantity of sensitive data, the construction module 303 is further configured to:

[0117] Multiple Bloom filters are created; each Bloom filter is used to manage different categories of sensitive data.

[0118] Multiple Bloom filters are used to record the data identifiers corresponding to different sensitive data.

[0119] Based on the same technical concept, embodiments of this application also provide an electronic device that can implement the end-to-end data traceability method provided in the above embodiments of this application. In one embodiment, the electronic device may be a server, a terminal device, or other electronic devices. See also... Figure 4 As shown, the electronic device may include:

[0120] At least one processor 401 and a memory 402 connected to at least one processor 401. In this embodiment, the specific connection medium between the processor 401 and the memory 402 is not limited. Figure 4 The example shown is the connection between processor 401 and memory 402 via bus 400. Bus 400 is... Figure 4 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. The 400 bus can be divided into address bus, data bus, control bus, etc., for ease of representation. Figure 4 The term is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, processor 401 can also be called a controller; there is no restriction on the name.

[0121] In this embodiment, memory 402 stores instructions executable by at least one processor 401. By executing the instructions stored in memory 402, at least one processor 401 can execute the end-to-end data traceability method described above. Processor 401 can implement... Figure 4 The functions of each module in the device shown.

[0122] The processor 401 is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory 402 and calling data stored in memory 402, the processor can perform various functions and process data, thereby monitoring the device as a whole.

[0123] In one possible design, processor 401 may include one or more processing units. Processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 401. In some embodiments, processor 401 and memory 402 may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0124] Processor 401 can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the end-to-end data traceability method disclosed in the embodiments of this application can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0125] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 502 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 402 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 402 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0126] By designing and programming the processor 401, the code corresponding to the frequency offset estimation method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute it during operation. Figure 2 The illustrated embodiment presents the steps of a full-link data tracing method. How to design and program the processor 401 is a technique well-known to those skilled in the art and will not be described further here.

[0127] Based on the same inventive concept, embodiments of this application also provide a storage medium storing computer instructions that, when executed on a computer, cause the computer to perform a full-link data traceability method described above.

[0128] In some possible implementations, this application also provides a method for end-to-end data traceability that can be implemented as a program product including program code. When the program product is run on a device, the program code is used to cause the control device to perform the steps in an end-to-end data traceability method according to various exemplary embodiments of this application as described above.

[0129] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0130] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0131] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0133] Program code for performing the operations of this application can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0134] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0135] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for end-to-end data traceability, characterized in that, The method includes: The acquired network traffic data is divided into a first network traffic data segment and a second network traffic data segment; wherein, the first network traffic data segment and the second network traffic data segment each have target identification information; Each feature data of the first window in the first network traffic data segment and the second window in the second network traffic data segment is extracted respectively; wherein, the feature data is used to characterize the operational behavior of the network traffic data segment; Based on the feature data in the first window and the second window, a network link record is constructed and saved to the data traceability database, including: Based on the M feature data corresponding to the first window and the K feature data corresponding to the second window, network link record identifiers are generated by cross-referencing them pairwise. Based on the feature similarity and / or time correlation of each network link record identifier, network link records are obtained and saved to the data traceability database; Based on the aforementioned data tracing database, data tracing is performed on the sensitive data to be traced, and the data tracing results are obtained, including: Based on the network link record identifier corresponding to the network traffic data to be traced, determine whether the occurrence frequency of the network link record identifier is recorded by the target Bloom filter; wherein, the target Bloom filter is determined based on the network link record matching probability; If so, the source tracing result corresponding to the network traffic data to be traced is obtained, and the network link record corresponding to the network traffic data to be traced is output. If not, then starting from the first Bloom filter, determine whether the occurrence count of the network link record identifier has been recorded, until the sum of the occurrence counts of the network traffic data to be traced in each Bloom filter is greater than a preset threshold, and output the network link record corresponding to the network traffic data to be traced.

2. The method as described in claim 1, characterized in that, After obtaining the tracing result corresponding to the network traffic data to be traced, the method further includes: Sensitive data extraction is performed on the network traffic data to be traced to obtain the quantity of sensitive data; Based on the quantity of the sensitive data and the quantity of different sensitive data items, the updated network link record matching probability is obtained.

3. The method as described in claim 2, characterized in that, Before extracting sensitive data from the network traffic data to be traced and obtaining the quantity of the sensitive data, the method further includes: Multiple Bloom filters are established; wherein each of the multiple Bloom filters is used to manage different categories of sensitive data. The aforementioned multiple Bloom filters are used to record the data identifiers corresponding to different sensitive data.

4. A full-link data traceability device, characterized in that, The device includes: The processing module is used to divide the acquired network traffic data into a first network traffic data segment and a second network traffic data segment; wherein the first network traffic data segment and the second network traffic data segment each have target identification information; The extraction module is used to extract feature data from the first window in the first network traffic data segment and the second window in the second network traffic data segment, respectively; wherein, the feature data is used to characterize the operational behavior of the network traffic data segment. A construction module is used to construct network link records based on various feature data in the first window and the second window, and save the network link records to a data traceability database, including: Based on the M feature data corresponding to the first window and the K feature data corresponding to the second window, network link record identifiers are generated by cross-referencing them pairwise. Based on the feature similarity and / or time correlation of each network link record identifier, network link records are obtained and saved to the data traceability database; The data tracing module is used to perform data tracing on sensitive data to be traced based on the data tracing database, and obtain data tracing results, including: Based on the network link record identifier corresponding to the network traffic data to be traced, determine whether the occurrence frequency of the network link record identifier is recorded by the target Bloom filter; wherein, the target Bloom filter is determined based on the network link record matching probability; If so, the source tracing result corresponding to the network traffic data to be traced is obtained, and the network link record corresponding to the network traffic data to be traced is output. If not, then starting from the first Bloom filter, determine whether the occurrence count of the network link record identifier has been recorded, until the sum of the occurrence counts of the network traffic data to be traced in each Bloom filter is greater than a preset threshold, and output the network link record corresponding to the network traffic data to be traced.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1-3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-3.

Images