Data processing method and device, equipment and storage medium
By processing buried point data through symmetric encryption and unified storage, the low efficiency and security issues of distributed buried point data processing are solved, and efficient and secure data acquisition and storage are achieved.
Patent Information
- Application Number
- CN202211175560.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-26
- Publication Date
- 2025-10-24
- Estimated Expiration
- 2042-09-26
AI Technical Summary
In existing technologies, distributed point-of-sale data processing is inefficient, real-time storage requires high database processing capabilities, and is prone to service downtime. Distributed log storage makes it difficult to collect and obtain point-of-sale data, sensitive information storage is unsafe, and incremental logs are difficult to distinguish and deduplicate.
A symmetric key is generated through a symmetric encryption algorithm to encrypt sensitive data in the tracking data. The symmetric key is then encrypted using the server public key to form encrypted information, which is then stored in a unified tracking data log folder. The tracking data is obtained using the Internet access load balancing system, and the directory folder of the website service node name is maintained to ensure the integrity and security of data acquisition.
It improves the transmission security and processing efficiency of buried point data, solves the problems of missed traversal and repeated traversal, ensures the security of sensitive data, and improves the efficiency and accuracy of data acquisition.
Smart Images

Figure CN115580454B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of data processing, and particularly relates to a data processing method and device, equipment and a storage medium. BACKGROUND
[0002] With the continuous development of information technology, the demand for data processing is increasing. In the related art, an Internet platform can store and obtain embedded point data in the following two ways. One way is to store the received embedded point data in a database through a background application, so as to process and analyze the embedded point data in the database in the form of separate batch calculation at a fixed time. The other way is to store the embedded point data in the form of a log record in a disk through a server, and process the embedded point log data stored in the disk through a timing task.
[0003] However, the former stores data in real time in the database, which greatly affects the processing capacity of the database and the throughput of the background service, and is prone to cause service downtime. The latter is difficult to collect and obtain the embedded point data logs because the embedded point data logs are stored in each independent server and the server is uncertain when forwarding the embedded point data acquisition request. SUMMARY
[0004] The present application provides a data processing method, device, equipment and storage medium to solve the problem of low processing efficiency of distributed embedded point data in the prior art.
[0005] In a first aspect, an embodiment of the present application provides a data processing method applied to a server, comprising:
[0006] receiving a URL request sent by a user end, the URL request comprising encrypted information, the encrypted information being information of embedded point data including sensitive data encrypted by a symmetric key, the symmetric key being calculated by a symmetric encryption algorithm;
[0007] splicing a website service node name of each website service node in N website service nodes of the server with the URL request respectively to obtain M target requests, N being a positive integer and M being a positive integer;
[0008] based on an embedded point data log record directory, obtaining a website service node subdirectory corresponding to the website service node name of each target request in the M target requests respectively, each website service node subdirectory corresponding to each embedded point data log file;
[0009] storing each target request to an embedded point data log folder corresponding to the website service node subdirectory.
[0010] In a second aspect, an embodiment of the present application provides a data processing method applied to a user end, comprising:
[0011] The buried point data is result data after data buried point processing.
[0012] In the case where the buried point data includes sensitive data, a symmetric key is generated by a symmetric encryption algorithm.
[0013] The sensitive data in the buried point data is encrypted by the symmetric key, and the symmetric key is encrypted by the public key of the server, to obtain encryption information corresponding to the buried point data.
[0014] The URL request is used to request the server to store the buried point data to a preset buried point data log folder.
[0015] In a third aspect, an embodiment of the present application provides a data processing apparatus applied to a server, comprising:
[0016] The receiving module is configured to receive a URL request sent by the user end, wherein the URL request includes encryption information, the encryption information is information of the buried point data including sensitive data encrypted by a symmetric key, and the symmetric key is calculated by a symmetric encryption algorithm.
[0017] The splicing module is configured to splice the website service node name of each website service node in N website service nodes of the server with the URL request respectively, to obtain M target requests, wherein N is a positive integer, and M is a positive integer.
[0018] The obtaining module is configured to obtain a website service node subdirectory corresponding to the website service node name of each target request in the M target requests based on a buried point data log record directory, and each website service node subdirectory corresponds to each buried point data log file.
[0019] The storage module is configured to store each target request to a buried point data log folder corresponding to the website service node subdirectory.
[0020] In a fourth aspect, an embodiment of the present application provides a data processing apparatus applied to a user end, comprising:
[0021] The obtaining module is configured to obtain buried point data, wherein the buried point data is result data after data buried point processing.
[0022] The generating module is configured to generate a symmetric key by a symmetric encryption algorithm in the case where the buried point data includes sensitive data.
[0023] The encryption module is configured to encrypt the sensitive data in the buried point data by the symmetric key, and encrypt the symmetric key by the public key of the server, to obtain encryption information corresponding to the buried point data.
[0024] The sending module is configured to send a URL request to the server, the URL request comprising the encryption information, and the URL request being used to request the server to store the trace data into the preset trace data log folder.
[0025] In a fifth aspect, an embodiment of the present application provides a data processing system, comprising:
[0026] The user end and the server, the server comprising N website service nodes and a trace data source service node, N being a positive integer; wherein,
[0027] The user end is configured to acquire the trace data, the trace data being result data after data trace processing; in the case that the trace data comprises sensitive data, generate a symmetric key through a symmetric encryption algorithm; encrypt the sensitive data in the trace data through the symmetric key and encrypt the symmetric key through a public key of the server to obtain encryption information corresponding to the trace data; and send a URL request to the server, the URL request comprising the encryption information.
[0028] The website service nodes are configured to receive the URL request; splice a website service node name of each website service node in the N website service nodes of the server with the URL request respectively to obtain M target requests, M being a positive integer.
[0029] The trace data source service node is configured to acquire, based on a trace data log record directory, a website service node subdirectory corresponding to a website service node name of each target request in the M target requests, each website service node subdirectory corresponding to each trace data log file; and store each target request into a trace data log folder corresponding to the website service node subdirectory.
[0030] In a sixth aspect, an embodiment of the present application provides a computing device, comprising a processor and a memory storing computer program instructions; and
[0031] The processor executes the computer program instructions to implement the data processing method shown in the first aspect or the second aspect.
[0032] In a seventh aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium storing computer program instructions, the computer program instructions being executed by a processor to implement the data processing method shown in the first aspect or the second aspect.
[0033] In an eighth aspect, an embodiment of the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to run programs or instructions to implement the data processing method shown in the first aspect or the second aspect.
[0034] In a ninth aspect, an embodiment of the present application provides a computer program product stored in a storage medium, which is executed by at least one processor to implement the data processing method shown in the first aspect.
[0035] The data processing method, device, equipment and storage medium provided by the embodiments of the present application receive a URL request sent by a user end, the URL request comprising encrypted information, the encrypted information being information of trace data comprising sensitive data encrypted by a symmetric key, the symmetric key being calculated by a symmetric encryption algorithm; the website service node name of each website service node in N website service nodes of a server is spliced with the URL request respectively to obtain M target requests, N being a positive integer and M being a positive integer; based on a trace data log record directory, a website service node subdirectory corresponding to the website service node name of each target request in the M target requests is obtained respectively, each website service node subdirectory corresponding to each trace data log file; and each target request is stored in a trace data log folder corresponding to the website service node subdirectory. In this way, on the one hand, for the user end, the user end strengthens the security of the transmission of the trace data by encrypting the sensitive data in the trace data. On the other hand, for the server, the website service nodes in the server can collect the received encrypted trace data to a unified storage space, i.e. a trace data log folder, to improve the subsequent acquisition efficiency of the trace data, and since each website service node can perform unified configuration on the received URL request, the corresponding trace data log file can be acquired regardless of which website service node is identified in the request during subsequent acquisition of the trace data, in addition, during acquisition of the data, the reverse process of the stored data, i.e. the directory folder of the website service node name of each website service node, can ensure that all website service node collected current trace data logs are acquired by the trace data acquisition script by traversing the trace data source website service node name list corresponding to the website service node name once, thereby solving the problems of missing traversal and repeated traversal and further improving the processing efficiency of the trace data. BRIEF DESCRIPTION OF DRAWINGS
[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments of the present application will be briefly introduced. For those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.
[0037] Figure 1 A flowchart of a data processing method based on a user end provided by an embodiment of the present application;
[0038] Figure 2 A flowchart of a data processing method provided by an embodiment of the present application;
[0039] Figure 3 Fig. 1 is a schematic diagram of a data processing flow of one embodiment of a data processing method provided by an embodiment of the present application;
[0040] Figure 4 Fig. 2 is a schematic diagram of another data processing flow of one embodiment of a data processing method provided by an embodiment of the present application;
[0041] Figure 5 Fig. 3 is a schematic diagram of a structure of a data processing system provided by an embodiment of the present application;
[0042] Figure 6 Fig. 4 is a schematic diagram of a structure of a user-side data processing apparatus provided by an embodiment of the present application;
[0043] Figure 7 Fig. 5 is a schematic diagram of a structure of a data processing apparatus provided by an embodiment of the present application;
[0044] Figure 8 Fig. 6 is a schematic diagram of a structure of a data processing device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0045] The features and exemplary embodiments of various aspects of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present application, and are not intended to limit the present application. The present application can be implemented without some of the specific details described below. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.
[0046] It should be noted that, in this document, the terms such as first and second are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms “include”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the elements defined by the statement “include” do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0047] The acquisition, storage, use, processing and the like of data in the technical solutions of the present application comply with the relevant provisions of national laws and regulations.
[0048] In the related art, the Internet platform adopts two ways to store and acquire the buried point data. One way is to directly receive the buried point data by a background application, store the buried point data into a database, and then process and analyze the buried point data in the database by means of a timing batch, that is, a separate batch calculation. Since this way stores data into the database in real time, the requirement for the processing capacity of the database is high. Moreover, since the data is stored into the database in real time, the real-time calling of the background service transaction for storing data is affected, which greatly influences the throughput of the background service, and easily causes service downtime or the situation that other normal business transactions cannot be processed due to the need to process a large amount of stored buried point data transactions. The other way is to store the buried point data in the form of a log record in a disk by a web server (such as nginx), and process the buried point log data stored in the disk by a timing task. For a web server using a traditional multi-node method to deploy nginx, when a data analysis server accesses an interconnected web server, the interconnected web server needs to pass through an interconnected visiting load balancing system. Since the buried point data logs are stored in each independent web server, and the target web server is uncertain when the visiting load balancing forwards the buried point data acquisition request, the above way still has the problems of difficulty in collecting and acquiring the buried point data logs, difficulty in distinguishing the daily incremental logs, and insecurity of the buried point data.
[0049] Therefore, when nginx is deployed as a web server using a traditional multi-node method, the data analysis server accesses the interconnected web server needs to pass through an interconnected visiting load balancing system, and the main problems in collecting and acquiring the buried point data are as follows: first, since the buried point data logs are stored in each independent web server, when the visiting load balancing forwards the buried point data acquisition request to the interconnected web server, the web server is uncertain when the visiting load balancing forwards the buried point data acquisition request, it is difficult to ensure that all the web servers can traverse the collected and stored buried point data logs, and it is also difficult to avoid repeatedly acquiring the collected and stored buried point data logs of a target web server; second, the buried point data is stored in the log file in the form of a log plaintext, and if the buried point log has sensitive information, there is a risk of information leakage; third, the incremental buried point data logs of different dates on the same interconnected web server are difficult to distinguish, and data deduplication is more troublesome.
[0050] Based on this, in order to solve the above problems, the embodiment of the application provides a data processing method based on embedded point data collection, storage and acquisition, wherein the user end can encrypt the embedded point data through the randomly generated symmetric key, and use the protocol format to transmit the embedded point data to the server in the uniform resource locator (URL) parameter format. Then, the server can mount the shared storage space (nas) on each web server, and maintain the embedded point data log record directory in the nas, and start a timing task on each web server to regularly move the embedded point data logs stored on the server to the embedded point data log record directory in the nas, and distinguish the embedded point data logs from different servers by the server name. In addition, the data analysis server maintains a list of embedded point data source server names of each web server name of the interconnected web, and regularly starts an embedded point data acquisition script (i.e. a script for acquiring embedded point data on each web server through interconnection and visiting load balancing). The script can traverse the embedded point data source server name list, forward the embedded point data acquisition request to any one of the web server clusters in the web server cluster through interconnection and visiting load balancing, and guide the request to the corresponding data source in the embedded point data log record directory through the nginx forwarding rule configured on the server, so as to acquire the embedded point data collected and stored in each web server. After acquiring the embedded point data log text, the data analysis server uses the server private key to decrypt the URL parameter to obtain the symmetric key, and uses the symmetric key to decrypt the URL parameter to obtain the embedded point data plaintext. In this way, on the one hand, for the user end, the user end encrypts the sensitive data in the embedded point data, thereby enhancing the security of the embedded point data transmission. On the other hand, for the server, the website service node in the server can summarize the received encrypted embedded point data to the unified storage space, i.e. the embedded point data log folder, to improve the subsequent embedded point data acquisition efficiency. In addition, since each website service node can be configured uniformly for the received URL request, when acquiring the subsequent embedded point data, the corresponding embedded point data log file can be acquired regardless of which website service node is identified in the request. In addition, when acquiring the data, the reverse process of storing the data can be used, i.e. based on the directory folder with the website service node name of each website service node, so that the embedded point data acquisition script can ensure that all website service node collected embedded point data logs are acquired by traversing the embedded point data source server name list corresponding to the website service node name once, thereby solving the problems of missing traversal and repeated traversal, and further improving the embedded point data processing efficiency.
[0051] It should be noted that the data processing method provided in the embodiment of the present application can be applied to scenarios of Internet projects where nginx is deployed as a web server in a traditional multi-node manner and the data analysis server needs to access the web server through the interconnected load balancing.
[0052] Therefore, the embodiment of the present application provides a data processing method, device, equipment and storage medium. Figures 1 to 5 , describes in detail the data processing method, device, server and storage medium of the embodiments of the present application. It should be noted that these embodiments are not intended to limit the scope of disclosure of the present application.
[0053] The data processing method provided in the embodiment of the present application includes data collection, data storage and data acquisition processes.
[0054] First, combine Figure 1 The data collection process in the data processing method provided in the embodiment of the present application is described in detail.
[0055] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application.
[0056] like Figure 1 As shown, the data processing method can be applied to user devices at the user end, which include but are not limited to mobile phones, tablet computers, computers and other devices. Based on this, the data processing method can specifically include the following steps:
[0057] Step 110, obtain the buried data, which is the result data after data buried processing; Step 120, when the buried data includes sensitive data, generate a symmetric key through a symmetric encryption algorithm; Step 130, encrypt the sensitive data in the buried data with the symmetric key, and encrypt the symmetric key with the public key of the server to obtain encryption information corresponding to the buried data; Step 140, send a URL request to the server, the URL request includes encryption information, and the URL request is used to request the server to store the buried data in a preset buried data log folder.
[0058] Therefore, the user end improves the security of buried data transmission by encrypting the sensitive data in the buried data.
[0059] The above steps are explained in detail below:
[0060] First, step 110 is involved. In one or more possible embodiments, when the embedded point data does not include sensitive data, the URL request includes the embedded point data in plain text and a first location identifier, and the first location identifier is used to identify the location of the embedded point data in the URL request.
[0061] In addition, to improve the processing efficiency of the subsequent data acquisition process, the URL request here also includes a first location identifier, which is used to identify the location of the encrypted information in the URL request. Therefore, it is possible to determine whether to decrypt the stored data based on the first location identifier to facilitate user query and browsing. Alternatively, the URL request may also include a second identification location, which is used to indicate that the embedded data does not contain sensitive information.
[0062] Exemplarily, during the data collection and storage process, the user side randomly generates a symmetric encryption key and uses the key to encrypt sensitive data (such as user identity information, user name, etc.), and at the same time uses the server's public key to encrypt the generated symmetric key, and puts the encrypted embedding data and the encrypted key into the agreed URL request parameters, and adds the first position marker as 1 to transmit it to the server (if there is no sensitive data, the embedding data is directly added to the agreed URL request in plain text and transmitted to the server).
[0063] In this way, the embodiment of the present application provides a user-side encryption and decryption process of the buried point data acquisition module, so that the sensitive data in the buried point data is in a ciphertext state when saved at the collection point, thereby improving the security of the buried point data storage.
[0064] Secondly, combined Figure 2 and Figure 3 The data storage process in the data processing method provided in the embodiment of the present application is described in detail.
[0065] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application.
[0066] like Figure 2 As shown, the data processing method can be applied to a service device on the server side, which includes but is not limited to a server, a server cluster, a service node with computing functions, a virtual device, etc. Based on this, the data processing method can specifically include the following steps:
[0067] Step 210, receiving the URL request sent by the user end, the URL request includes encryption information, the encrypted information is information of the buried data including sensitive data encrypted with a symmetric key, and the symmetric key is calculated by a symmetric encryption algorithm; Step 220, respectively concatenating the website service node name of each website service node in the N website service nodes of the server end with the URL request to obtain M target requests, where N is a positive integer and M is a positive integer; Step 230, based on the buried data log record directory, respectively obtain the website service node subdirectory corresponding to the website service node name of each target request in the M target requests, and each website service node subdirectory corresponds to each buried data log file; Step 240, storing each target request in the buried data log folder corresponding to the website service node subdirectory.
[0068] Therefore, for the server, the website service node in the server can aggregate the received encrypted tracking data into a unified storage space, that is, the tracking data log folder, to improve the efficiency of subsequent tracking data acquisition. In addition, since each website service node can uniformly configure the received URL request, when the subsequent tracking data is acquired, no matter which website service node is identified in the request, the corresponding tracking data log file can be obtained. In addition, when acquiring data, the reverse process of storing data can be used, that is, the directory folder named after the website service node of each website service node is maintained, so that the tracking data acquisition script only needs to traverse the tracking data source service node name list corresponding to the website service node name once to ensure that the current tracking data log collected by all website service nodes is obtained, which solves the problems of missed traversal and repeated traversal, and further improves the processing efficiency of tracking data.
[0069] The above steps are explained in detail below:
[0070] First, referring to step 230, in one or more possible embodiments, the website service node includes a storage space. Based on this, before step 230, the data processing method may further include:
[0071] In the storage space, build a logging directory for embedded data;
[0072] Under the embedding data log record directory, a directory folder is constructed with the website service node name of each website service node. The directory folder is used to distinguish different embedding data source service nodes corresponding to the website service nodes.
[0073] Then, step 240 is involved, in one or more possible embodiments, based on the timing task in the website service node, the target request is transmitted to the directory file folder under the tracking data log recording directory of the website service node with the network service node name of the website service node; the log file name of the target folder includes the date information of the execution timing task.
[0074] Exemplarily, the server mounts a shared nas on the web service node, and newly creates a tracking data log recording directory with a convention name (such as named nas_tracking_data) in the nas, and newly creates a directory file folder named with the name of each web service node under the tracking data log recording directory to distinguish different tracking data source service nodes, while starting a tracking data storage timing task (executed once a day) on each web service node to execute a tracking data storage script to move the tracking data log to the directory under the nas_tracking_data directory in the nas with the name of the web service node, and add the date information of the day to the log file name. Then, the data analysis service node starts a tracking data acquisition timing task (such as executed once a day, and the time is after the execution of the tracking data storage timing task of the web service node, the delay time is determined according to the actual situation of the project, which can be delayed for one hour for example), according to the list of tracking data source service node names, splices each web service node name into the URL request, and initiates a request to the web service node through interconnection visit load balancing, and the nginx deployed on the web service node guides the request to the corresponding tracking data log recording directory, so as to acquire the corresponding tracking data log file, and store it to the data analysis directory on the data analysis service node.
[0075] Therefore, in order to better illustrate the above content, the following will be combined with Figure 3 to be described in detail.
[0076] As Figure 3 shown, the data collection and storage overall framework includes the following parts:
[0077] Tracking data transaction, the user end transmits the tracking data to the server with URL request, and loads the URL request to each web service node in the tracking data collection module through interconnection visit load balancing.
[0078] Here, if there is sensitive data in the buried point data, an example of the URL request is as follows: https: / / <website domain> / track_data?tkData=<encrypted buried point data>&enKey=<symmetric key encrypted by the server public key>&isEn=1. Among them, track_data is the agreed buried point data, the value is the buried point data encrypted by the user end random symmetric key, enKey is the key of the server, the value is the symmetric key encrypted by the server public key, and isEn is the first position identifier, and 1 means encryption. Conversely, if there is no sensitive data in the buried point data, an example of the URL request is as follows: https: / / <website domain> / track_data?tkData=<buried point data>&isEn=0. Among them, track_data is the agreed buried point data, the value is the clear text of the unencrypted buried point data, and isEn is the second position identifier, and 0 means unencrypted.
[0079] The buried point data collection module, in the embodiment of the application, taking a web service node cluster as an example, each web service node in the cluster is deployed nginx to receive the buried point data transaction request forwarded by the interconnection load balancing, and the buried point data is stored as a log in the local_track_data directory of the web service node local disk; The key configuration of nginx is as follows:
[0080] Location / track_data{##Identify buried point data transaction identifier
[0081] access_log / <local_track_data directory absolute path> / tk.access..log;##Generate buried point data log to record buried point data transaction request, log file name format is tk.access..log}.
[0082] Buried point data storage timing task, used to execute buried point data storage script regularly (such as once a day, for example, at 24 o'clock in the morning), buried point data storage timing task can be realized by using linux crontab, crontab example configuration as follows:
[0083] 0 24***sh / <buried point data storage script absolute path>
[0084] The pseudo code of the buried point data storage script is as follows:
[0085] dt=`date“+%Y-%m-%d”`##Define date format as year-month-day, such as 2022-01-25
[0086] logSrcDir=<local_track_data directory absolute path>## Define the tracking data source directory
[0087] h=`hostname`## Get the web server name through the server environment variable
[0088] logDstDir=<nas_tracking_data directory absolute path> / $h## Define the mobile tracking data log directory as the directory named after the web server name under the nas_tracking_data directory on the nas disk
[0089] mkdir –p$logDstDir## Create the mobile tracking data log directory if it does not exist
[0090] cd$logSrcDir## Enter the tracking data source directory
[0091] mv tk.access.log$logDstDir / tk.access.log.$dt## Move all current tracking data logs to the mobile tracking data log directory and add a date suffix to the end of the log name, such as tk.access.log.2022-01-25.
[0092] Tracking data storage module, this article takes the shared nas as an example, establishes the tracking data log directory according to the agreement under the shared nas, and establishes subdirectories according to the names of each web service node under the directory, which is used to store the tracking data log files transferred from the corresponding web service node tracking data source directory.
[0093] Tracking data cleaning module, this module starts a timing task to clean up tracking logs before a specified period of time, its execution time is after the tracking data storage timing task, for example, clean up the tracking log data seven days ago each time, for example, the cleaning script pseudo code is: find<nas_tracking_data directory absolute path> -type f -mtime +7 –exec rm{}\;.
[0094] Therefore, the data processing architecture provided by the embodiment of the present application provides a data collection module, a data storage module, a data cleaning module and a data acquisition module, realizes the data collection and storage process, and the data collection and storage process uniformly configures each data collection point, so that when subsequent data is acquired, the data acquisition module can acquire corresponding data log files from the data storage module regardless of which collection point is identified in the request; the data collection and storage sub-protocol stipulates that the data collection point executes the action of moving the full local data log file to the data storage module once a day, and adds the current identifier according to the protocol, thereby conveniently solving the problem of distinguishing different date incremental data logs.
[0095] Then, after step 240, in combination with Figure 4 and Figure 5 The data acquisition process in the data processing method provided by the embodiment of the present application is described in detail.
[0096] In this way, the URL request also includes a first position identifier, the first position identifier is used to identify the position of the encryption information in the URL request, and the symmetric key is encrypted by the public key of the server. Based on this, after step 240, the data processing method can further include:
[0097] Step 2501, acquiring a query task;
[0098] Step 2502, based on the query task, querying the data log file in the data log folder, and collecting the data log file to a data analysis directory;
[0099] Step 2503, reading the data log file in the data analysis directory according to a preset order, and acquiring an encryption parameter bit corresponding to the data log file;
[0100] Step 2504, in the case where the identifier of the encryption parameter bit is the first position identifier, decrypting the symmetric key by the private key of the server to obtain a decrypted symmetric key;
[0101] Step 2505, decrypting the sensitive data corresponding to the first position identifier by the decrypted key to obtain the data in plaintext form;
[0102] Step 2506, transmitting the data to a query folder corresponding to the query task, so as to feed back to the user end.
[0103] In addition, in the case where the identifier of the encryption parameter bit is the second position identifier, the data log file is transmitted to the query folder.
[0104] Exemplarily, after the data analysis server obtains the buried point log file, the file content in the data analysis directory is read by line and the agreed encryption parameter bit is read. If the encryption parameter bit is 1, the symmetric key corresponding to the value of the agreed key parameter in the URL parameter is obtained by using the server private key to decrypt, then the buried point data plaintext is obtained by using the obtained symmetric key to decrypt the value corresponding to the agreed buried point data parameter in the URL parameter, and is summarized to the agreed file (if the encryption parameter bit is 0, the buried point data plaintext is directly summarized to the agreed file).
[0105] In this way, the data acquisition can solve the problems of missing traversal and repeated traversal by maintaining the buried point data source service node name list, greatly improve the efficiency of acquiring the buried point data from the collection point, and set the encrypted position identifier in the process of data collection and storage. At this time, whether the buried point data is encrypted or decrypted can be determined according to the position identifier, and the security and encryption efficiency of the buried point data are taken into account.
[0106] Therefore, in order to better illustrate the above content, the following will be combined with Figure 4 to be described in detail.
[0107] As Figure 4 shown, the data acquisition overall framework includes the following parts:
[0108] The buried point data source service node name list is used to store the list of names of each buried point data source service node, which is the list of names of each web server here. The data analysis directory is used to store the directory of the acquired buried point data log file, which is named track_data_store_dir in this paper. The buried point data acquisition timing task is used to execute the buried point data acquisition script at a fixed time (such as once a day, and after the execution time of the buried point data storage timing task, for example, at 1 o'clock in the morning) to realize the decryption and summarization of the buried point data.
[0109] The buried point data acquisition timing task can be implemented using linux crontab, and the crontab example configuration is as follows: 0 1***sh / <buried point data acquisition script absolute path>; The buried point data acquisition script pseudo code is as follows: host_name_list=(“web service node 1 host name”, “web service node 2 host name”, …, “web service node n host name”)###Buried point data source service node name list; URL=http: / / <interconnection visit load balancing address and port>###Configure the address of the visit load balancing and the port used to receive the buried point data acquisition request. When the buried point data acquisition request is received, the request will be forwarded to a web service node, and the request will be processed by the nginx deployed on the web service node; dt=`date–d“yesterday”+%Y%m%d`##Define the date as the date of the previous day of the current execution timing task. Each time the buried point data acquisition script is executed, the date of the previous day is obtained, and the format is year-month-day. For example, if the current execution timing task date is 2022-01-26, the obtained date is 2022-01-25; track_data_store_dir=<data analysis directory absolute path>###Define the data analysis directory;
[0110] sum TkDataFile=$track_data_store_dir / sumTkDataFile_$dt.txt##Agree to specify the date summary file, mkdir–p$track_data_store_dir###If the data analysis directory does not exist, create it, index=0####Start from index 0 to traverse the web service node, host_cnt=${#host_name_list[@]}###Obtain the number of web service nodes; data_file_name=tk.access.log.${dt}###Buried point data log file name to be acquired; while[${index}–lt${host_cnt}]###Traverse all interconnected web service nodes; do;host_name=${host_name_list[${index}]}###Obtain the web service node name of the corresponding index subscript;
[0111] URL_str=${URL} / track_data_store / ${host_name} / ${data_file_name}###Final request URL, where / track_data_store is the prefix used by nginx on the web service node to match the request for collecting buried point data, and the final request URL format is http: / / <Internet visit load balancing address and port> / track_data_store / <current traversed web server name> / tk.access.log.<the previous day of the current date when the timing task is executed>; data_store_file=${track_data_store_dir} / ${host_name}.${data_file_name}###The absolute path of the file where the buried point data of this request is finally stored, in the format of / <data analysis directory absolute path> / <host name of the web service node traversed this time>.tk.access.log.<the previous day of the current date when the timing task is executed>;
[0112] wget –O ${track_data_store_dir} / ${host_name}.${data_file_name}.${dt} ${URL_str}###Use wget to initiate a request to the Internet visit load balancing for collecting buried point data
[0113] let index++###Index is incremented, and the next web service node name in the list of buried point data source service node names is obtained to collect the buried point data log file stored by the web service node; do; isEn=getIsEn(ct);###Get the encryption identification bit; if(isEn){##Buried point data is encrypted; tkData=getTkData(ct);##Get encrypted buried point data; realTkData=deCriptTkData(tkData);##Decrypt the buried point data in clear text using the public key of the service node;}else{##Buried point data is not encrypted; realTkData=getTkData(ct);##Directly get the buried point data in clear text;};cat $realTkData>$sumTkDataFile;##Merge the buried point data in clear text of this line into the daily buried point data summary file; done#File content traversal ends; done#Server traversal ends.
[0114] In addition, the web server cluster, each web service node in the cluster is deployed nginx for receiving the embedded data acquisition request, which is consistent with the web service node in the above collection and storage device, the difference is that the embedded data acquisition request hits the nginx configuration block, and the corresponding nginx configuration block key configuration is as follows: Location / track_data_store / {alias / <embedded data log record directory absolute path> / ### Direct the embedded data acquisition request starting with track_data_store in the request to the embedded data log record directory in the nas, and according to the web service node name information in the request, direct to the directory named with the corresponding web service node name under the embedded data log record directory, so as to obtain the embedded data log file collected and stored by the corresponding web service node}; the embedded data storage module, the embedded data log record directory under the shared nas, which is the same as the embedded data log record directory of the shared nas disk in the above collection and storage process.
[0115] It should be noted that the embedded data cleaning module cleans the expired embedded data, which can clean the data before a specified number of days as needed, which will not be described here.
[0116] In summary, the data processing method provided by the embodiment of the application can encrypt sensitive data in the embedded data according to data collection, data storage and data acquisition, and can enhance the security of embedded data transmission. For the user end, first, each web service node can collect embedded data logs and aggregate them to a unified storage space, improving the efficiency of subsequent embedded data acquisition; second, each web service node can configure the received URL request uniformly, and can obtain the corresponding embedded data log file regardless of which web service node is identified in the request; third, each web service node can execute the action of moving the local embedded data log file to the embedded data storage module once a day, ensuring that the embedded data log file of the current period is moved each time the action is executed, and adding the current identifier according to the agreed protocol, which conveniently solves the problem of distinguishing different date incremental embedded data logs; fourth, the embedded data acquisition module can maintain a list of embedded data source server names, so that the embedded data acquisition script on the data analysis server can ensure that all current embedded data logs collected by the web server are obtained by traversing the list of embedded data source server names once, solving the problem of missing traversal and repeated traversal; fifth, the embedded data collected by the user end can be decrypted, so that the sensitive data of the embedded data is in ciphertext state when it is saved at the collection point, improving the security of embedded data storage; sixth, the location identifier represents the encryption flag bit, which can only encrypt and decrypt sensitive data, improving the flexibility and efficiency of encryption and decryption actions.
[0117] Based on the same inventive concept, the embodiment of the present application further provides a data processing system based on the above content. Figure 5 Provide detailed explanation.
[0118] Figure 5 A structural diagram of a data processing system 50 provided in an embodiment of the present application.
[0119] like Figure 5 As shown, the data processing system may include:
[0120] The client 501 and the server 502, the server 502 includes N website service nodes 5021 and buried data source service nodes 5022, where N is a positive integer;
[0121] The client is used to obtain tracking data, which is the result of data tracking processing. If the tracking data includes sensitive data, a symmetric key is generated using a symmetric encryption algorithm. The sensitive data in the tracking data is encrypted using the symmetric key, and the symmetric key is encrypted using the server's public key to obtain encrypted information corresponding to the tracking data. A URL request is sent to the server, and the URL request includes the encrypted information.
[0122] The website service node is used to receive URL requests; the website service node name of each website service node in the N website service nodes of the server is concatenated with the URL request to obtain M target requests, where M is a positive integer;
[0123] The tracking point data source service node is used to obtain the website service node subdirectory corresponding to the website service node name of each target request in M target requests based on the tracking point data log record directory, and each website service node subdirectory corresponds to each tracking point data log file; each target request is stored in the tracking point data log folder corresponding to the website service node subdirectory.
[0124] Based on this, in one or more possible embodiments, the URL request further includes a first location identifier, which is used to identify the location of the encrypted information in the URL request, and the symmetric key is encrypted by the public key of the server; based on this, the server 50 further includes a data analysis service node 5023, which is used to query the burial point log files in the burial point data log folder based on the query task, and aggregate the burial point log files into the data analysis directory;
[0125] Read the tracking log files in the data analysis directory in a preset order and obtain the encryption parameter bits corresponding to the tracking log files;
[0126] When the identifier of the encryption parameter bit is the first position identifier, the symmetric key is decrypted using the private key of the server to obtain the decrypted symmetric key;
[0127] Decrypt the sensitive data corresponding to the first location identifier using the decryption key to obtain the burial point data in plain text form;
[0128] The tracking data is transferred to the query folder corresponding to the query task so as to provide feedback to the user end.
[0129] It should be noted that the above-mentioned website service node 5021, embedded data source service node 5022 and data analysis service node 5023 can all be at least one of the following: server, computing module, server cluster, virtual node, etc. Here, only nodes are used as examples, and their specific forms are not limited.
[0130] In this way, on the one hand, for the user side, the user side strengthens the security of the buried data transmission by encrypting the sensitive data in the buried data. On the other hand, for the server side, the website service node in the server side can aggregate the encrypted buried data received into a unified storage space, that is, the buried data log folder, to improve the efficiency of subsequent buried data acquisition. In addition, since each website service node can uniformly configure the received URL request, when the subsequent buried data is acquired, no matter which website service node is identified in the request, the corresponding buried data log file can be obtained. In addition, when acquiring data, the reverse process of the stored data can be used, that is, based on the directory folder named after the website service node of each website service node, so that the buried data acquisition script only needs to traverse the buried data source service node name list corresponding to the website service node name once to ensure that the current buried data log collected by all website service nodes is obtained, solving the problem of missed traversal and repeated traversal, and further improving the processing efficiency of buried data.
[0131] Based on the same inventive concept, the present application also provides a data processing device. Figure 6 Provide detailed explanation.
[0132] Figure 6 It is a structural diagram of a data processing device provided by an embodiment of the present application.
[0133] In some embodiments of the present application, Figure 6 The data processing device shown may be arranged in a user equipment at the user end.
[0134] like Figure 6 As shown, the data processing device 60 may specifically include:
[0135] The obtaining module 601 is configured to obtain the buried point data, the buried point data being result data after data buried point processing.
[0136] The generating module 602 is configured to, in a case where the buried point data includes sensitive data, generate a symmetric key by using a symmetric encryption algorithm.
[0137] The encryption module 603 is configured to encrypt the sensitive data in the buried point data by using the symmetric key, and encrypt the symmetric key by using a public key of the server, to obtain encrypted information corresponding to the buried point data.
[0138] The sending module 604 is configured to send a URL request to the server, the URL request including the encrypted information, the URL request being used to request the server to store the buried point data in a preset buried point data log folder.
[0139] In the embodiments of the present application, the URL request further includes a first position identifier, the first position identifier being used to identify a position of the encrypted information in the URL request. In addition, in a case where the buried point data does not include sensitive data, the URL request includes the buried point data in a plaintext form and the first position identifier, the first position identifier being used to identify a position of the buried point data in the URL request.
[0140] In this way, for the user end, the user end strengthens the security of the transmission of the buried point data by encrypting the sensitive data in the buried point data.
[0141] Figure 7 FIG. 1 is a structural schematic diagram of a data processing apparatus provided in an embodiment of the present application.
[0142] In some embodiments of the present application, Figure 7 The data processing apparatus shown in the figure can be arranged in a service device of the server.
[0143] As shown in the figure, Figure 7 The data processing apparatus 70 specifically can include:
[0144] The receiving module 701 is configured to receive a URL request sent by the user end, the URL request including encrypted information, the encrypted information being information of the buried point data including sensitive data that is encrypted by using a symmetric key, the symmetric key being calculated by using a symmetric encryption algorithm.
[0145] The splicing module 702 is configured to splice a website service node name of each website service node in N website service nodes of the server with the URL request respectively, to obtain M target requests, N being a positive integer, and M being a positive integer.
[0146] The acquisition module 703 is configured to acquire, based on the log record directory of the embedded data, a website service node subdirectory corresponding to a website service node name of each target request in the M target requests, and each website service node subdirectory corresponds to each embedded data log file.
[0147] The storage module 704 is configured to store each target request into an embedded data log file folder corresponding to the website service node subdirectory.
[0148] The data processing apparatus 70 in the embodiments of the present application will be described in detail as follows.
[0149] In one or more optional embodiments, the data processing apparatus 70 in the embodiments of the present application can further include a construction module, wherein,
[0150] The construction module is configured to, in the case that the website service node includes a storage space, construct the log record directory of the embedded data in the storage space.
[0151] Under the log record directory of the embedded data, a directory folder named with the website service node name of each website service node is constructed, and the directory folder is used to distinguish different embedded data source service nodes corresponding to the website service node.
[0152] In another one or more optional embodiments, the storage module 704 can be specifically configured to, based on a timing task in the website service node, transmit the target request to a directory folder with the network service node name of the website service node under the log record directory of the embedded data; and the log file name of the target folder includes date information of executing the timing task.
[0153] In yet another one or more optional embodiments, the data processing apparatus 70 in the embodiments of the present application can further include a query module, a decryption module and a transmission module, wherein,
[0154] The acquisition module 703 can be further configured to acquire the query task in the case that the URL request further includes a first location identifier used to identify the location of the encrypted information in the URL request, and the symmetric key is encrypted by the public key of the server.
[0155] The query module is configured to query the embedded log file in the embedded data log file folder based on the query task, and aggregate the embedded log file to a data analysis directory.
[0156] The acquisition module 703 can be further configured to read the embedded log file in the data analysis directory in a preset order, and acquire the encrypted parameter bit corresponding to the embedded log file.
[0157] decrypt the symmetric key by using the private key of the server to obtain a decrypted symmetric key, and
[0158] decrypt the sensitive data corresponding to the first location identifier by using the decrypted key to obtain the buried point data in the form of plaintext;
[0159] The transmission module is configured to transmit the buried point data to a query folder corresponding to the query task, so as to feed back to the user end.
[0160] In another or more optional embodiments, the transmission module can also be configured to, in a case where the identifier of the encryption parameter bit is the second location identifier, transmit the buried point log file to the query folder.
[0161] In this way, for the server, the website service node in the server can aggregate the received encrypted buried point data to a unified storage space, i.e., a buried point data log folder, to improve the subsequent acquisition efficiency of the buried point data, and since each website service node can be uniformly configured for the received URL request, when the subsequent buried point data is acquired, no matter which website service node is identified in the request, the corresponding buried point data log file can be acquired, in addition, when the data is acquired, the reverse process of the stored data can be used, i.e., based on the directory folder of the website service node name maintained for each website service node, the buried point data acquisition script can ensure that all website service node collected current buried point data logs are acquired by traversing the buried point data source service node name list corresponding to the website service node name only once, thereby solving the problems of missing traversal and repeated traversal, and further improving the processing efficiency of the buried point data.
[0162] Based on the same inventive concept, the present application also provides a data processing device. The specific combination Figure 8 will be described in detail.
[0163] Figure 8 is a structural schematic diagram of the data processing device provided by an embodiment of the present application.
[0164] As Figure 8 shown, the data processing device can include at least one of the following described in the embodiments of the present application: a computing device, a server. The data processing device can include a processor 801 and a memory 802 having computer program instructions stored therein.
[0165] In particular, the processor 801 can include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or can be configured to implement one or more integrated circuits that embody the embodiments of the present application.
[0166] The memory 802 can include mass storage for data or instructions. By way of example, and not limitation, the memory 802 can include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The memory 802 can be removable and / or non-removable (or fixed) as appropriate. The memory 802 can be internal or external as appropriate. In certain embodiments, the memory 802 is non-volatile solid-state memory. In certain embodiments, the memory 802 includes solid-state storage (SSS). Where appropriate, this SSS can include battery backup. The SSS can be a flash memory or other appropriate memory.
[0167] The processor 801 implements any of the data processing methods described above by reading and executing computer program instructions stored in the memory 802.
[0168] In one example, the data processing device can also include a communication interface 803 and a bus 810. As shown, the processor 801, the memory 802, and the communication interface 803 are connected through the bus 810 and complete communication with each other. Figure 8
[0169] The communication interface 803 is mainly used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application.
[0170] Bus 810 includes a hardware, software, or both that couples components of the traffic control device to each other. As an example and not by way of limitation, bus can include an accelerated graphics port (AGP) or other graphics bus, an enhanced industry standard architecture (EISA) bus, a front-side bus (FSB), a HyperTransport (HT) interconnect, an industry standard architecture (ISA) bus, an InfiniBand (IB) interconnect, a low pin count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a peripheral component interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a serial advanced technology attachment (SATA) bus, a video electronics standards board (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 810 can include one or more buses. Although this application describes and shows a particular bus, this application contemplates any suitable bus or interconnect.
[0171] The data processing device can execute the data processing method in the embodiments of the application, so as to realize the data processing method and device described in combination Figures 1 to 7 with the data processing method and device.
[0172] In addition, in combination with the data processing method in the above embodiments, the embodiments of the application can provide a computer readable storage medium to realize. The computer readable storage medium has computer program instructions stored thereon; the computer program instructions are executed by the processor to realize any one of the data processing methods in the above embodiments.
[0173] It needs to be clear that the application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the application.
[0174] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. Programs or code segments can be stored in machine-readable media, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segments can be downloaded via computer networks such as the Internet, intranets, etc.
[0175] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps. In other words, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0176] The above is only a specific implementation method of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the scope of protection of the present application is not limited to this. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application.
Claims
1. A data processing method, characterized by, Applied to a server, comprising: Receiving a URL request sent by a user end, the URL request comprising encrypted information, the encrypted information being information of trace data comprising sensitive data encrypted by a symmetric key, the symmetric key being calculated by a symmetric encryption algorithm; Concatenating a website service node name of each website service node in N website service nodes of the server with the URL request respectively to obtain M target requests, N being a positive integer, M being a positive integer, the website service node comprising a storage space shared by N website service nodes; In the storage space, a trace data log record directory is constructed; Under the trace data log record directory, a directory folder named with the website service node name of each website service node is constructed, the directory folder being used to distinguish different trace data source service nodes corresponding to the website service node; Based on the trace data log record directory, a website service node subdirectory corresponding to the website service node name of each target request in the M target requests is obtained respectively, each website service node subdirectory corresponding to each trace data log file; The each target request is stored into a trace data log file folder corresponding to the website service node subdirectory.
2. The method of claim 1, wherein, The each target request is stored into a trace data log file folder corresponding to the website service node subdirectory, comprising: Based on a timing task in the website service node, the target request is transmitted to a directory folder under the trace data log record directory and the network service node name of the website service node; the log file name of the directory folder comprises date information of executing the timing task.
3. The method of claim 2, wherein, The URL request further comprises a first position identifier, the first position identifier being used to identify a position of the encrypted information in the URL request, the symmetric key being encrypted by a public key of the server; After the each target request is stored into the trace data log file folder corresponding to the website service node subdirectory, the method further comprises: Obtaining a query task; Based on the query task, a trace log file in the trace data log file folder is queried, and the trace log file is summarized to a data analysis directory; In a preset order, the trace log file in the data analysis directory is read to obtain an encrypted parameter bit corresponding to the trace log file; In a case where an identifier of the encrypted parameter bit is the first position identifier, the symmetric key is decrypted by a private key of the server to obtain a decrypted symmetric key; The sensitive data corresponding to the first position identifier is decrypted by the decrypted symmetric key to obtain trace data existing in plaintext form; The trace data is transmitted to a query folder corresponding to the query task to feed back to the user end.
4. The method of claim 3, wherein, After the encrypted parameter bit corresponding to the trace log file is obtained, the method further comprises: In a case where an identifier of the encrypted parameter bit is a second position identifier, the trace log file is transmitted to the query folder.
5. A data processing method, characterized by, Applied to a user end, comprising: Obtaining trace data, the trace data being result data after data trace processing; In a case where the buried point data includes sensitive data, a symmetric key is generated by a symmetric encryption algorithm; The sensitive data in the buried point data is encrypted by the symmetric key, and the symmetric key is encrypted by a public key of the server to obtain encryption information corresponding to the buried point data; The URL request includes the encryption information, and the URL request is used to request the server to store the buried point data into a preset buried point data log folder.
6. The method of claim 5, wherein, The URL request further includes a first position identifier, and the first position identifier is used to identify a position of the encryption information in the URL request.
7. The method of claim 5, wherein, In a case where the buried point data does not include sensitive data, the URL request includes the buried point data in a plaintext form and the first position identifier, and the first position identifier is used to identify a position of the buried point data in the URL request.
8. A data processing system, characterized by Comprise: A user end and a server, the server comprising N website service nodes and buried point data source service nodes, N being a positive integer; wherein, The user end is configured to acquire buried point data, the buried point data being result data after data buried point processing; in a case where the buried point data includes sensitive data, generate a symmetric key by a symmetric encryption algorithm; encrypt the sensitive data in the buried point data by the symmetric key, and encrypt the symmetric key by a public key of the server to obtain encryption information corresponding to the buried point data; and send a URL request to the server, the URL request including the encryption information. The website service nodes are configured to receive the URL request; splice a website service node name of each website service node of the N website service nodes of the server with the URL request to obtain M target requests, M being a positive integer, and the website service nodes include a storage space shared by the N website service nodes. The buried point data source service nodes are configured to construct a buried point data log record directory in the storage space; construct a directory folder named by the website service node name of each website service node under the buried point data log record directory, the directory folder being used to distinguish different buried point data source service nodes corresponding to the website service nodes; based on the buried point data log record directory, acquire a website service node subdirectory corresponding to the website service node name of each target request in the M target requests, each website service node subdirectory corresponding to each buried point data log file; and store each target request into a buried point data log folder corresponding to the website service node subdirectory.
9. The system of claim 8, wherein, The URL request further includes a first position identifier, and the first position identifier is used to identify a position of the encryption information in the URL request, and the symmetric key is encrypted by the public key of the server. The server further comprises a data analysis service node, configured to query the buried point log files in the buried point data log folder based on a query task, and aggregate the buried point log files to a data analysis directory; read the buried point log files in the data analysis directory in a preset order, and obtain an encryption parameter bit corresponding to the buried point log files; in a case where the identification of the encryption parameter bit is the first position identification, decrypt the symmetric key by using a private key of the server to obtain a decrypted symmetric key; decrypt the sensitive data corresponding to the first position identification by using the decrypted symmetric key to obtain the buried point data in plaintext form; transmit the buried point data to a query folder corresponding to the query task, so as to feed back to the user terminal.
10. A data processing apparatus, characterized by, Applied to a server, comprising: a receiving module configured to receive a URL request sent by a user terminal, wherein the URL request comprises encryption information, the encryption information is information of buried point data encrypted by a symmetric key, and the symmetric key is calculated by a symmetric encryption algorithm; a splicing module configured to splice a website service node name of each website service node in N website service nodes of the server with the URL request respectively to obtain M target requests, N is a positive integer, M is a positive integer, and the website service node comprises a storage space shared by N website service nodes; a construction module configured to construct a buried point data log record directory in the storage space, and construct a directory folder named by the website service node name of each website service node under the buried point data log record directory, wherein the directory folder is used to distinguish different buried point data source service nodes corresponding to the website service node; an acquisition module configured to acquire a website service node subdirectory corresponding to the website service node name of each target request in the M target requests based on the buried point data log record directory, and each website service node subdirectory corresponds to each buried point data log file; a storage module configured to store each target request to a buried point data log folder corresponding to the website service node subdirectory.
11. A data processing apparatus, characterized by Applied to a user terminal, comprising: an acquisition module configured to acquire buried point data, wherein the buried point data is result data after data burying processing; a generation module configured to generate a symmetric key by using a symmetric encryption algorithm in a case where the buried point data comprises sensitive data; an encryption module configured to encrypt the sensitive data in the buried point data by using the symmetric key, and encrypt the symmetric key by using a public key of a server to obtain encryption information corresponding to the buried point data; a sending module configured to send a URL request to the server, wherein the URL request comprises the encryption information, and the URL request is used to request the server to store the buried point data to a preset buried point data log folder, and the server is based on the method of claim 1 to store the buried point data to the preset buried point data log folder.
12. A computing device, comprising: comprising: a processor, and a memory in communication connection with the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the data processing method according to any one of claims 1-7.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the data processing method according to any one of claims 1-7.
14. A computer program product, characterised in that, The computer program is executed by the processor to implement the data processing method according to any one of claims 1-7.
Citation Information
Patent Citations
Log management method and system and computer-readable storage medium
CN108052675A
Data encryption reporting method and device and data decryption method and device
CN109474619A