Log storage system construction method and system based on multidimensional logic relation interaction

Abstract

The invention relates to the technical field of information, in particular to a log storage system construction method and system based on multidimensional logic relation interaction.The method comprises the steps that a log storage instruction is received, an original log set is collected based on the log storage instruction, analysis operation is executed on the original log set, and a multidimensional data set is obtained; calculating a user weight according to the multi-dimensional data and the priority user sequence, calculating a time weight according to the transaction time and the priority time sequence in the multi-dimensional data, obtaining a headquarters position, calculating a position weight according to an initiation position and the headquarters position in the multi-dimensional data, and obtaining a headquarters position; and calculating the log priority according to the user weight, the time weight, the position weight and the transaction amount value, performing classification compression operation on the backlog set to obtain a log compression package, and constructing a log graph database by using the common log set to complete the construction of the log storage system. According to the method, the log storage space can be saved, and the log retrieval efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a log storage system construction method, system, electronic device and computer-readable storage medium based on multi-dimensional logical relationship interaction. Background Art

[0002] In the modern information society, with the expansion of business scale and the popularization of network services, various complex business scenarios generate massive amounts of log data. Logs are important text files that record information such as system operations and business transactions, and are crucial for system monitoring and transaction information analysis.

[0003] Currently, log storage systems generally simply record system operation information or business transaction information based on time or event sequence, and store the information directly in the memory in the form of raw log files.

[0004] While existing log storage systems can store logs, they treat all logs equally, failing to prioritize their importance. For example, logs with high transaction amounts or those involving critical business operations are prioritized, while ordinary logs should be properly archived or compressed. Furthermore, key information is not extracted from logs during storage, resulting in a high level of redundant text data, which consumes more storage space and slows down retrieval. Summary of the Invention

[0005] The present invention provides a log storage system construction method based on multi-dimensional logical relationship interaction and a computer-readable storage medium, the main purpose of which is to save log storage space and improve log retrieval efficiency.

[0006] To achieve the above objectives, the present invention provides a method for constructing a log storage system based on multi-dimensional logical relationship interaction, comprising:

[0007] receiving a log storage instruction, collecting an original log set based on the log storage instruction, performing a parsing operation on the original log set to obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount;

[0008] Obtaining a retrieval record set, wherein the retrieval record set includes: a user retrieval records and b time retrieval records, and obtaining a priority user sequence and a priority time sequence using the a user retrieval records and the b time retrieval records respectively;

[0009] The following operations are performed on each multidimensional data in the multidimensional dataset:

[0010] Calculate user weights based on multidimensional data and priority user sequences, and calculate time weights based on transaction times and priority time sequences in multidimensional data;

[0011] Get the headquarters location and calculate the location weight based on the originating location and headquarters location in the multidimensional data;

[0012] Calculate log priority based on user weight, time weight, location weight and transaction amount;

[0013] Comparing the log priority with a preset priority threshold, if the log priority is greater than or equal to the priority threshold, marking the original log file corresponding to the log priority as a common log file; if the log priority is less than the priority threshold, marking the original log file corresponding to the log priority as a backlog log file;

[0014] Summarize the commonly used log files and the backlog log files respectively to obtain the commonly used log set and the backlog log set;

[0015] Perform classification and compression operations on the backlog log set to obtain log compression packages, and use the common log sets to build a log graph database;

[0016] Import the log compression package into the pre-built basic server to obtain a compressed database, import the log graph database into the pre-built high-speed server to obtain a high-speed database, and complete the construction of the log storage system.

[0017] Optionally, collecting the original log set based on the log storage instruction includes:

[0018] Sending the log storage instruction to a pre-built log management system, wherein the log management system includes: a log database;

[0019] When the log management system receives the log storage instruction, it extracts a general log set from the log database, wherein the general log set includes multiple general log files;

[0020] Perform the following operations on each of the multiple common log files:

[0021] Identifying a log storage capacity of a general log file and comparing the log storage capacity with a preset storage capacity threshold;

[0022] If the log storage capacity is greater than the storage capacity threshold, the general log file is marked as a basic log file;

[0023] Aggregate basic log files to obtain multiple basic log files;

[0024] Perform the following operations on each of the multiple basic log files:

[0025] Read the log text of the basic log file and search whether the log text contains the preset first keyword or the preset second keyword;

[0026] If the first keyword or the second keyword exists in the log text, the basic log file corresponding to the log text is identified as the original log file;

[0027] Aggregate the original log files to obtain the original log set.

[0028] Optionally, the obtaining of a priority user sequence and a priority time sequence using a user search records and b time search records respectively includes:

[0029] Identify a initial user IDs from a user search records, perform a unique operation on the a initial user IDs, and obtain A unique user IDs, where the user search records correspond one-to-one to the initial user IDs, and a ≥ A;

[0030] For each of the A unique user IDs, perform the following operations:

[0031] Determine the number of user searches based on the unique user ID and the a initial user IDs, where the number of user searches is the number of initial user IDs that are identical to the unique user ID among the a initial user IDs;

[0032] Sorting A unique user IDs in descending order of the number of user searches corresponding to the unique user IDs to obtain a priority user sequence;

[0033] Identifying b search times from b time search records, and determining an earliest time and a latest time based on the b search times, wherein the time search records correspond one-to-one to the search times, the earliest time is the earliest search time among the b search times, and the latest time is the latest search time among the b search times;

[0034] Determine the search time range based on the earliest time and the latest time, where the minimum value of the search time range is the earliest time and the maximum value of the search time range is the latest time;

[0035] Perform uniform sampling on the search time range using the preset time sampling interval to obtain B sampling time ranges;

[0036] The following operations are performed on each of the B sampling time ranges:

[0037] Determine the time search quantity based on the sampling time range and the b search times, wherein the time search quantity is the number of search times within the sampling time range among the b search times;

[0038] The B sampling time ranges are sorted in descending order of the number of time retrievals corresponding to the sampling time ranges to obtain a priority time sequence.

[0039] Optionally, calculating user weights based on multidimensional data and a priority user sequence includes:

[0040] Determine whether there is a unique user ID in the priority user sequence that is the same as the initiating user ID in the multidimensional data;

[0041] If there is a unique user ID in the priority user sequence that is identical to the initiating user ID, then the unique user ID in the priority user sequence that is identical to the initiating user ID is used as the target user ID;

[0042] If there is no unique user ID identical to the initiating user ID in the priority user sequence, determining whether there is a unique user ID identical to the receiving user ID in the multidimensional data in the priority user sequence;

[0043] If there is a unique user ID in the priority user sequence that is identical to the receiving user ID, then the unique user ID in the priority user sequence that is identical to the receiving user ID is used as the target user ID;

[0044] Retrieve the user level number from the pre-built user database based on the target user ID;

[0045] Determine a user ranking value in the priority user sequence based on the target user ID, wherein the user ranking value is the position of the target user ID in the priority user sequence;

[0046] The user weight is calculated based on the user ranking value and the number of user levels. The calculation formula is as follows:

[0047]

[0048] Among them, α id is the user weight, rank id is the user ranking value, level is the number of user levels, level max is the preset maximum number of user levels, and tanh is the hyperbolic tangent function;

[0049] If there is no unique user ID identical to the receiving user ID in the priority user sequence, the user weight is recorded as 0.

[0050] Optionally, calculating the time weight according to the transaction time and priority time series in the multidimensional data includes:

[0051] Determine whether the transaction time is within the retrieval time range;

[0052] If the transaction time is within the retrieval time range, perform the following operations on each of the B sampling time ranges in turn:

[0053] Determine whether the transaction time is within the sampling time range, and if the transaction time is within the sampling time range, use the sampling time range as the target time range;

[0054] Determine a time ranking value in the priority time sequence based on the target time range, wherein the time ranking value is the position of the target time range in the priority time sequence;

[0055] Get the current time and calculate the time difference based on the current time and transaction time. The time difference is the absolute difference between the current time and transaction time.

[0056] The time weight is calculated based on the time difference and time ranking value. The calculation formula is as follows:

[0057]

[0058] Among them, β t is the time weight, rank t is the time ranking value, t0 is the time difference, and e is a natural constant;

[0059] If the transaction time is not within the retrieval time range, the time weight is recorded as 0.

[0060] Optionally, calculating the location weight according to the origination location and the headquarters location in the multidimensional data includes:

[0061] The transaction distance is obtained using the origination location and headquarters location, and the location weight is calculated using the transaction distance. The calculation formula is as follows:

[0062]

[0063] Among them, γ p is the position weight, P x is the transaction distance, and P0 is the preset standard distance.

[0064] Optionally, the calculation formula for the log priority is as follows:

[0065] χ log =(α id +β t +γ p )×ln(1+sum x )

[0066] Among them, χ log is the log priority, sum x is the transaction amount, and ln is the natural logarithm.

[0067] Optionally, performing a classification compression operation on the backlog log set to obtain a log compression package includes:

[0068] Performing a merging operation on multiple backlog log files in the backlog log set to obtain multiple log folders, wherein each of the multiple log folders includes: one or more backlog log files;

[0069] Using multiple log folders to obtain multiple file reserves, where the file reserves and log folders correspond one to one;

[0070] Determine the maximum and minimum reserves based on multiple file reserves, where the maximum reserve is the file reserve with the largest storage capacity among the multiple file reserves, and the minimum reserve is the file reserve with the smallest storage capacity among the multiple file reserves;

[0071] Perform the following operations on each of the multiple log folders:

[0072] The compression level is calculated using the maximum storage, minimum storage, and the file storage corresponding to the log folder. The calculation formula is as follows:

[0073]

[0074] Among them, L e is the compression level, C max is the maximum reserve, C min is the minimum reserve, C L is the file storage corresponding to the log folder, Z e is the preset maximum compression level, Represents the rounding operation;

[0075] Perform compression on the log folder based on the compression level to obtain a compressed file;

[0076] Aggregate the compressed files to obtain the log compression package.

[0077] Optionally, constructing a log graph database using a common log set includes:

[0078] Perform the following operations on each common log file in the common log set:

[0079] Combine the initiating user ID and initiating location corresponding to the common log files into an initiating information group, and combine the receiving user ID and receiving location corresponding to the common log files into a receiving information group;

[0080] Identify both the initiating information group and the receiving information group as user information groups;

[0081] Aggregating the user information groups to obtain multiple user information groups;

[0082] Performing a uniqueness operation on the plurality of user information groups to obtain a plurality of unique user information, and using the plurality of unique user information to create a plurality of unique user nodes in a pre-built graph database construction software, wherein the unique user nodes correspond to the unique user information in a one-to-one manner;

[0083] Perform the following operations on each common log file in the common log set:

[0084] The initiating user ID corresponding to the common log file is used as the index initiating ID, and the index initiating ID is used to retrieve the initiating node from multiple unique user nodes, wherein the initiating user ID or the receiving user ID in the unique user information corresponding to the initiating node is the same as the index initiating ID;

[0085] Get the receiving node based on the receiving user ID corresponding to the common log file;

[0086] Create relationship edges in the graph database building software based on the transaction time, transaction amount, initiating node, and receiving node corresponding to the common log files;

[0087] The relationship edges are aggregated to obtain multiple relationship edges, and a log graph database is confirmed based on multiple unique user nodes and multiple relationship edges.

[0088] To achieve the above objectives, the present invention further provides a log storage system construction system based on multi-dimensional logical relationship interaction, comprising:

[0089] a multidimensional data parsing module, configured to receive a log storage instruction, collect an original log set based on the log storage instruction, and perform a parsing operation on the original log set to obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount;

[0090] A retrieval record analysis module is used to obtain a retrieval record set, wherein the retrieval record set includes: a user retrieval records and b time retrieval records, and a user retrieval record and b time retrieval record are used to obtain a priority user sequence and a priority time sequence respectively;

[0091] A priority log classification module is used to perform the following operations on each multidimensional data in the multidimensional data set: calculate the user weight according to the multidimensional data and the priority user sequence, calculate the time weight according to the transaction time and the priority time sequence in the multidimensional data, obtain the headquarters location, calculate the location weight according to the initiation location and the headquarters location in the multidimensional data, calculate the log priority according to the user weight, time weight, location weight and transaction amount value, compare the log priority with a preset priority threshold, if the log priority is greater than or equal to the priority threshold, then mark the original log file corresponding to the log priority as a common log file, if the log priority is less than the priority threshold, then mark the original log file corresponding to the log priority as a backlog log file;

[0092] The log database construction module is used to summarize the commonly used log files and backlog log files respectively to obtain the commonly used log sets and backlog log sets, perform classification and compression operations on the backlog log sets to obtain log compression packages, use the commonly used log sets to build a log graph database, import the log compression packages into the pre-built basic servers to obtain a compressed database, import the log graph database into the pre-built high-speed servers to obtain a high-speed database, and complete the construction of the log storage system.

[0093] In order to solve the above problem, the present invention further provides an electronic device, comprising:

[0094] a memory storing at least one instruction; and

[0095] The processor executes the instructions stored in the memory to implement the above-mentioned method for constructing a log storage system based on multi-dimensional logical relationship interaction.

[0096] In order to solve the above problems, the present invention also provides a computer-readable storage medium, which stores at least one instruction. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned method for constructing a log storage system based on multi-dimensional logical relationship interaction.

[0097] The present invention is to solve the problem described in the background technology. The present invention receives a log storage instruction, collects an original log set based on the log storage instruction, performs a parsing operation on the original log set, and obtains a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: initiating user ID, initiating location, receiving user ID, receiving location, transaction time and transaction amount value. It can be seen that the embodiment of the present invention parses the original log file into multiple multidimensional data, eliminates redundant text data, saves log storage space, and facilitates subsequent classification of the original log file according to the multidimensional data, thereby obtaining a retrieval record set, wherein the retrieval record The set includes: a user search record and b time search record, and the priority user sequence and priority time sequence are obtained by using a user search record and b time search record respectively. It can be seen that the embodiment of the present invention provides reference data for subsequent calculation of log priority by analyzing historical search records, thereby improving the accuracy of subsequent classification of original log files. The following operations are performed on each multidimensional data in the multidimensional data set: user weight is calculated according to the multidimensional data and the priority user sequence, time weight is calculated according to the transaction time in the multidimensional data and the priority time sequence, headquarters location is obtained, and location weight is calculated according to the initiation location and headquarters location in the multidimensional data. It can be seen that the embodiment of the present invention performs the data of the three dimensions of user, transaction time and initiation location on the basis of the following three dimensions: user, transaction time and initiation location. A comprehensive analysis is conducted to confirm the user weight, time weight and location weight, so as to facilitate the subsequent comprehensive and accurate calculation of the log priority based on the user weight, time weight and location weight. The log priority is calculated based on the user weight, time weight, location weight and transaction amount value, and the log priority is compared with the preset priority threshold. If the log priority is greater than or equal to the priority threshold, the original log file corresponding to the log priority is identified as a common log file. If the log priority is less than the priority threshold, the original log file corresponding to the log priority is identified as a backlog log file. It can be seen that the embodiment of the present invention divides the original log files by log priority and priority threshold, identifies frequently retrieved and important files as common log files, and identifies occasionally retrieved and important files as common log files. Files that are not important and need to be retrieved are marked as backlog log files, which facilitates the subsequent use of targeted storage methods for the two types of files to save log storage space and improve log retrieval efficiency. Common log files and backlog log files are summarized separately to obtain common log sets and backlog log sets. Classification and compression operations are performed on the backlog log sets to obtain log compression packages. A log graph database is constructed using the common log sets. It can be seen that the embodiment of the present invention saves storage space by classifying and compressing the backlog log files, and constructs a log graph database using the multidimensional data corresponding to the common log files, so that only key multidimensional data is stored in the log graph database, redundant text data is removed, and storage space is saved. At the same time, the graph database is a database that is easy to retrieve.Therefore, the log retrieval efficiency is also improved. The log compression package is imported into the pre-built basic server to obtain a compressed database, and the log graph database is imported into the pre-built high-speed server to obtain a high-speed database, completing the construction of the log storage system. It can be seen that the embodiment of the present invention improves the log retrieval efficiency by importing the log graph database into a high-speed server with a faster reading speed, and stores the log compression package on the basic server, saving the computing power and storage space of other efficient servers, while also properly storing infrequently used log data. Therefore, the present invention can save log storage space and improve log retrieval efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Figure 1 A flow chart of a method for constructing a log storage system based on multi-dimensional logical relationship interaction provided by one embodiment of the present invention;

[0099] Figure 2 A functional module diagram of a log storage system construction system based on multi-dimensional logical relationship interaction provided by an embodiment of the present invention;

[0100] Figure 3 A schematic diagram of the structure of an electronic device for implementing the method for constructing a log storage system based on multi-dimensional logical relationship interaction provided by an embodiment of the present invention.

[0101] Description of reference numerals:

[0102] 1. Electronic device; 10. Processor; 11. Storage; 12. Bus.

[0103] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0104] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0105] The embodiments of the present application provide a method for constructing a log storage system based on multi-dimensional logical relationship interactions. The execution subject of the method for constructing a log storage system based on multi-dimensional logical relationship interactions includes, but is not limited to, at least one of electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiments of the present application. In other words, the method for constructing a log storage system based on multi-dimensional logical relationship interactions can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0106] Reference Figure 1FIG. 1 is a flow chart of a method for constructing a log storage system based on multi-dimensional logical relationship interaction according to an embodiment of the present invention. In this embodiment, the method for constructing a log storage system based on multi-dimensional logical relationship interaction includes:

[0107] S1. Receive a log storage instruction, collect an original log set based on the log storage instruction, perform a parsing operation on the original log set, and obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount.

[0108] In this embodiment of the present invention, the constructed log storage system is used to store transaction logs in the financial field. A transaction log is a series of logs automatically recorded by the server when a transaction (such as a payment or transfer) occurs, containing transaction information (such as the user ID and the transaction amount). Each transaction corresponds to a transaction log.

[0109] It should be noted that log storage instructions are generally initiated by system administrators. For example, consider Xiao Zhang, a system administrator at a financial company. Periodically, a large number of transaction logs accumulate on the company's servers. Failure to promptly archive these transaction logs to other storage devices can lead to insufficient server hard drive storage space, impacting system operation. Furthermore, excessive transaction log accumulation can reduce log retrieval efficiency. Therefore, to store these transaction logs and facilitate future retrieval and review, Xiao Zhang initiates a log storage instruction.

[0110] In detail, the collecting of the original log set based on the log storage instruction includes:

[0111] Sending the log storage instruction to a pre-built log management system, wherein the log management system includes: a log database;

[0112] When the log management system receives the log storage instruction, it extracts a general log set from the log database, wherein the general log set includes multiple general log files;

[0113] Perform the following operations on each of the multiple common log files:

[0114] Identifying a log storage capacity of a general log file and comparing the log storage capacity with a preset storage capacity threshold;

[0115] If the log storage capacity is greater than the storage capacity threshold, the general log file is marked as a basic log file;

[0116] Aggregate basic log files to obtain multiple basic log files;

[0117] Perform the following operations on each of the multiple basic log files:

[0118] Read the log text of the basic log file and search whether the log text contains the preset first keyword or the preset second keyword;

[0119] If the first keyword or the second keyword exists in the log text, the basic log file corresponding to the log text is identified as the original log file;

[0120] Aggregate the original log files to obtain the original log set.

[0121] It's important to explain that the log management system is a server used by financial companies to record various logs (such as operation logs, error logs, and transaction logs). The log management system also includes a log database. The log database is the database used to store logs within the log management system. A general log set consists of multiple general log files, which are log files stored in the log database over a certain period of time.

[0122] It should be understood that the embodiment of the present invention filters out transaction logs from multiple logs (such as operation logs or error logs) by comparing storage capacity and search keywords.

[0123] It should be noted that log storage capacity refers to the size of the general log file. For example, if the general log file size is 1 KB, the log storage capacity is 1 KB. The storage capacity threshold is related to the file size of the transaction log and is set by the system administrator.

[0124] It should be understood that the technology for reading the log text of the basic log file is an existing technology and will not be described in detail here.

[0125] Log text refers to the string of characters recorded in the basic log file, containing key event information that helps developers, operations personnel, and system administrators understand system behavior. Alternatively, select user_id as the first keyword and amount as the second keyword. Since transaction logs differ from other logs in that they contain data such as user IDs and billing information, setting the first and second keywords allows you to filter transaction log files from multiple basic log files. The original log file is the filtered transaction log file.

[0126] It should be understood that performing a parsing operation on the original log set to obtain a multidimensional data set means performing a parsing operation on the original log files in the original log set in sequence to obtain multidimensional data, and summarizing the multidimensional data to obtain a multidimensional data set. Performing a parsing operation on the original log files in the original log set means parsing the information contained in the original log files from the text of the original log files. For example: Xiao Zhao's user ID is 12345, and the longitude and latitude coordinates of Xiao Zhao's location are (39.9062, 116.3912). Xiao Wang's user ID is 67890, and the longitude and latitude coordinates of Xiao Wang's location are (39.5518, 116.2324). At 15:00:01 on 2024-11-21, Xiao Zhao transferred 500 yuan to Xiao Wang, then the text of the original log file will contain the following information: "timestamp":"2024-11-21T15:00:01Z"

[0127] "amount":500.00

[0128] "sender":{"user_id":"12345","coordinates":{"latitude":39.9062,"longitude":116.3912}}

[0129] "receiver":{"user_id":"67890","coordinates":{"latitude":39.5518,"longitude":116.2324}}

[0130] Here, timestamp represents the transaction time, amount represents the transaction amount, sender represents the initiator of the transfer, receiver represents the receiver of the transfer, user_id represents the user ID, coordinates represents the longitude and latitude coordinates, latitude represents the latitude, and longitude represents the longitude. The information in parentheses after "sender" represents the initiator's information. For example, the "user_id" in the parentheses after "sender" refers to the initiator's ID, and the "coordinates" in the parentheses after "sender" refers to the initiator's location. The receiver's ID and coordinates can be obtained similarly. Parsing the text of the original log file reveals that the transaction time is 15:00:01 on November 21, 2024, the transaction amount is 500, the initiating user ID is 12345, the initiating location is the real-world location corresponding to the longitude and latitude coordinates (39.9062, 116.3912), the receiving user ID is 67890, and the receiving location is the real-world location corresponding to the longitude and latitude coordinates (39.5518, 116.2324), where the latitude comes first and the longitude comes last. The technique for parsing the original log files in the original log set is prior art and will not be further elaborated here.

[0131] S2. Obtain a search record set, wherein the search record set includes: a user search records and b time search records, and respectively use a user search records and b time search records to obtain a priority user sequence and a priority time sequence.

[0132] For example, when a finance company employee needs to query a user's transaction records by searching the logs, they search the log database for the user's ID. The log management system records the user ID during the search, and then counts a user ID recorded by the log management system over a period of time as a user search record. When a finance company employee needs to query transaction records around a certain time point, they search the log database for that time point. Similarly, they count b time points recorded by the log management system over a period of time as b time search records. The search record set consists of a user search records and b time search records, and can be read directly from the log management system.

[0133] Generally speaking, in many searches within a period of time, the same user ID may be searched multiple times, so that the user ID may be repeated in the recorded a user IDs. Therefore, it is necessary to remove the repeated user IDs through subsequent unique operations.

[0134] Specifically, the method of obtaining a priority user sequence and a priority time sequence by using a user search record and b time search record respectively includes:

[0135] Identify a initial user IDs from a user search records, perform a unique operation on the a initial user IDs, and obtain A unique user IDs, where the user search records correspond one-to-one to the initial user IDs, and a ≥ A;

[0136] For each of the A unique user IDs, perform the following operations:

[0137] Determine the number of user searches based on the unique user ID and the a initial user IDs, where the number of user searches is the number of initial user IDs that are identical to the unique user ID among the a initial user IDs;

[0138] Sorting A unique user IDs in descending order of the number of user searches corresponding to the unique user IDs to obtain a priority user sequence;

[0139] Identifying b search times from b time search records, and determining an earliest time and a latest time based on the b search times, wherein the time search records correspond one-to-one to the search times, the earliest time is the earliest search time among the b search times, and the latest time is the latest search time among the b search times;

[0140] Determine the search time range based on the earliest time and the latest time, where the minimum value of the search time range is the earliest time and the maximum value of the search time range is the latest time;

[0141] Perform uniform sampling on the search time range using the preset time sampling interval to obtain B sampling time ranges;

[0142] The following operations are performed on each of the B sampling time ranges:

[0143] Determine the time search quantity based on the sampling time range and the b search times, wherein the time search quantity is the number of search times within the sampling time range among the b search times;

[0144] The B sampling time ranges are sorted in descending order of the number of time retrievals corresponding to the sampling time ranges to obtain a priority time sequence.

[0145] It should be explained that the initial user ID refers to the user ID in the user search record, and the search time refers to the time point in the time search record. Performing a unique operation on a number of initial user IDs to obtain A unique user IDs means removing duplicate initial user IDs from the number of initial user IDs, so that only one unique user ID remains among multiple identical initial user IDs. For example, if the six initial user IDs are 123, 124, 124, 125, 125, and 125, removing one duplicate user ID, which is 124, and two duplicate user IDs, which are 125, yields three unique user IDs: 123, 124, and 125. The unique operation can be implemented using the numpy.unique() function in Python.

[0146] It is understandable that the sampling interval is related to the earliest time and the latest time. Optionally, the sampling interval is set to one thirtieth of the absolute difference between the earliest time and the latest time.

[0147] For example, the retrieval time range is: (October 1, 2024 00:00:00, October 31, 2024 00:00:00), and the sampling interval is set to 24 hours, then 30 sampling time ranges are obtained: (October 1, 2024 00:00:00, October 2, 2024 00:00:00), ..., (October 30, 2024 00:00:00, October 31, 2024 00:00:00).

[0148] S3. Perform the following operations on each multidimensional data in the multidimensional data set: calculate the user weight based on the multidimensional data and the priority user sequence, and calculate the time weight based on the transaction time and the priority time sequence in the multidimensional data.

[0149] Specifically, the calculation of user weights based on multidimensional data and priority user sequences includes:

[0150] Determine whether there is a unique user ID in the priority user sequence that is the same as the initiating user ID in the multidimensional data;

[0151] If there is a unique user ID in the priority user sequence that is identical to the initiating user ID, then the unique user ID in the priority user sequence that is identical to the initiating user ID is used as the target user ID;

[0152] If there is no unique user ID identical to the initiating user ID in the priority user sequence, determining whether there is a unique user ID identical to the receiving user ID in the multidimensional data in the priority user sequence;

[0153] If there is a unique user ID in the priority user sequence that is identical to the receiving user ID, then the unique user ID in the priority user sequence that is identical to the receiving user ID is used as the target user ID;

[0154] Retrieve the user level number from the pre-built user database based on the target user ID;

[0155] Determine a user ranking value in the priority user sequence based on the target user ID, wherein the user ranking value is the position of the target user ID in the priority user sequence;

[0156] The user weight is calculated based on the user ranking value and the number of user levels. The calculation formula is as follows:

[0157]

[0158] Among them, α id is the user weight, rank id is the user ranking value, level is the number of user levels, level max is the preset maximum number of user levels, and tanh is the hyperbolic tangent function;

[0159] If there is no unique user ID identical to the receiving user ID in the priority user sequence, the user weight is recorded as 0.

[0160] It should be explained that the user database is composed of user information stored in the server in advance, and the user information includes user level. The user level is the level assigned to each user by the financial company based on the user's account flow and credit, etc. The specific rating standards are related to the financial company.

[0161] For example, Xiao Zhao is a user of a financial company with a user level of 8, that is, the number of user levels is 8, and his user ID is 1234567. It is found that the user ID 1234567 is ranked 10th in the priority user sequence, that is, the user ranking value is 10. If there are 100 unique user IDs in the priority user sequence, that is, A is 100, if the highest user level of the financial company during rating is 10, that is, the maximum number of user levels is 10, and the user weight is calculated to be 0.5976.

[0162] It should be understood that the more frequently the user ID corresponding to an original log file is retrieved and the higher the level of the user corresponding to the original log file, the more important the original log file is. Therefore, the user weight reflects the importance of the original log file. The higher the user weight, the more important the original log file is, and the higher the priority it has in subsequent storage.

[0163] Specifically, the time weight is calculated based on the transaction time and priority time series in the multidimensional data, including:

[0164] Determine whether the transaction time is within the retrieval time range;

[0165] If the transaction time is within the retrieval time range, perform the following operations on each of the B sampling time ranges in turn:

[0166] Determine whether the transaction time is within the sampling time range, and if the transaction time is within the sampling time range, use the sampling time range as the target time range;

[0167] Determine a time ranking value in the priority time sequence based on the target time range, wherein the time ranking value is the position of the target time range in the priority time sequence;

[0168] Get the current time and calculate the time difference based on the current time and transaction time. The time difference is the absolute difference between the current time and transaction time.

[0169] The time weight is calculated based on the time difference and time ranking value. The calculation formula is as follows:

[0170]

[0171] Among them, β t is the time weight, rank t is the time ranking value, t0 is the time difference, and e is a natural constant;

[0172] If the transaction time is not within the retrieval time range, the time weight is recorded as 0.

[0173] It should be understood that the method for determining the time ranking value in the priority time sequence based on the target time range is the same as the method for determining the user ranking value in the priority user sequence based on the target user ID, and will not be repeated here.

[0174] It should be explained that the current time refers to the time when the time ranking value is confirmed. Optionally, when the time ranking value is confirmed, the Beijing time at that time is queried as the current time.

[0175] It should be understood that the smaller the time difference, the later the original log file was generated, the newer the original log file is, the greater the probability of being used for retrieval, and the more important the original log file is. Therefore, the time weight reflects the importance of the original log file. The higher the time weight, the more important the original log file is, and the higher the priority it has in subsequent storage.

[0176] S4. Obtain the headquarters location and calculate the location weight based on the initiation location and headquarters location in the multidimensional data.

[0177] It should be explained that the headquarters location refers to the latitude and longitude coordinates of the financial company in the real world. Optionally, the headquarters location can be obtained through a global positioning system.

[0178] Specifically, the calculation of the location weight according to the origination location and the headquarters location in the multidimensional data includes:

[0179] The transaction distance is obtained using the origination location and headquarters location, and the location weight is calculated using the transaction distance. The calculation formula is as follows:

[0180]

[0181] Among them, γ p is the position weight, P x is the transaction distance, and P0 is the preset standard distance.

[0182] It should be explained that the transaction distance refers to the distance between the initiating location and the headquarters location in the real world. The transaction distance can be calculated by the latitude and longitude coordinates of the initiating location and the latitude and longitude coordinates of the headquarters location. The technology for calculating the transaction distance by the latitude and longitude of the initiating location and the latitude and longitude of the headquarters location is an existing technology and will not be repeated here.

[0183] For example, in order to facilitate communication between the finance company and users, the finance company only accepts users within 50 km of the finance company, and thus the standard distance is set to 50 km.

[0184] It should be understood that the smaller the transaction distance, the closer the user is to the financial company at the time of the transaction, and the greater the probability that the user will come to the financial company for consultation, that is, the greater the probability that the original log file corresponding to the transaction distance is used for retrieval, which means that the importance of the original log file is higher. Therefore, the position weight reflects the importance of the original log file. The higher the position weight, the more important the original log file is, and the higher the priority in subsequent storage.

[0185] S5. Calculate the log priority based on the user weight, time weight, location weight, and transaction amount.

[0186] In detail, the calculation formula of the log priority is as follows:

[0187] χ log =(α id +β t +γ p )×ln(1+sum x )

[0188] Among them, χlog is the log priority, sum x is the transaction amount, and ln is the natural logarithm.

[0189] It should be understood that the log priority reflects the importance of the original log file. The higher the log priority, the more important the original log file.

[0190] S6. Compare the log priority with the preset priority threshold. If the log priority is greater than or equal to the priority threshold, the original log file corresponding to the log priority is identified as a common log file. If the log priority is less than the priority threshold, the original log file corresponding to the log priority is identified as a backlog log file.

[0191] Optionally, the system administrator selects multiple original log files that have been retrieved, and calculates the log priority of each of the multiple original log files in sequence, and finally uses the smallest log priority among the calculated multiple log priorities as the priority threshold.

[0192] It should be understood that the embodiment of the present invention divides the original log files into frequently used log files and backlog log files through a priority threshold, so that the frequently used log files and the backlog log files are stored separately in the future. Among them, the frequently used log files are original log files with high importance and are expected to be frequently retrieved. Therefore, a graph database storage method that is easy to retrieve is adopted in the future. The backlog log files are original log files with low importance and are expected to be occasionally retrieved. Therefore, they are stored in a classified and compressed manner in the future.

[0193] S7. Summarize the commonly used log files and the backlog log files respectively to obtain a commonly used log set and a backlog log set, perform classification and compression operations on the backlog log set to obtain a log compression package, and use the commonly used log set to build a log graph database.

[0194] Specifically, the classified compression operation is performed on the backlog log set to obtain a log compression package, including:

[0195] Performing a merging operation on multiple backlog log files in the backlog log set to obtain multiple log folders, wherein each of the multiple log folders includes: one or more backlog log files;

[0196] Using multiple log folders to obtain multiple file reserves, where the file reserves and log folders correspond one to one;

[0197] Determine the maximum and minimum reserves based on multiple file reserves, where the maximum reserve is the file reserve with the largest storage capacity among the multiple file reserves, and the minimum reserve is the file reserve with the smallest storage capacity among the multiple file reserves;

[0198] Perform the following operations on each of the multiple log folders:

[0199] The compression level is calculated using the maximum storage, minimum storage, and the file storage corresponding to the log folder. The calculation formula is as follows:

[0200]

[0201] Among them, L e is the compression level, C max is the maximum reserve, C min is the minimum reserve, C L is the file storage corresponding to the log folder, Z e is the preset maximum compression level, Represents the rounding operation;

[0202] Perform compression on the log folder based on the compression level to obtain a compressed file;

[0203] Aggregate the compressed files to obtain the log compression package.

[0204] It should be explained that performing a merging operation on multiple backlog log files in a backlog log set means: integrating the backlog log files with the same initiating user ID in multiple backlog log files into one folder, and the merged folder is the log folder. For example: there are 5 backlog log files in total, the initiating user ID corresponding to the first backlog log file is 123, the initiating user ID corresponding to the second backlog log file is 124, the initiating user ID corresponding to the third backlog log file is 125, the initiating user ID corresponding to the fourth backlog log file is 123, and the initiating user ID corresponding to the fifth backlog log file is 124. Therefore, the first backlog log file and the fourth backlog log file are integrated into one folder, the second backlog log file and the fifth backlog log file are integrated into one folder, and the third backlog log file is integrated into one folder, and finally 3 log folders are obtained.

[0205] It's understood that file size refers to the size of the log folder. For example, if the log folder is 100 KB, the file size is 100 KB. The compression level reflects the compression ratio used when compressing the log folder. The higher the compression level, the greater the compression ratio. Different compression software offers different levels. For example, 7-Zip offers levels from 0 to 9, while Zstandard offers levels from 1 to 22. The maximum compression level depends on the compression software used. If 7-Zip is used, the maximum compression level is 9.

[0206] It should be understood that the larger the file storage capacity of the log folder corresponding to a compressed file, the more backlog log files there are in the log folder, and the greater the probability that the compressed file will be retrieved when the financial company retrieves the log. When the compressed file is retrieved, the compressed file needs to be decompressed to extract the backlog log files required by the financial company in the compressed file. Because the greater the compression level, the longer the time it takes to decompress the compressed file, a lower compression level will be used when compressing the log folder with a greater probability of being retrieved, that is, the log folder with a larger file storage capacity, thereby reducing the time consumed in decompression.

[0207] For example, if the compression level is 8, the compression level of 7-Zip is set to 8 when performing the compression operation on the log folder.

[0208] Optionally, the following command is used to set the compression level of 7-Zip to level 8 when performing compression operation on the log folder: 7z a-t7z-mx=8example.7z example.txt. The technology of performing compression operation on the log folder is prior art and will not be described in detail here.

[0209] In detail, the construction of a log graph database using a common log set includes:

[0210] Perform the following operations on each common log file in the common log set:

[0211] Combine the initiating user ID and initiating location corresponding to the common log files into an initiating information group, and combine the receiving user ID and receiving location corresponding to the common log files into a receiving information group;

[0212] Identify both the initiating information group and the receiving information group as user information groups;

[0213] Aggregating the user information groups to obtain multiple user information groups;

[0214] Performing a uniqueness operation on the plurality of user information groups to obtain a plurality of unique user information, and using the plurality of unique user information to create a plurality of unique user nodes in a pre-built graph database construction software, wherein the unique user nodes correspond to the unique user information in a one-to-one manner;

[0215] Perform the following operations on each common log file in the common log set:

[0216] The initiating user ID corresponding to the common log file is used as the index initiating ID, and the index initiating ID is used to retrieve the initiating node from multiple unique user nodes, wherein the initiating user ID or the receiving user ID in the unique user information corresponding to the initiating node is the same as the index initiating ID;

[0217] Get the receiving node based on the receiving user ID corresponding to the common log file;

[0218] Create relationship edges in the graph database building software based on the transaction time, transaction amount, initiating node, and receiving node corresponding to the common log files;

[0219] The relationship edges are aggregated to obtain multiple relationship edges, and a log graph database is confirmed based on multiple unique user nodes and multiple relationship edges.

[0220] It should be understood that the method of performing a unique operation on multiple user information groups to obtain multiple unique user information is the same as the method of performing a unique operation on a initial user ID to obtain A unique user IDs, and will not be repeated here.

[0221] In this embodiment of the present invention, the graph database construction software used is the Neo4j graph database. A unique user node is a basic unit in the Neo4j graph database for storing unique user information. A relationship edge is a structure located between two unique user nodes in the Neo4j graph database, reflecting the relationship between the two unique user nodes.

[0222] For example, if the initiating user ID or receiving user ID in a unique user information is 12345, and the initiating location or receiving location is the latitude and longitude coordinates (39.9062, 116.3912), then write the following code in the command line of Neo4j Browser and run it to create a unique user node:

[0223] CREATE(U1:User{

[0224] id:"12345",

[0225] latitude:39.9062,

[0226] longitude:116.3912

[0227] }), where CREATE represents creation, User is the label of the unique user node, indicating that the unique user node is of the "user" type. Labels can help system administrators categorize unique user nodes or optimize the query process, id represents the initiating user ID or the receiving user ID, latitude represents the latitude in longitude and latitude coordinates, and longitude represents the longitude in longitude and latitude coordinates. U1 is the reference name of the node. The reference name is created by the system administrator when creating a unique user node, and each unique user node corresponds to a reference name. When creating relationship edges later, the reference names corresponding to the initiating node and the receiving node are used to write code to create the relationship edges.

[0228] It should be understood that the method of obtaining the receiving node based on the receiving user ID corresponding to the common log file is the same as the method of obtaining the initiating node using the initiating user ID corresponding to the common file, and will not be repeated here.

[0229] For example, if the transaction time is 10:00:00 on November 21, 2024, the transaction amount is 200, the reference name corresponding to the initiating node is U1, and the reference name corresponding to the receiving node is U2, then write the following code in the command line of Neo4j Browser and run it to create the relationship edge:

[0230] CREATE(U1)-[:TRANSACTION{

[0231] amount:200,

[0232] timestamp:datetime("2024-11-21T10:00:00")

[0233] }]->(U2), where TRANSACTION indicates that the edge type is a "transaction" relationship, amount represents the transaction amount, timestamp represents the transaction time, and datetime() is a function used to define the data in the brackets as a date and time type. It is understood that the confirmation of the log graph database based on multiple unique user nodes and multiple relationship edges means that when multiple unique user nodes and multiple relationship edges are successfully created in the graph database construction software, the log graph database can be confirmed to have been successfully constructed.

[0234] S8. Import the log compression package into the pre-built basic server to obtain a compressed database, import the log graph database into the pre-built high-speed server to obtain a high-speed database, and complete the construction of the log storage system.

[0235] It should be understood that the basic server is a server that uses a mechanical hard disk to store log compression packages. Since the log compression packages store infrequently used and unimportant original log files, they do not require very high retrieval or reading speeds. Therefore, a mechanical hard disk is used in the basic server to prioritize storage capacity and economically store more log compression packages. The high-speed server is a server that uses a solid-state hard disk to store a log graph database. Since the log graph database corresponds to frequently used and important original log files and needs to be frequently retrieved and read, a solid-state hard disk is used in the high-speed server to prioritize reading speed, thereby efficiently realizing log retrieval. The compressed database is a database that stores log compression packages in the basic server. The high-speed database is a log graph database stored in the high-speed server. The technology for importing the log graph database into a pre-built high-speed server is prior art and will not be repeated here.

[0236] The present invention is to solve the problem described in the background technology. The present invention receives a log storage instruction, collects an original log set based on the log storage instruction, performs a parsing operation on the original log set, and obtains a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: initiating user ID, initiating location, receiving user ID, receiving location, transaction time and transaction amount value. It can be seen that the embodiment of the present invention parses the original log file into multiple multidimensional data, eliminates redundant text data, saves log storage space, and facilitates subsequent classification of the original log file according to the multidimensional data, thereby obtaining a retrieval record set, wherein the retrieval record The set includes: a user search record and b time search record, and the priority user sequence and priority time sequence are obtained by using a user search record and b time search record respectively. It can be seen that the embodiment of the present invention provides reference data for subsequent calculation of log priority by analyzing historical search records, thereby improving the accuracy of subsequent classification of original log files. The following operations are performed on each multidimensional data in the multidimensional data set: user weight is calculated according to the multidimensional data and the priority user sequence, time weight is calculated according to the transaction time in the multidimensional data and the priority time sequence, headquarters location is obtained, and location weight is calculated according to the initiation location and headquarters location in the multidimensional data. It can be seen that the embodiment of the present invention performs the data of the three dimensions of user, transaction time and initiation location on the basis of the following three dimensions: user, transaction time and initiation location. A comprehensive analysis is conducted to confirm the user weight, time weight and location weight, so as to facilitate the subsequent comprehensive and accurate calculation of the log priority based on the user weight, time weight and location weight. The log priority is calculated based on the user weight, time weight, location weight and transaction amount value, and the log priority is compared with the preset priority threshold. If the log priority is greater than or equal to the priority threshold, the original log file corresponding to the log priority is identified as a common log file. If the log priority is less than the priority threshold, the original log file corresponding to the log priority is identified as a backlog log file. It can be seen that the embodiment of the present invention divides the original log files by log priority and priority threshold, identifies frequently retrieved and important files as common log files, and identifies occasionally retrieved and important files as common log files. Files that are not important and need to be retrieved are marked as backlog log files, which facilitates the subsequent use of targeted storage methods for the two types of files to save log storage space and improve log retrieval efficiency. Common log files and backlog log files are summarized separately to obtain common log sets and backlog log sets. Classification and compression operations are performed on the backlog log sets to obtain log compression packages. A log graph database is constructed using the common log sets. It can be seen that the embodiment of the present invention saves storage space by classifying and compressing the backlog log files, and constructs a log graph database using the multidimensional data corresponding to the common log files, so that only key multidimensional data is stored in the log graph database, redundant text data is removed, and storage space is saved. At the same time, the graph database is a database that is easy to retrieve.Therefore, the log retrieval efficiency is also improved. The log compression package is imported into the pre-built basic server to obtain a compressed database, and the log graph database is imported into the pre-built high-speed server to obtain a high-speed database, completing the construction of the log storage system. It can be seen that the embodiment of the present invention improves the log retrieval efficiency by importing the log graph database into a high-speed server with a faster reading speed, and stores the log compression package on the basic server, saving the computing power and storage space of other efficient servers, while also properly storing infrequently used log data. Therefore, the present invention can save log storage space and improve log retrieval efficiency.

[0237] like Figure 2 FIG. 1 is a functional module diagram of a log storage system construction system based on multi-dimensional logical relationship interaction provided by an embodiment of the present invention.

[0238] The log storage system construction system 100 based on multidimensional logical relationship interaction described in the present invention can be installed in an electronic device. Depending on the functions implemented, the log storage system construction system 100 based on multidimensional logical relationship interaction can include a multidimensional data parsing module 101, a retrieval record analysis module 102, a priority log classification module 103, and a log database construction module 104. The modules described in the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can perform fixed functions, and are stored in the memory of the electronic device.

[0239] The multidimensional data parsing module 101 is configured to receive a log storage instruction, collect an original log set based on the log storage instruction, and perform a parsing operation on the original log set to obtain a multidimensional data set, wherein the original log set includes multiple original log files, and the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount;

[0240] The retrieval record analysis module 102 is used to obtain a retrieval record set, wherein the retrieval record set includes: a user retrieval records and b time retrieval records, and a user retrieval record and b time retrieval record are used to obtain a priority user sequence and a priority time sequence respectively;

[0241] The priority log classification module 103 is used to perform the following operations on each multidimensional data in the multidimensional data set: calculate the user weight according to the multidimensional data and the priority user sequence, calculate the time weight according to the transaction time and the priority time sequence in the multidimensional data, obtain the headquarters location, calculate the location weight according to the initiation location and the headquarters location in the multidimensional data, calculate the log priority according to the user weight, time weight, location weight and transaction amount value, compare the log priority with a preset priority threshold, if the log priority is greater than or equal to the priority threshold, mark the original log file corresponding to the log priority as a common log file, if the log priority is less than the priority threshold, mark the original log file corresponding to the log priority as a backlog log file;

[0242] The log database construction module 104 is used to summarize the commonly used log files and the backlog log files respectively to obtain the commonly used log set and the backlog log set, perform classification and compression operations on the backlog log set to obtain a log compression package, use the commonly used log set to build a log graph database, import the log compression package into a pre-built basic server to obtain a compressed database, import the log graph database into a pre-built high-speed server to obtain a high-speed database, and complete the construction of the log storage system.

[0243] In detail, the modules in the log storage system construction system 100 based on multi-dimensional logical relationship interaction in the embodiment of the present invention are used in the same manner as above. Figure 1 The same technical means are used as the method for building a log storage system based on multi-dimensional logical relationship interaction described in , and can produce the same technical effects, so I will not go into details here.

[0244] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing a method for constructing a log storage system based on multi-dimensional logical relationship interaction provided by an embodiment of the present invention.

[0245] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a method program for constructing a log storage system based on multi-dimensional logical relationship interaction.

[0246] The memory 11 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 1. Furthermore, the memory 11 also includes an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed on the electronic device 1, such as the code of a log storage system construction method program based on multi-dimensional logical relationship interaction, but can also be used to temporarily store data that has been output or is to be output.

[0247] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing the programs or modules stored in the memory 11 (such as a log storage system construction method program based on multi-dimensional logical relationship interaction, etc.), as well as calling the data stored in the memory 11, to perform various functions of the electronic device 1 and process data.

[0248] The bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 may be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0249] Figure 3 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 3The structure shown does not constitute a limitation on the electronic device 1 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0250] For example, although not shown, the electronic device 1 may further include a power source (such as a battery) for powering the various components. Preferably, the power source may be logically connected to the at least one processor 10 via a power management device, thereby implementing functions such as charging management, discharging management, and power consumption management through the power management device. The power source may further include any components such as one or more DC or AC power sources, a recharging device, a power failure detection circuit, a power converter or inverter, a power status indicator, etc. The electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0251] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0252] Optionally, the electronic device 1 may further include a user interface, which may be a display or an input unit (such as a keyboard). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0253] The program of the method for constructing a log storage system based on multi-dimensional logical relationship interaction stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When executed in the processor 10, the following can be achieved:

[0254] receiving a log storage instruction, collecting an original log set based on the log storage instruction, performing a parsing operation on the original log set to obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount;

[0255] Obtaining a retrieval record set, wherein the retrieval record set includes: a user retrieval records and b time retrieval records, and obtaining a priority user sequence and a priority time sequence using the a user retrieval records and the b time retrieval records respectively;

[0256] The following operations are performed on each multidimensional data in the multidimensional dataset:

[0257] Calculate user weights based on multidimensional data and priority user sequences, and calculate time weights based on transaction times and priority time sequences in multidimensional data;

[0258] Get the headquarters location and calculate the location weight based on the originating location and headquarters location in the multidimensional data;

[0259] Calculate log priority based on user weight, time weight, location weight and transaction amount;

[0260] Comparing the log priority with a preset priority threshold, if the log priority is greater than or equal to the priority threshold, marking the original log file corresponding to the log priority as a common log file; if the log priority is less than the priority threshold, marking the original log file corresponding to the log priority as a backlog log file;

[0261] Summarize the commonly used log files and the backlog log files respectively to obtain the commonly used log set and the backlog log set;

[0262] Perform classification and compression operations on the backlog log set to obtain log compression packages, and use the common log sets to build a log graph database;

[0263] Import the log compression package into the pre-built basic server to obtain a compressed database, import the log graph database into the pre-built high-speed server to obtain a high-speed database, and complete the construction of the log storage system.

[0264] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 3 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0265] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0266] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor of an electronic device, the computer program can implement:

[0267] receiving a log storage instruction, collecting an original log set based on the log storage instruction, performing a parsing operation on the original log set to obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data have a one-to-one correspondence, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount;

[0268] Obtaining a retrieval record set, wherein the retrieval record set includes: a user retrieval records and b time retrieval records, and obtaining a priority user sequence and a priority time sequence using the a user retrieval records and the b time retrieval records respectively;

[0269] The following operations are performed on each multidimensional data in the multidimensional dataset:

[0270] Calculate user weights based on multidimensional data and priority user sequences, and calculate time weights based on transaction times and priority time sequences in multidimensional data;

[0271] Get the headquarters location and calculate the location weight based on the originating location and headquarters location in the multidimensional data;

[0272] Calculate log priority based on user weight, time weight, location weight and transaction amount;

[0273] Comparing the log priority with a preset priority threshold, if the log priority is greater than or equal to the priority threshold, marking the original log file corresponding to the log priority as a common log file; if the log priority is less than the priority threshold, marking the original log file corresponding to the log priority as a backlog log file;

[0274] Summarize the commonly used log files and the backlog log files respectively to obtain the commonly used log set and the backlog log set;

[0275] Perform classification and compression operations on the backlog log set to obtain log compression packages, and use the common log sets to build a log graph database;

[0276] Import the log compression package into the pre-built basic server to obtain a compressed database, import the log graph database into the pre-built high-speed server to obtain a high-speed database, and complete the construction of the log storage system.

[0277] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only exemplary, and actual implementations may have other division methods.

[0278] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of the modules may be selected to achieve the purpose of the solution of this embodiment according to actual needs.

[0279] In addition, the functional modules in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional modules.

[0280] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0281] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for constructing a log storage system based on multi-dimensional logical relationship interaction, characterized in that: The method comprises: Receive a log storage instruction, collect an original log set based on the log storage instruction, perform a parsing operation on the original log set, and obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data correspond one to one, wherein the multidimensional data includes: an initiating user ID, an initiating location, a receiving user ID, a receiving location, a transaction time, and a transaction amount value; Obtaining a search record set, wherein the search record set includes: a user search records and b time search records, and obtaining a priority user sequence and a priority time sequence using the a user search records and the b time search records respectively; The following operations are performed on each multidimensional data in the multidimensional dataset: Calculate user weights based on multidimensional data and priority user sequences, and calculate time weights based on transaction times and priority time sequences in multidimensional data; Get the headquarters location and calculate the location weight based on the originating location and headquarters location in the multidimensional data; Calculate log priority based on user weight, time weight, location weight and transaction amount value; Compare the log priority with the preset priority threshold, if the log priority is greater than or equal to the priority threshold, mark the original log file corresponding to the log priority as a common log file, if the log priority is less than the priority threshold, mark the original log file corresponding to the log priority as a backlog log file; Summarize the commonly used log files and the backlog log files respectively to obtain the commonly used log set and the backlog log set; Perform classification and compression operations on the backlog log sets to obtain log compression packages, and use common log sets to build a log graph database; Import the log compression package into the pre-built basic server to obtain a compressed database, import the log graph database into the pre-built high-speed server to obtain a high-speed database, and complete the construction of the log storage system.

2. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 1, characterized in that: The collecting of the original log set based on the log storage instruction includes: Sending the log storage instruction to a pre-built log management system, wherein the log management system includes: a log database; When the log management system receives the log storage instruction, it extracts a general log set from the log database, wherein the general log set includes a plurality of general log files; Perform the following operations on each of the multiple common log files: Identify the log storage capacity of the general log file, and compare the log storage capacity with a preset storage capacity threshold; If the log storage capacity is greater than the storage capacity threshold, the general log file is marked as a basic log file; Aggregate basic log files to obtain multiple basic log files; For each of the multiple basic log files, perform the following operations: Read the log text of the basic log file, and search whether a preset first keyword or a preset second keyword exists in the log text; If the first keyword or the second keyword exists in the log text, the basic log file corresponding to the log text is identified as the original log file; Aggregate the original log files to obtain the original log set.

3. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 2, characterized in that: The method of respectively using a user search record and b time search record to obtain a priority user sequence and a priority time sequence includes: Identify a initial user IDs from a user search records, perform a unique operation on the a initial user IDs, and obtain A unique user IDs, where the user search records correspond to the initial user IDs one by one, and a≥A; For each of the A unique user IDs, perform the following operations: Determine the number of user searches based on the unique user ID and the a initial user IDs, wherein the number of user searches is the number of initial user IDs that are the same as the unique user ID among the a initial user IDs; Sorting A unique user IDs in descending order of the number of user searches corresponding to the unique user IDs to obtain a priority user sequence; Identify b search times in b time search records, and confirm the earliest time and the latest time based on the b search times, wherein the time search record corresponds to the search time one by one, the earliest time is the earliest search time among the b search times, and the latest time is the latest search time among the b search times; Determine the search time range based on the earliest time and the latest time, wherein the minimum value of the search time range is the earliest time and the maximum value of the search time range is the latest time; Perform uniform sampling on the search time range using a preset time sampling interval to obtain B sampling time ranges; The following operations are performed for each of the B sampling time ranges: Determine the time search quantity based on the sampling time range and the b search times, wherein the time search quantity is the number of search times in the b search times that are within the sampling time range; The B sampling time ranges are sorted in descending order of the number of time retrievals corresponding to the sampling time ranges to obtain a priority time sequence.

4. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 3, characterized in that: The calculating of user weights according to the multi-dimensional data and the priority user sequence includes: Determine whether there is a unique user ID in the priority user sequence that is the same as the initiating user ID in the multidimensional data; If there is a unique user ID in the priority user sequence that is the same as the initiating user ID, then the unique user ID in the priority user sequence that is the same as the initiating user ID is used as the target user ID; If there is no unique user ID in the priority user sequence that is the same as the initiating user ID, determining whether there is a unique user ID in the priority user sequence that is the same as the receiving user ID in the multidimensional data; If there is a unique user ID in the priority user sequence that is identical to the receiving user ID, then the unique user ID in the priority user sequence that is identical to the receiving user ID is used as the target user ID; Retrieve the user level number from the pre-built user database based on the target user ID; Determine a user ranking value in the priority user sequence based on the target user ID, wherein the user ranking value is the position of the target user ID in the priority user sequence; The user weight is calculated based on the user ranking value and the number of user levels. The calculation formula is as follows: Among them, α id is the user weight, rank id is the user ranking value, level is the number of user levels, level max is the preset maximum number of user levels, tanh is the hyperbolic tangent function; If there is no unique user ID identical to the receiving user ID in the priority user sequence, the user weight is recorded as 0.

5. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 4, characterized in that: The time weight is calculated according to the transaction time and the priority time series in the multidimensional data, including: Determine whether the transaction time is within the retrieval time range; If the transaction time is within the retrieval time range, perform the following operations on each of the B sampling time ranges in turn: Determine whether the transaction time is within the sampling time range, and if the transaction time is within the sampling time range, use the sampling time range as the target time range; Determine a time ranking value in the priority time sequence based on the target time range, wherein the time ranking value is the position of the target time range in the priority time sequence; Get the current time, and calculate the time difference based on the current time and the transaction time, where the time difference is the absolute difference between the current time and the transaction time; The time weight is calculated based on the time difference and time ranking value. The calculation formula is as follows: Among them, β t is the time weight, rank t is the time ranking value, t0 is the time difference, and e is the natural constant; If the transaction time is not within the retrieval time range, the time weight is recorded as 0.

6. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 5, characterized in that: The calculating of the location weight according to the originating location and the headquarters location in the multi-dimensional data includes: The transaction distance is obtained using the initiation location and the headquarters location, and the location weight is calculated using the transaction distance. The calculation formula is as follows: Among them, γ p is the position weight, P x is the transaction distance, and P0 is the preset standard distance.

7. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 6, characterized in that: The calculation formula of the log priority is as follows: x log =(a id +b t +g p )×ln(1+sum x ) Among them, χ log is the log priority, sum x is the transaction amount, and ln is the natural logarithm.

8. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 7, characterized in that: The classified compression operation is performed on the backlog log set to obtain a log compression package, including: Performing a merging operation on multiple backlog log files in the backlog log set to obtain multiple log folders, wherein each of the multiple log folders includes: one or more backlog log files; Using multiple log folders to obtain multiple file storages, where the file storages correspond to the log folders one by one; Determine the maximum and minimum reserves based on multiple file reserves, where the maximum reserve is the file reserve with the largest storage capacity among the multiple file reserves, and the minimum reserve is the file reserve with the smallest storage capacity among the multiple file reserves; Perform the following operations on each of the multiple log folders: The compression level is calculated using the maximum storage, minimum storage, and the file storage corresponding to the log folder. The calculation formula is as follows: Among them, L e is the compression level, C max is the maximum reserve, C min is the minimum reserve, C L is the file storage corresponding to the log folder, Z e is the preset maximum compression level, Represents a round-up operation; Perform compression operation on the log folder based on the compression level to obtain a compressed file; Aggregate the compressed files to get the log compression package.

9. The method for constructing a log storage system based on multi-dimensional logical relationship interaction according to claim 8, characterized in that: The method of constructing a log graph database using a common log set includes: Perform the following operations for each common log file in the common log set: The initiating user ID and the initiating position corresponding to the common log files are combined into an initiating information group, and the receiving user ID and the receiving position corresponding to the common log files are combined into a receiving information group; Identify both the initiating information group and the receiving information group as user information groups; Aggregating the user information groups to obtain multiple user information groups; Performing a unique operation on the multiple user information groups to obtain multiple unique user information, and using the multiple unique user information to create multiple unique user nodes in the pre-built graph database building software, wherein the unique user nodes correspond to the unique user information one by one; Perform the following operations for each common log file in the common log set: The initiating user ID corresponding to the common log file is used as the index initiating ID, and the index initiating ID is used to retrieve the initiating node from multiple unique user nodes, wherein the initiating user ID or the receiving user ID in the unique user information corresponding to the initiating node is the same as the index initiating ID; Get the receiving node based on the receiving user ID corresponding to the common log file; Create relationship edges in the graph database building software based on the transaction time, transaction amount, initiating node, and receiving node corresponding to the common log files; The relationship edges are aggregated to obtain multiple relationship edges, and a log graph database is confirmed based on multiple unique user nodes and multiple relationship edges.

10. A log storage system construction system based on multi-dimensional logical relationship interaction, characterized in that: The system comprises: The multidimensional data parsing module is used to receive the log storage instruction, collect the original log set based on the log storage instruction, perform parsing operations on the original log set, and obtain a multidimensional data set, wherein the original log set includes multiple original log files, the multidimensional data set includes multiple multidimensional data, and the original log files and the multidimensional data correspond one to one, wherein the multidimensional data includes: initiating user ID, initiating location, receiving user ID, receiving location, transaction time and transaction amount value; A search record analysis module is used to obtain a search record set, wherein the search record set includes: a user search records and b time search records, and a user search record and b time search records are used to obtain a priority user sequence and a priority time sequence respectively; The priority log classification module is used to perform the following operations on each multidimensional data in the multidimensional data set: calculate the user weight according to the multidimensional data and the priority user sequence, calculate the time weight according to the transaction time and the priority time sequence in the multidimensional data, obtain the headquarters location, calculate the location weight according to the initiation location and the headquarters location in the multidimensional data, calculate the log priority according to the user weight, time weight, location weight and transaction amount value, compare the log priority with the preset priority threshold, if the log priority is greater than or equal to the priority threshold, then mark the original log file corresponding to the log priority as a common log file, if the log priority is less than the priority threshold, then mark the original log file corresponding to the log priority as a backlog log file; The log database construction module is used to summarize the commonly used log files and backlog log files respectively, obtain the commonly used log sets and the backlog log sets, perform classification and compression operations on the backlog log sets to obtain log compression packages, use the commonly used log sets to build a log graph database, import the log compression packages into the pre-built basic servers to obtain a compressed database, import the log graph database into the pre-built high-speed servers to obtain a high-speed database, and complete the construction of the log storage system.

Images