Distributed digital ID generation method and device, electronic equipment and storage medium

Abstract

The application provides a distributed digital ID generation method and device, electronic equipment and a storage medium. The method comprises: obtaining an encryption string corresponding to each of at least one target data to be generated and an encryption type of the encryption string; determining a character ID of each target data based on the encryption string and the encryption type of the target data; the character ID is used to identify the target data; determining a group number of the character ID based on a character at a preset position in the character ID; the target group represented by the group number comprises at least one target data corresponding to the same character at the preset position; determining a base of the character ID based on a base format of the character at the preset position in the character ID; determining a serial number of the character ID in the target group based on the character ID of the target data; and determining a digital ID of the target data based on the group number, the base and the serial number. The digital ID generated based on the application has idempotency.

Description

Technical Field

[0001] This invention relates to the field of information processing technology, and in particular to a distributed digital ID generation method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the application and development of distributed systems, it is often necessary to generate identity documents (IDs) for each piece of data in the system to identify them. Numeric IDs are one type of ID, and can be, for example, IDs of integer data type. In some distributed system scenarios, it is necessary to generate numeric IDs that are highly unique and idempotent. For example, in scenarios based on large-scale graph computation, it is necessary to generate unique and idempotent numeric IDs for the vertex data in the graph.

[0003] In existing technologies, the Snowflake algorithm can generate highly unique numeric IDs for distributed systems. However, the numeric IDs generated by the Snowflake algorithm are not idempotent and cannot be used in distributed systems that require idempotency. Therefore, how to generate idempotent numeric IDs for distributed systems has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a distributed digital ID generation method, apparatus, electronic device, and storage medium to overcome the shortcomings of existing technologies that cannot generate idempotent digital IDs, thereby achieving the goal of generating idempotent digital IDs.

[0005] This invention provides a distributed digital ID generation method, comprising:

[0006] Obtain the encrypted string corresponding to at least one target data for which a numeric ID is to be generated, and the encryption type of the encrypted string;

[0007] For each of the target data, a character ID is determined based on the encrypted string and the encryption type of the target data, and the character ID is used to identify the target data;

[0008] Based on the character at a preset position in the character ID, the group number of the character ID is determined; the target group represented by the group number includes at least one target data corresponding to the character at the preset position;

[0009] Based on the radix format of the character at the preset position in the character ID, the radix of the character ID is determined, and the radix is ​​used to represent the maximum number of groups of all target data;

[0010] Based on the character ID of the target data, determine the sequence number of the character ID within the target group;

[0011] The digital ID of the target data is determined based on the group number, the cardinality, and the sequence number.

[0012] According to a distributed digital ID generation method provided by the present invention, the character at the preset position is the last two characters of the character ID of the target data.

[0013] According to a distributed digital ID generation method provided by the present invention, determining the radix of the character ID based on the radix format of the character at the preset position in the character ID includes:

[0014] Based on the number system format, determine the maximum character corresponding to the character at the preset position, where the maximum character is the maximum value corresponding to the number system format;

[0015] Perform a decimal conversion on the largest character to obtain the corresponding largest decimal number;

[0016] The sum obtained by adding 1 to the largest decimal number is determined as the base of the character ID.

[0017] According to a distributed digital ID generation method provided by the present invention, determining the digital ID of the target data based on the group number, the cardinality, and the sequence number includes:

[0018] The product is obtained by multiplying the cardinality and the ordinal number.

[0019] The sum obtained by adding the group number to the product is determined as the digital ID of the target data.

[0020] According to a distributed digital ID generation method provided by the present invention, determining the character ID of the target data based on the encrypted string of the target data and the encryption type includes:

[0021] The encrypted string and the encryption type are concatenated to obtain the source data of the target data;

[0022] The source data based on the target data is mapped using the SHA-256 hash function to obtain the hash value corresponding to the source data;

[0023] The hash value is determined as the character ID of the target data.

[0024] According to a distributed digital ID generation method provided by the present invention, obtaining at least one target data for generating a digital ID includes:

[0025] Based on at least one graph in graph computing, determine at least one vertex data in each of the graphs;

[0026] Based on the vertex data, at least one target data for generating a digital ID is obtained.

[0027] According to a distributed digital ID generation method provided by the present invention, the step of obtaining at least one target data for generating a digital ID based on the vertex data includes:

[0028] For each of the aforementioned graphs, based on the log stream information corresponding to the graph, target vertex data for generating numeric IDs is determined from the vertex data of each vertex in the graph; the target vertex data includes vertex data that are first added to the graph;

[0029] The target vertex data is determined as the target data.

[0030] The present invention also provides a distributed digital ID generation device, comprising:

[0031] The acquisition module is used to acquire the encrypted string corresponding to at least one target data to be generated as a digital ID and the encryption type of the encrypted string;

[0032] The determining module is used to determine a character ID for each target data based on the encrypted string and the encryption type of the target data, wherein the character ID is used to identify the target data;

[0033] The determining module is further configured to determine the group number of the character ID based on the character at a preset position in the character ID; the group number represents a target group that includes at least one target data corresponding to the character at the preset position;

[0034] The determining module is further configured to determine the radix of the character ID based on the radix format of the character at the preset position in the character ID, wherein the radix is ​​used to characterize the maximum number of groups of all target data;

[0035] The determining module is further configured to determine the sequence number of the character ID in the target group based on the character ID of the target data;

[0036] The determining module is further configured to determine the digital ID of the target data based on the group number, the base number, and the sequence number.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the distributed digital ID generation method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the distributed digital ID generation method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the distributed digital ID generation method as described above.

[0040] This invention provides a distributed digital ID generation method, apparatus, electronic device, and storage medium. The method involves obtaining the encrypted string and encryption type corresponding to at least one target data for which a digital ID is to be generated; for each target data, a character ID is determined based on the encrypted string and encryption type, whereby the character ID identifies the target data. Since the encrypted string and encryption type of the target data have high uniqueness, a unique character ID can be determined based on these elements. Further, a group number is determined based on a character at a preset position in the character ID; the group number indicates that the target group includes at least one target data corresponding to the character at the preset position; the radix of the character ID is determined based on the radix of the character at the preset position, whereby the radix represents the maximum number of groups for all target data; the sequence number of the character ID within the target group is determined based on the character ID of the target data; and the digital ID of the target data is determined based on the group number, radix, and sequence number. In this way, based on the characters in the unique character ID, the group number, cardinality, and sequence number corresponding to the character ID can be obtained. Based on the group number, cardinality, and sequence number corresponding to the character ID, a numeric ID uniquely corresponding to the character ID can be obtained. Since the encryption string and encryption type of the target data are fixed, and the algorithm for obtaining the character ID and numeric ID remains unchanged, when the numeric ID of the target data needs to be calculated repeatedly in the future, the same numeric ID as when it was first generated can be obtained. Therefore, the numeric ID is idempotent. Thus, based on this method, idempotent numeric IDs can be generated. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the distributed digital ID generation method provided in an embodiment of the present invention;

[0043] Figure 2 This is a flowchart of graph computation preprocessing provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of the distributed digital ID generation device provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention. It should be noted that the serial numbers assigned to the objects described in this invention, such as "first," "second," etc., are only used to distinguish the described objects and have no sequential or technical meaning.

[0047] In distributed systems, if multiple nodes generate numeric IDs simultaneously, some of these IDs may be identical, leading to data conflicts. Traditional auto-incrementing sequences cannot guarantee the uniqueness of numeric IDs in distributed applications. The Snowflake algorithm, however, combines timestamps, machine IDs, and sequence numbers to generate highly unique numeric IDs.

[0048] Snowflake algorithm is an open-source distributed digital ID generation algorithm from Twitter, designed to solve the problem of generating unique digital IDs in distributed environments. Digital IDs generated using the Snowflake algorithm are 64 bits long. The timestamp occupies 41 bits, representing a time range of up to 69 years; the machine ID occupies 10 bits, supporting 1024 different machines, serving as the device's identifier within the system; and the serial number occupies 12 bits, generating 4096 different serial numbers within the same millisecond. For example, in big data scenarios, HiveUDF can be used to call Snowflake algorithm functions for efficient generation of unique digital IDs in distributed environments.

[0049] While the Snowflake algorithm is a simple and efficient algorithm for generating numeric IDs, it suffers from a lack of idempotency. For example, when an event is rolled back in a distributed system, the current system time is used to recalculate the numeric ID, potentially resulting in a different ID than the initially generated one. This means the initially generated ID is not idempotent. Furthermore, when a single point of failure occurs in a distributed system, such as a device disconnecting, errors can occur in the machine ID allocation process. In such cases, if a recalculation of the numeric ID is required, the machine ID of the failed device might be skipped, leading to a different ID than the one generated before the failure. This causes the entire distributed system's ID generation to fail, demonstrating that the existing method also lacks idempotency.

[0050] To address the aforementioned problems, this invention provides a distributed digital ID generation method. This method obtains the encrypted string and encryption type corresponding to at least one target data for which a digital ID is to be generated. For each target data, a character ID is determined based on the encrypted string and encryption type. Because the encrypted string and encryption type of the target data have high uniqueness, a unique character ID can be determined based on these elements. Further, the group number of the character ID is determined based on a character at a preset position within the character ID; the radix of the character ID is determined based on the radix of the character at the preset position; the sequence number of the character ID within the target group is determined based on the character ID of the target data; and the digital ID of the target data is determined based on the group number, radix, and sequence number. In this way, based on the characters in the unique character ID, the group number, cardinality, and sequence number corresponding to the character ID can be obtained. Based on the group number, cardinality, and sequence number corresponding to the character ID, a numeric ID uniquely corresponding to the character ID can be obtained. Since the encryption string and encryption type of the target data are fixed, and the algorithm for obtaining the character ID and numeric ID remains unchanged, when the numeric ID of the target data needs to be calculated repeatedly in the future, the same numeric ID as when it was first generated can be obtained. Therefore, the numeric ID is idempotent. Thus, based on this method, idempotent numeric IDs can be generated.

[0051] The distributed digital ID generation method provided in this invention is applicable to various types of distributed systems that require the generation of digital IDs. For example, it can be applied to distributed systems that need to generate idempotent digital IDs from massive amounts of data. The distributed system can be, for example, a system based on large data graph computation.

[0052] Graph computing models data in a graph-like manner to achieve results that are difficult to obtain from a flat perspective. Large-scale graph computing requires vertex data input, which has several requirements, such as integer data type, data length within 19 bits, unique data values, and the requirement that the complementation step in graph computing be performed only once when performing complementation at historical breakpoints, meaning that the generated numeric IDs must be idempotent. Typical large-scale scenarios place certain resource demands on numeric ID generation algorithms. The following embodiments of this invention illustrate the preprocessing and generation of numeric IDs in a large-scale graph computing scenario. Numeric ID generation methods in other application scenarios are similar and will not be elaborated upon here.

[0053] The execution subject of this method can be an electronic device such as a computer, server, server cluster, or specially designed distributed digital ID generation device, or it can be a distributed digital ID generation device installed in such an electronic device. This distributed digital ID generation device can be implemented through software, hardware, or a combination of both. The following section will combine... Figure 1 and Figure 2 The distributed digital ID generation method provided in the embodiments of the present invention will be described. Figure 1 This is a flowchart illustrating the distributed digital ID generation method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the distributed digital ID generation method includes steps 110 to 160.

[0054] Step 110: Obtain the encrypted string and encryption type corresponding to at least one target data for which a digital ID is to be generated.

[0055] In this step, the target data for which numeric IDs are to be generated is the data for which numeric IDs need to be generated. For example, in a graph computation scenario, a corresponding numeric ID needs to be generated for each vertex data in each graph before graph computation. A numeric ID can be understood as an ID of numeric type, such as an integer numeric ID of type Java long.

[0056] For example, when generating target data upstream of the target data, the target data itself already possesses a corresponding encrypted string and an encryption type corresponding to that encrypted string. The encrypted string can be a string generated from the target data itself using an encryption algorithm; it can also be called a device ID, for example, a 32-bit MD5 hash. When generating the target data, based on the generating device parameters, the generated target data has an encryption type corresponding to the encrypted string. This encryption type can also be called a device type, for example, an IMEI or IDFA type. Obtaining the encrypted string and encryption type of the target data for which the numeric ID is to be generated can be done by reading the target data.

[0057] Step 120: For each target data, determine the character ID of the target data based on the encrypted string and encryption type of the target data. The character ID is used to identify the target data.

[0058] In this step, after obtaining the encrypted string and encryption type of the target data, the encrypted string and encryption type can be used as input parameters to generate the character ID of the target data. Based on these input parameters, the character ID used to identify the target data is determined.

[0059] For example, a hash function or other mapping algorithm can be used to map the encrypted string and the encryption type of the encrypted string to obtain a character ID that is idempotent and highly unique.

[0060] For example, the device ID and device type of the target data are used as input parameters to generate a character ID. The unique key formed by the device ID and device type is mapped using a hash function such as MD5, SHA-1, or SHA-256 to obtain the character ID of the target data. This character ID can be a 64-bit hexadecimal string, where each hexadecimal digit can be one of 16 numbers: 0-9 and af. The input parameters used in this invention are the device ID and device type of the target data. Unlike the input parameters of existing algorithms, the device ID and device type are inherent data information of the target data. The character ID generated in this step does not depend on the timestamp, machine ID, or sequence number of the distributed system. Therefore, even in the event of system event callback or single point of failure, when the character ID of the target data is generated repeatedly, each generated character ID is the same as the initially generated character ID. Therefore, the generated character ID is idempotent.

[0061] Step 130: Determine the group number of the character ID based on the character at the preset position in the character ID; the group number represents the target group that includes at least one target data corresponding to the character at the preset position.

[0062] In this step, target grouping can be understood as dividing the target data to be generated into groups according to the characters in the preset positions of each character ID. Each character ID has a corresponding group, and each group includes at least one character ID corresponding to the target data to be generated. The group number is the group number of the character ID of each target data.

[0063] For example, when grouping character IDs, character IDs that share the same character at a preset position can be grouped into the same group. Here, the character at the preset position can be understood as a character in a preset position within the character ID; for example, the preset position could be the first character, the last character, or the last two characters of the character ID.

[0064] Taking the character at a preset position as the last character as an example, when grouping the 64-bit hexadecimal characters ID1 "...30f", ID2 "...571", ID3 "...91f", ID4 "...7a1", and ID5 "...5b0", characters ID1 and ID3 will be grouped together, with their group number being the decimal number corresponding to the last character "f". Therefore, the group number of their target group is 15. Character ID2 and ID4 will be grouped together, with their group number being the decimal number corresponding to the last character "1". Therefore, the group number of their target group is 1. Character ID5 will be grouped together, with its group number being the decimal number corresponding to the last character "0". Therefore, the group number of its target group is 0. The ellipses in characters ID1 to ID5 indicate characters omitted from the character ID. It should be noted that the group number can be understood as the number of each group, which is a decimal number starting from 0. Therefore, when determining the group number of a character ID, the character at the preset position of the character ID needs to be converted to decimal according to the base format of the character ID to obtain its corresponding decimal group number.

[0065] Step 140: Determine the radix of the character ID based on the radix format of the character at a preset position in the character ID. The radix is ​​used to represent the maximum number of groups of all target data.

[0066] In this step, the cardinality of the character ID is determined based on the characters in the same preset positions when determining the group number of the character ID. This cardinality represents the maximum number of groups for all target data.

[0067] Taking characters ID1 to ID5 as examples, since their default position is the last character, when grouping based on a single hexadecimal digit, a maximum of 16 groups can be formed. Therefore, the base number of the determined character IDs is 16. Specifically, character ID5 "...5b0" has a group number of 0, which is the first group; character ID2 "...571" has a group number of 1, which is the second group; and so on. Character ID1 "...30f" has a group number of 15, which is the sixteenth group. It can be seen that the maximum number of groups based on a single hexadecimal digit is 16.

[0068] Step 150: Determine the sequence number of the character ID within the target group based on the character ID of the target data.

[0069] In this step, after each character ID is assigned to its target group, the character IDs within each target group are sorted to obtain their sequence number within that group. This sequence number is a decimal number starting from 1. The sorting can be in ascending order (from smallest to largest character ID value) or descending order (from largest to smallest character ID value). For example, assuming ascending order, if character ID2 and character ID4 are in target group number 1, and character ID2 has the smallest value in that group, then its sequence number within that group is 1. The sequence numbers for the other character IDs can also be determined accordingly.

[0070] Step 160: Determine the numeric ID of the target data based on the group number, cardinality, and sequence number.

[0071] In this step, based on the group number, cardinality, and sequence number, the digital ID of the target data can be determined through unique mapping methods or algorithms such as digital encoding.

[0072] For example, group numbers, radix, and sequence numbers can be used for numerical encoding to obtain highly unique numeric IDs corresponding to character IDs. During encoding, the characteristics of number systems can be utilized to set encoding rules, which can reduce the probability of collisions between the resulting numeric IDs, thereby improving their uniqueness.

[0073] For example, a mapping configuration table can be set up, which can include the mapping relationship between each character ID and its corresponding numeric ID. Based on the configured mapping configuration table, the target numeric ID corresponding to the character ID can be determined by querying, and this target numeric ID is the numeric ID of the target data.

[0074] The distributed digital ID generation method provided in this invention involves obtaining the encrypted string and encryption type corresponding to at least one target data for which a digital ID is to be generated; for each target data, determining a character ID based on the encrypted string and encryption type, whereby the character ID identifies the target data. Since the encrypted string and encryption type of the target data have high uniqueness, a unique character ID can be determined based on these elements. Further, the group number of the character ID is determined based on a character at a preset position within the character ID; the group number indicates that the target group includes at least one target data corresponding to a character with the same character at the preset position; the radix of the character ID is determined based on the radix of the character at the preset position within the character ID, whereby the radix represents the maximum number of groups for all target data; the sequence number of the character ID within the target group is determined based on the character ID of the target data; and the digital ID of the target data is determined based on the group number, radix, and sequence number. In this way, based on the characters in the unique character ID, the group number, cardinality, and sequence number corresponding to the character ID can be obtained. Based on the group number, cardinality, and sequence number corresponding to the character ID, a numeric ID uniquely corresponding to the character ID can be obtained. Since the encryption string and encryption type of the target data are fixed, and the algorithm for obtaining the character ID and numeric ID remains unchanged, when the numeric ID of the target data needs to be calculated repeatedly in the future, the same numeric ID as when it was first generated can be obtained. Therefore, the numeric ID is idempotent. Thus, based on this method, idempotent numeric IDs can be generated.

[0075] In practical applications, the amount of data in distributed systems is usually large. When dealing with massive amounts of data, uneven grouping can lead to low efficiency in the application of computing resources. This can be understood as follows: in uneven grouping, some groups contain a small amount of target data, resulting in less computational pressure on the devices responsible for generating numeric IDs, while other groups contain a large amount of target data, leading to greater computational pressure on the devices responsible for generating numeric IDs. Therefore, it is necessary to improve the uniformity of grouping to reasonably enhance the overall efficiency of numeric ID generation in distributed systems.

[0076] In one embodiment, the character at the preset position is the last two characters of the character ID in the target data.

[0077] Specifically, the maximum number of groups for all character IDs can be determined based on the character at a preset position in the character ID. For example, for a 64-bit hexadecimal character ID, if grouping is based on the last two characters, the maximum number of groups is 256, i.e., groups 0 to 255.

[0078] For example, the 64-bit hexadecimal character ID6 "...a00" and character ID7 "...5ff", the last two characters of character ID6 are "00", which is 0 in decimal, so its group number is 0; the last two characters of character ID7 are "ff", which is 255 in decimal, so its group number is 255.

[0079] In this embodiment, when the last two characters of the target data's character ID are used as the characters at preset positions to determine the group number, a more reasonable number of groups can be divided. The amount of target data in each group is relatively even. When generating numeric IDs for each target data, the uneven computing pressure of each computing unit can be avoided. Even grouping and sorting can enable the computing engine to generate multiple computing units for sorting when processing massive amounts of data, thereby achieving parallel processing, improving computing efficiency and reducing server pressure, and thus improving the overall efficiency of the distributed system during operation.

[0080] The following section details the specific methods for determining the cardinality of a character ID.

[0081] In one embodiment, the radix of a character ID is determined based on the radix format of the character at a preset position in the character ID, which can be done in the following way:

[0082] Based on the number system format, determine the maximum character corresponding to the character at the preset position. The maximum character is the maximum value corresponding to the number system format. Convert the maximum character to decimal to obtain the corresponding maximum decimal number. Add 1 to the maximum decimal number to obtain the sum, which is used as the base of the character ID.

[0083] Specifically, based on the number system format, the maximum character corresponding to the character at a preset position can be determined. The maximum character is the maximum value corresponding to the specified number system format, where the number system format is the same as the number system format of the character ID, such as hexadecimal. When the number system format is fixed, the character at the preset position will have a corresponding maximum character. For example, in decimal, the minimum value of each character is "0" and the maximum value is "9". When the character at the preset position is the last character, the corresponding maximum character is "9". Similarly, in hexadecimal, the minimum value of each character is "0" and the maximum value is "f". When the character at the preset position is the last character, the corresponding maximum character is "f". In decimal, if the character at the preset position is the last two characters in the character ID of the target data, the corresponding maximum value is "99"; in hexadecimal, if the character at the preset position is the last two characters in the character ID of the target data, the corresponding maximum value is "ff".

[0084] For example, if the character at the preset position is the last two characters of the target data's character ID, in a 64-bit hexadecimal character ID, after determining the largest character position "ff" corresponding to the last two characters, it is converted to decimal, i.e., converted to 255. 255 is the corresponding largest decimal number. The sum of the largest decimal number 255 and 1 is 256, which is determined as the base of the character ID. This base is 256, and 256 is also the maximum number of groups that can be formed when the character at the preset position is the last two characters of the target data's character ID. This step can be understood as the group number starting from 0 and going up to the corresponding largest decimal number, while the base is the corresponding largest number of groups, which is the result of adding one to the largest decimal number.

[0085] In this embodiment, when determining the radix of a character ID based on the radix of the character at a preset position in the character ID, the largest character corresponding to the character at the preset position can be determined based on the radix. The largest character is the maximum value corresponding to the character in the radix. The largest character is then converted to decimal to obtain the corresponding maximum decimal number. The sum of the maximum decimal number and 1 is determined as the radix of the character ID. Based on this, the radix of the character ID can be accurately determined, providing accurate basic parameters for the subsequent generation of numeric IDs.

[0086] The following section details how to determine the numerical ID of target data based on the group number, cardinality, and sequence number.

[0087] In one embodiment, the numeric ID of the target data can be determined based on the group number, cardinality, and sequence number in the following manner:

[0088] The product is obtained by multiplying the cardinality and the ordinal number; the sum of the product and the group number is determined as the numerical ID of the target data.

[0089] Specifically, based on the character at a preset position in the character ID, the group number and cardinality of the character ID can be determined. A can represent the group number, and B can represent the cardinality. Sorting within the target group determines the sequence number of the character ID within that group, which can be represented by C. Multiplying the cardinality and the sequence number yields a product B*C. Adding this product to the group number gives a sum A+B*C, which can be used as the numeric ID of the target data. Furthermore, if max(A) represents the largest decimal number corresponding to the largest character, then max(A)+1 is the cardinality B.

[0090] Taking the 64-bit hexadecimal characters ID6 "...a00" and ID7 "...5ff" as examples, the last two characters of ID6 are "00", which is 0 in decimal. Therefore, its group number is 0. The largest decimal number corresponding to the last two characters in hexadecimal is 255. 255 + 1 = 256, so the base is 256. Assuming that the sequence number of ID6 in group 0 is 10, then 0 + 256 * 10 = 2560. We can determine 2560 as the numeric ID corresponding to ID6. The last two characters of character ID7 are "ff", which is 255 in decimal. Therefore, its group number is 255. The largest decimal number corresponding to the last two characters in hexadecimal is 255. 255 + 1 = 256, so the base is 256. Assuming that the sequence number of character ID7 in the 255th group is 11, then 255 + 256 * 11 = 3071. We can determine 3071 as the numeric ID corresponding to character ID7.

[0091] This method of determining numeric IDs guarantees a high degree of uniqueness. For example, the 64-bit hexadecimal character ID8 is "...5fe". The last two characters of character ID8, "fe", are converted to decimal 254, so its group number is 254. The largest decimal number corresponding to the last two characters in hexadecimal is 255. 255 + 1 = 256, so the base is 256. Assuming that the sequence number of character ID8 in group 254 is 11, then 254 + 256 * 11 = 3070. 3070 can be determined as the numeric ID corresponding to character ID8. Although character ID8 and character ID7 have the same base and sequence number, because their group numbers are different, the numeric IDs generated by these two methods will not overlap, i.e., there will be no collision. Within the same group, although the group numbers corresponding to each character ID are the same (i.e., the value of A is the same) and the radix B is also the same, the numeric IDs corresponding to each character ID in the group are different because their sequence numbers within the group are different, meaning there will be no collision. Since the maximum value of group number A is always 1 less than the radix B, no matter how A and / or C change, the numeric IDs generated by any two character IDs will not be the same.

[0092] As can be seen, the concept behind this method is to multiply the higher base number by the higher base number and add the lower base step size, using the final calculated result as the corresponding numeric ID. Based on this, the calculated result, i.e., the numeric ID, is always unique globally, ensuring that there are no collisions between numeric IDs, thus giving the numeric ID a high degree of uniqueness.

[0093] In this embodiment, when determining the digital ID of the target data based on the group number, radix, and sequence number, the radix and sequence number can be multiplied to obtain a product. The sum obtained by adding the group number to the product is then determined as the digital ID of the target data. This method implements the calculation logic of high-radix digit * high-radix number + low-radix step size. This calculation logic ensures that the generated digital IDs do not collide. Therefore, while ensuring the idempotency of the generated digital IDs, the uniqueness of the generated digital IDs can be improved.

[0094] For example, in order to further ensure the uniqueness and idempotency of the generated character ID, the SHA-256 hash function, which has advantages such as high security, fixed-length output, high collision resistance, fast calculation and irreversibility, can be used to generate the character ID, so that the generated character ID has uniqueness and idempotency.

[0095] In one embodiment, the character ID of the target data is determined based on the encrypted string and encryption type of the target data. This can be achieved in the following way:

[0096] The encrypted string and encryption type are concatenated to obtain the source data of the target data; the source data of the target data is mapped using the SHA-256 hash function to obtain the hash value corresponding to the source data; the hash value is determined as the character ID of the target data.

[0097] Specifically, the encrypted string and the encryption type are concatenated, that is, the encrypted string and the string representing the encryption type of the encrypted string are concatenated to obtain the source data of the target data.

[0098] Based on the source data of the target data, the SHA-256 hash function can be used to map the source data to obtain the hash value corresponding to the source data. This hash value can then be used to determine the character ID of the target data.

[0099] In this embodiment, when determining the character ID of the target data based on the encrypted string and encryption type, the encrypted string and encryption type are concatenated to obtain the source data of the target data. The source data is then mapped using the SHA-256 hash function to obtain the corresponding hash value. This hash value is then used as the character ID of the target data. Therefore, the SHA-256 hash function can perform a highly unique string mapping on the source data, providing high security and yielding a highly unique and irreversible hash value. This hash value is then used as the character ID of the target data, resulting in a low collision rate and a highly unique and idempotent character ID.

[0100] For example, graph computation is the process of expressing and solving problems using graphs as data models. A graph in graph computation is an abstract data structure used to represent the relationships between objects. When described using vertex data and edges, vertex data represents objects, and edges represent the relationships between objects. Data that can be abstracted as a graph description is graph data. Graph computation requires Java long type data input; therefore, the IDs of each vertex in the graph must be Java long type numeric IDs. During graph computation, massive amounts of vertex data need to be preprocessed to generate numeric IDs corresponding to each vertex, so as to realize graph computation such as intersection and fusion of vertex data from various graphs.

[0101] In one embodiment, obtaining at least one target data for generating a digital ID can specifically involve: determining at least one vertex data in each graph based on at least one graph in graph computation; and obtaining at least one target data for generating a digital ID based on each vertex data.

[0102] Specifically, in a graph-based computing scenario, for each graph in the graph computing, the vertex data for which digital IDs need to be generated is identified as the target data. The digital IDs of the vertex data for which digital IDs need to be generated are generated using the methods described in the above embodiments.

[0103] In one embodiment, based on the vertex data, at least one target data for generating a numeric ID is obtained, which can be achieved in the following way:

[0104] For each graph, based on the log stream information corresponding to the graph, the target vertex data to be generated with a numeric ID is determined from the vertex data of each vertex in the graph; the target vertex data includes the vertex data that is added to the graph for the first time; the target vertex data is determined as the target data.

[0105] Specifically, the log stream information corresponding to the graph can include all log data related to graph computation. For example, the log stream information can be data information from a log table that records data related to graph computation.

[0106] Based on the log stream information corresponding to the graph, the target vertex data for which a numeric ID needs to be generated can be determined from the vertex data of each vertex in the graph. This target vertex data is the vertex data for which a numeric ID needs to be generated. The target vertex data includes the vertex data that is added to the graph for the first time. The vertex data that is added to the graph for the first time can be understood as the vertex data that has not appeared in the graph before. By traversing the vertex data of each vertex in the graph based on the log table, the vertex data that is added to the graph for the first time can be determined. After the target vertex data is determined, it is designated as the target data. The numeric ID of the target vertex data in each graph can be obtained by generating a numeric ID for the target data using the methods in the above embodiments.

[0107] Because big data graph computation scenarios involve data volumes in the tens of billions, performing full batch computations on all vertex data from previous days and the current day is time-consuming. Therefore, the computation process is divided into initialization and daily incremental computation. Initialization can be understood as generating numeric IDs for all current vertex data in the graph when generating numeric IDs for the first time. After that, daily incremental computation means generating corresponding numeric IDs for newly added vertex data each day. Vertex data that has already generated numeric IDs does not need to generate numeric IDs again. It should be understood that "not needing to generate numeric IDs again" here means that numeric IDs do not need to be generated just for the purpose of generating numeric IDs; this does not include situations where the numeric ID is repeatedly calculated due to application requirements.

[0108] Figure 2 This is a flowchart of graph computation preprocessing provided in an embodiment of the present invention, such as... Figure 2 As shown, after the preprocessing for graph computation begins, the character IDs corresponding to the vertex data are obtained using the SHA-256 algorithm. The system then determines whether this character ID corresponds to a vertex being added to the graph for the first time, i.e., whether it is the target vertex data. If not, it means the vertex data is not the target vertex data, and a numeric ID has already been generated for it. If it is, it means the vertex data is the target vertex data, and its corresponding group number, cardinality, and ordinal number should be determined based on the character ID. A numeric ID corresponding to this character ID is then generated based on the corresponding group number, cardinality, and ordinal number. After this, the previously generated numeric IDs and the currently generated numeric IDs can be applied to graph computation, and the numeric IDs can be mapped back to character IDs after the graph computation, so that the vertex data can be represented by a character ID.

[0109] For example, the results of graph computation are converted from numeric IDs to character IDs through mapping relationships and returned as a result table. Daily incremental computation results can be merged with yesterday's full computation results to generate a full snapshot table. Based on this daily mapping method, character IDs are stored before preprocessing and after result return, while numeric IDs are used during graph computation. Numeric IDs are unique in each computation, while character IDs are globally unique. Furthermore, in complement scenarios, numeric IDs are idempotent, ensuring that complement is only needed once, eliminating the need for rolling complements. This numeric ID supports breakpoint complements.

[0110] For example, in the case of data completion, if the data from ten days ago has changed and needs to be completed, since the generated numeric IDs are idempotent, the preprocessing stage of graph computing only needs to complete the data completion for the corresponding partition from ten days ago, without needing to complete the data completion for all data within the past ten days. Therefore, it can reduce the frequent data completion in the early stages of graph computing projects, reduce time and resource consumption, and improve work efficiency.

[0111] In this embodiment, when obtaining at least one target data for generating a digital ID based on the vertex data, the target vertex data for generating the digital ID is determined from the vertex data of each graph based on the log stream information corresponding to the graph. The target vertex data includes the vertex data that is first added to the graph. The target vertex data is then determined as the target data. Based on this, digital IDs can be generated based on the target vertex data. That is, based on the design of full computation and incremental computation, repetitive digital ID generation can be avoided, the daily computational resource requirements can be reduced, and the working efficiency of the distributed system can be improved.

[0112] The distributed digital ID generation apparatus provided in the embodiments of the present invention is described below. The distributed digital ID generation apparatus described below can be referred to in correspondence with the distributed digital ID generation method described above.

[0113] Figure 3 This is a schematic diagram of the structure of the distributed digital ID generation device provided in an embodiment of the present invention, with reference to... Figure 3 As shown, the distributed digital ID generation device 300 includes:

[0114] The acquisition module 310 is used to acquire the encryption string and encryption type of each of the at least one target data to be generated as a digital ID;

[0115] The determination module 320 is used to determine the character ID of each target data based on the encrypted string and encryption type of the target data. The character ID is used to identify the target data.

[0116] The determining module 320 is also used to determine the group number of the character ID based on the character at a preset position in the character ID; the group number represents that the target group includes at least one target data corresponding to the character at the preset position;

[0117] The determination module 320 is also used to determine the radix of the character ID based on the radix format of the character at a preset position in the character ID. The radix is ​​used to represent the maximum number of groups of all target data.

[0118] The determination module 320 is also used to determine the sequence number of the character ID in the target group based on the character ID of the target data;

[0119] The determination module 320 is also used to determine the numeric ID of the target data based on the group number, cardinality, and sequence number.

[0120] In one example embodiment, the character at the preset position is the last two characters of the character ID in the target data.

[0121] In one example embodiment, the determining module 320 is specifically used for:

[0122] Based on the number system format, determine the maximum character corresponding to the character at the preset position. The maximum character is the maximum value corresponding to the number system format.

[0123] Convert the largest character to decimal to obtain the corresponding largest decimal number;

[0124] The sum of the largest decimal number plus 1 is used as the base of the character ID.

[0125] In one example embodiment, the determining module 320 is specifically used for:

[0126] The product is obtained by multiplying the cardinality and the ordinal number;

[0127] The sum obtained by adding the group number to the product is determined as the numeric ID of the target data.

[0128] In one example embodiment, the determining module 320 is specifically used for:

[0129] The source data of the target data is obtained by concatenating the encrypted string and the encryption type.

[0130] The source data based on the target data is mapped using the SHA-256 hash function to obtain the hash value corresponding to the source data.

[0131] The hash value is used to determine the character ID of the target data.

[0132] In one example embodiment, the acquisition module 310 is specifically used for:

[0133] Based on at least one graph in graph computing, determine at least one vertex data in each graph;

[0134] Based on the data of each vertex, obtain at least one target data for the numeric ID to be generated.

[0135] In one example embodiment, the acquisition module 310 is specifically used for:

[0136] For each graph, based on the log stream information corresponding to the graph, the target vertex data to be generated with a numeric ID is determined from the vertex data of each vertex in the graph; the target vertex data includes the vertex data that is added to the graph for the first time;

[0137] The target vertex data is determined as the target data.

[0138] The apparatus of this embodiment can be used to execute the method of any embodiment in the distributed digital ID generation method side embodiment. Its specific implementation process and technical effects are similar to those in the distributed digital ID generation method side embodiment. For details, please refer to the detailed description in the distributed digital ID generation method side embodiment, which will not be repeated here.

[0139] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a distributed digital ID generation method, which includes: obtaining the encryption string and encryption type corresponding to at least one target data for which a digital ID is to be generated; for each target data, determining a character ID for the target data based on the encryption string and encryption type, the character ID being used to identify the target data; determining the group number of the character ID based on the character at a preset position in the character ID; the group number representing the target group containing at least one target data corresponding to the character at the preset position; determining the radix of the character ID based on the radix of the character at the preset position in the character ID, the radix being used to represent the maximum number of groups for all target data; determining the sequence number of the character ID within the target group based on the character ID of the target data; and determining the digital ID of the target data based on the group number, the radix, and the sequence number.

[0140] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] On the other hand, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the distributed digital ID generation method provided by the above methods. The method includes: obtaining an encrypted string and an encryption type corresponding to at least one target data for which a digital ID is to be generated; for each target data, determining a character ID of the target data based on the encrypted string and encryption type of the target data, wherein the character ID is used to identify the target data; determining the group number of the character ID based on a character at a preset position in the character ID; wherein the group number represents a target group that includes at least one target data corresponding to a character with the same character as the character at the preset position; determining the radix of the character ID based on the radix of the character ID, wherein the radix is ​​used to represent the maximum number of groups of all target data; determining the sequence number of the character ID within the target group based on the character ID of the target data; and determining the digital ID of the target data based on the group number, the radix, and the sequence number.

[0142] In another aspect, embodiments of the present invention also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the distributed digital ID generation method provided by the above methods. The method includes: obtaining an encrypted string and an encryption type corresponding to at least one target data for which a digital ID is to be generated; for each target data, determining a character ID of the target data based on the encrypted string and encryption type of the target data, wherein the character ID is used to identify the target data; determining the group number of the character ID based on a character at a preset position in the character ID; wherein the group number represents a target group that includes at least one target data corresponding to a character with the same character as the character at the preset position; determining the radix of the character ID based on the radix of the character at the preset position in the character ID, wherein the radix is ​​used to represent the maximum number of groups of all target data; determining the sequence number of the character ID within the target group based on the character ID of the target data; and determining the digital ID of the target data based on the group number, the radix, and the sequence number.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0144] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A distributed digital ID generation method, characterized in that, include: Obtain the encrypted string corresponding to at least one target data for which a digital ID is to be generated, and the encryption type of the encrypted string; For each of the target data, a character ID is determined based on the encrypted string and the encryption type of the target data, and the character ID is used to identify the target data; Based on the character at a preset position in the character ID, determine the group number to which the character ID belongs; The target group represented by the group number includes at least one target data corresponding to the character at the preset position that is the same as the character at the preset position; Based on the radix format of the character at the preset position in the character ID, the radix of the character ID is determined, and the radix is ​​used to represent the maximum number of groups of all target data; The character IDs in each target group are sorted according to their value to determine the sequence number of each character ID within the target group based on the character IDs of the target data. The target data is determined by digital encoding based on the group number, the cardinality, and the sequence number.

2. The distributed digital ID generation method according to claim 1, characterized in that, The character at the preset position is the last two characters in the character ID of the target data.

3. The distributed digital ID generation method according to claim 1, characterized in that, Determining the radix of the character ID based on the radix format of the character at the preset position in the character ID includes: Based on the number system format, determine the maximum character corresponding to the character at the preset position, where the maximum character is the maximum value corresponding to the number system format; Perform a decimal conversion on the largest character to obtain the corresponding largest decimal number; The sum obtained by adding 1 to the largest decimal number is determined as the base of the character ID.

4. The distributed digital ID generation method according to claim 1, characterized in that, Determining the digital ID of the target data based on the group number, the cardinality, and the sequence number includes: The product is obtained by multiplying the cardinality and the ordinal number. The sum obtained by adding the group number to the product is determined as the digital ID of the target data.

5. The distributed digital ID generation method according to any one of claims 1-4, characterized in that, Determining the character ID of the target data based on the encrypted string and the encryption type includes: The encrypted string and the encryption type are concatenated to obtain the source data of the target data; The source data based on the target data is mapped using the SHA-256 hash function to obtain the hash value corresponding to the source data; The hash value is determined as the character ID of the target data.

6. The distributed digital ID generation method according to claim 5, characterized in that, The process of obtaining at least one target data for generating a numeric ID includes: Based on at least one graph in graph computing, determine at least one vertex data in each of the graphs; Based on the vertex data, at least one target data for generating a digital ID is obtained.

7. The distributed digital ID generation method according to claim 6, characterized in that, The step of obtaining at least one target data for generating a digital ID based on the vertex data includes: For each of the aforementioned graphs, based on the log stream information corresponding to the graph, target vertex data for generating numeric IDs is determined from the vertex data of each vertex in the graph; the target vertex data includes vertex data that are first added to the graph; The target vertex data is determined as the target data.

8. A distributed digital ID generation device, characterized in that, include: The acquisition module is used to acquire the encrypted string corresponding to at least one target data to be generated as a digital ID and the encryption type of the encrypted string; The determining module is used to determine a character ID for each target data based on the encrypted string and the encryption type of the target data, wherein the character ID is used to identify the target data; The determining module is further configured to determine the group number of the character ID based on the character at a preset position in the character ID; the group number represents a target group that includes at least one target data corresponding to the character at the preset position; The determining module is further configured to determine the radix of the character ID based on the radix format of the character at the preset position in the character ID, wherein the radix is ​​used to characterize the maximum number of groups of all target data; The determining module is further configured to sort each character ID in each target group according to the size of the character ID value, so as to determine the sequence number of the character ID in the target group based on the character ID of the target data; The determining module is further configured to perform digital encoding based on the group number, the base number, and the sequence number to determine the digital ID of the target data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the distributed digital ID generation method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the distributed digital ID generation method as described in any one of claims 1 to 7.

Images