Unique primary key identifier generation method and device

Abstract

The invention discloses a unique primary key identifier generation method and device, and the method comprises the steps: reading a to-be-processed data file, obtaining a total number value of the file record number of the to-be-processed data file, generating non-repeated initial random numbers with the number corresponding to the total number value according to a preset script, and generating a unique primary key identifier of the to-be-processed data file; and storing the non-repeated initial random numbers with the number corresponding to the total number value into a list container, for each file record in the to-be-processed data file, obtaining the date of the primary key identifier and the batch number of the primary key identifier from the file record, and reading a target random number from the list container according to a preset reading mode, the preset reading mode is used for ensuring the uniqueness of the read target random number, and generating a unique primary key identifier recorded by the file according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to a method and device for generating a unique primary key identifier. Background Art

[0002] When storing in a database system or importing large data files into a database, it is necessary to distinguish the uniqueness of each record in the database table. The column that records the unique attribute is called the primary key. The design of the database primary key generally follows the principles of efficiency, reliability, and ease of use. The primary key generation method must ensure that the generated primary key is unique.

[0003] A big data platform uses an Oracle database and sends storage files to the server every day. The data files need to be processed and imported into the Mysql database. Since the Mysql database requires that the table must have a primary key, while the Oracle database does not require a primary key, the original data file lacks a primary key field and needs to be generated before the processing and import can be successful. Due to the large amount of data every day, the generated primary key must not be repeated and the processing time is very high, resulting in low efficiency in generating unique primary key identifiers.

[0004] Therefore, how to improve the efficiency of generating unique primary key identifiers is a problem that needs to be solved urgently in this application. Summary of the invention

[0005] In view of this, the present application discloses a method and device for generating a unique primary key identifier, aiming to ensure the uniqueness, availability and efficiency of the generated primary key identifier and improve the efficiency of generating a unique primary key identifier.

[0006] In order to achieve the above-mentioned purpose, the disclosed technical solution is as follows:

[0007] The first aspect of the present application discloses a method for generating a unique primary key identifier, the method comprising:

[0008] Read the data file to be processed and obtain the total value of the number of file records in the data file to be processed;

[0009] Generate a number of non-repeating initial random numbers corresponding to the total value according to the preset script;

[0010] The number of non-repeating initial random numbers corresponding to the total number value is stored in a list container;

[0011] For each file record in the data file to be processed, obtaining the date of the primary key identifier and the batch number of the primary key identifier from the file record;

[0012] Reading the target random number from the list container according to a preset reading method; wherein the preset reading method is used to ensure the uniqueness of the read target random number;

[0013] A unique primary key identifier for the file record is generated according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

[0014] Preferably, the reading of the data file to be processed and obtaining the total value of the number of file records of the data file to be processed includes:

[0015] Read all the lines of the data file to be processed;

[0016] Get the line numbers corresponding to all lines of the file to be processed through the preset built-in function;

[0017] The number of rows corresponding to all rows of the to-be-processed file is counted to obtain a total value of the number of file records of the to-be-processed data file.

[0018] Preferably, obtaining the date of the primary key identifier and the batch number of the primary key identifier from the file record includes:

[0019] Define and initialize the variable number of the data file to be processed; wherein the variable number is used to record the batch number of the day of reading;

[0020] The first row of the data file to be processed is determined as the start row, and the last row of the data file to be processed is determined as the end row;

[0021] Determine a reading order according to the start row and the end row;

[0022] Read all rows of the data file to be processed according to the reading order, obtain a reading date, and determine the reading date as the date of the primary key identifier;

[0023] Read the initialized batch number of the data file to be processed to obtain the batch number identified by the primary key.

[0024] Preferably, the step of reading the target random number from the list container in a preset reading manner includes:

[0025] For each file record in the data file to be processed, the file record obtains a random number from the list container, and the random number is determined as a non-repeating target random number; wherein the number of times all the random numbers read in the list container are read is 1.

[0026] Preferably, after generating a unique primary key identifier for a file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number, the method further comprises:

[0027] The unique primary key identifier of each file record is written into the data file to be processed to complete the generation process of the primary key identifier field.

[0028] The second aspect of the present application discloses a device for generating a unique primary key identifier, the device comprising:

[0029] A first reading unit is used to read the data file to be processed and obtain a total value of the number of file records in the data file to be processed;

[0030] A first generating unit, used to generate a number of non-repeating initial random numbers corresponding to the total value according to a preset script;

[0031] A storage unit, used for storing a number of non-repeating initial random numbers corresponding to the total value into a list container;

[0032] An acquisition unit, for acquiring the date of the primary key identifier and the batch number of the primary key identifier from each file record in the data file to be processed;

[0033] A second reading unit, configured to read a target random number from the list container in a preset reading manner; wherein the preset reading manner is configured to ensure the uniqueness of the read target random number;

[0034] The second generating unit is used to generate a unique primary key identifier for a file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

[0035] Preferably, the first reading unit includes:

[0036] A first reading module, used for reading all rows of the data file to be processed;

[0037] The acquisition module is used to obtain the line numbers corresponding to all lines of the file to be processed through a preset built-in function;

[0038] The statistical module is used to count the number of rows corresponding to all rows of the to-be-processed file to obtain the total value of the number of file records of the to-be-processed data file.

[0039] Preferably, the acquisition unit includes:

[0040] A definition module is used to define and initialize the variable numbers of the data file to be processed; wherein the variable numbers are used to record the batch number of the day of reading;

[0041] A first determining module, used to determine the first line of the data file to be processed as the starting line, and determine the last line of the data file to be processed as the ending line;

[0042] A second determining module, used for determining a reading order according to the start row and the end row;

[0043] A third determination module is used to read all rows of the data file to be processed according to the reading order, obtain a reading date, and determine the reading date as a date identified by the primary key;

[0044] The second reading module is used to read the initialized batch number of the data file to be processed to obtain the batch number identified by the primary key.

[0045] Preferably, the second reading unit is specifically used for:

[0046] For each file record in the data file to be processed, the file record obtains a random number from the list container, and the random number is determined as a non-repeating target random number; wherein the number of times all the random numbers read in the list container are read is 1.

[0047] Preferably, it also includes:

[0048] The writing unit is used to write the unique primary key identifier of each file record into the data file to be processed to complete the generation process of the primary key identifier field.

[0049] It can be known from the above technical scheme that the present application discloses a method and device for generating a unique primary key identifier, which reads a data file to be processed and obtains the total value of the number of file records of the data file to be processed, generates a number of non-repetitive initial random numbers corresponding to the total value according to a preset script, stores the number of non-repetitive initial random numbers corresponding to the total value in a list container, obtains the date of the primary key identifier and the batch number of the primary key identifier from the file record for each file record in the data file to be processed, and reads the target random number from the list container according to a preset reading method, wherein the preset reading method is used to ensure the uniqueness of the read target random number, and generates a unique primary key identifier for the file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

[0050] Through the above scheme, random numbers are generated based on the total amount of data, so that primary key conflicts caused by repeated timestamps due to too much data can be avoided, and the uniqueness requirement of the primary key can be met. In addition, a larger list container is used to store the generated random numbers, and then the numbers are taken from the container to shorten the processing time, so that the algorithm efficiency of the primary key identifier generated by this scheme is relatively high while meeting the availability. At the same time, the date and batch number are added to the primary key identifier, which greatly facilitates subsequent business operations such as data query and statistics, and improves business functionality. Therefore, compared with the self-increment primary key algorithm and the traditional timestamp-based primary key identifier generation algorithm, the primary key identifier generation method of this scheme can not only save database performance, but also will not generate repeated random numbers when generating primary key identifiers for large data files in a short time, thereby ensuring the uniqueness, availability and efficiency of the generated primary key identifier, and improving the efficiency of generating unique primary key identifiers. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0052] Figure 1 A schematic diagram of a process for generating a unique primary key identifier disclosed in an embodiment of the present application;

[0053] Figure 2 A schematic diagram of the structure of a unique primary key identifier generating device disclosed in an embodiment of the present application;

[0054] Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0056] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0057] As can be seen from the background technology, in the prior art, a certain big data platform uses an oracle database to send storage files to a server every day, and the data files need to be processed and imported into a Mysql database. Since the Mysql database requires that the table must have a primary key, and the oracle database does not require a primary key, the original data file lacks a primary key field, and the primary key field needs to be generated before the processing and import can be successful. Due to the large amount of data every day, the generated primary key is not repeated, and the processing time is very high, resulting in low efficiency in generating a unique primary key identifier. Therefore, how to improve the efficiency of generating a unique primary key identifier is a problem that this application urgently needs to solve.

[0058] In order to solve the above problems, the present application discloses a method and device for generating a unique primary key identifier, which generates random numbers based on the total amount of data, so as to avoid primary key conflicts caused by repeated timestamps due to too large a data volume, and meet the uniqueness requirements of the primary key. In addition, a larger list container is used to store the generated random numbers, and then the numbers are taken from the container to shorten the processing time, so that the algorithm efficiency of the primary key identifier generated by this scheme is also high while meeting the availability. At the same time, the date and batch number are added to the primary key identifier, which greatly facilitates subsequent business operations such as data query and statistics, and improves business functionality. Therefore, compared with the self-increment primary key algorithm and the traditional timestamp-based primary key identifier generation algorithm, the primary key identifier generation method of this scheme can not only save the performance of the database, but also will not generate repeated random numbers when generating primary key identifiers for large data files in a short time, thereby ensuring the uniqueness, availability and efficiency of the generated primary key identifier, and improving the efficiency of generating unique primary key identifiers. The specific implementation method is specifically described by the following embodiments.

[0059] When the data file does not have a primary key field and a unique identifier needs to be added to each record as the primary key field, a primary key identifier generation algorithm is needed. Since the primary key field of the database is required to be unique and the amount of data is large, the primary key generation algorithm must simultaneously meet the uniqueness and efficiency of the primary key ID. In order to address the problem of generating a primary key identifier for each record of a large data file in a short time, the present application provides a unique primary key identifier generation method based on the total amount of data. In order to solve the problem of generating a unique primary key identifier for a large data file in a short time, the present application no longer generates a random number based on a timestamp, but generates a random number based on the total amount of data, so that the primary key conflict caused by the duplication of the timestamp due to the large amount of data can be avoided, and the uniqueness requirement of the primary key can be met. At the same time, the date and batch number are added to the primary key identifier, which greatly facilitates subsequent business operations such as data query and statistics, and improves business functionality. After experimental testing of data file processing, the algorithm processes 1kw data in less than 30 seconds (s), which ensures the effectiveness and high availability of the algorithm and meets the timeliness requirements of processing large data files in a short time.

[0060] refer to Figure 1 As shown, a method for generating a unique primary key identifier disclosed in an embodiment of the present application is provided. The method for generating a unique primary key identifier mainly includes the following steps:

[0061] S101: Read the data file to be processed, and obtain the total value (Max) of the number of file records of the data file to be processed.

[0062] The specific process of reading the data file to be processed and obtaining the total value of the number of file records of the data file to be processed is shown in A1-A3.

[0063] A1: Reads all rows of the data file to be processed.

[0064] In A1, all lines of the data file to be processed can be read using the readlines() method.

[0065] It should be noted that the method of reading all lines of the data file to be processed includes but is not limited to the readlines() method.

[0066] A2: Obtain the line numbers of all lines in the file to be processed using the preset built-in function.

[0067] The preset built-in functions include but are not limited to the len() function.

[0068] A3: Count the number of rows corresponding to all rows in the file to be processed to obtain the total number of file records in the data file to be processed.

[0069] S102: Generate a number of non-repeating initial random numbers corresponding to the total value according to a preset script.

[0070] The preset scripts include but are not limited to Python scripts.

[0071] In S102, Max non-repeating random numbers may be generated according to the Python script, and the maximum number generated does not exceed 2 to the power of 31 - 1 (approximately 2.1 billion).

[0072] S103: storing the number of non-repeating initial random numbers corresponding to the total value into a list container.

[0073] It should be noted that the List container is a data container in the Python language, such as [1, 2, 3, 4, 5, 6, 7…].

[0074] S104: For each file record in the data file to be processed, obtain the date identified by the primary key and the batch number identified by the primary key from the file record.

[0075] Establish a loop, starting from the first line to the last line of the data file to be processed, read each line of the data file to be processed in turn, obtain the current 8-digit date, and then obtain the batch number num of this data file. The current date and batch number are the first two parts of the primary key identifier.

[0076] The primary key is also called the primary keyword. The primary key is a main field in the MYSQL database table. As the unique identifier of each row of records, it can uniquely identify a row of data in the table.

[0077] MYSQL is a relational database. Each database can have multiple tables, and each table requires a non-repeating field as the primary key.

[0078] The specific process of obtaining the date identified by the primary key and the batch number identified by the primary key from the file record is shown in B1-B5.

[0079] B1: Define and initialize the variable number (num) of the data file to be processed; num is used to record the batch number of the day read.

[0080] Among them, the variable num is defined and initialized to 1. Numb records today's batch number and is reset to zero every day.

[0081] B2: Determine the first row of the data file to be processed as the start row, and determine the last row of the data file to be processed as the end row.

[0082] B3: Determine the reading order based on the start row and end row.

[0083] B4: Read all rows of the data file to be processed according to the reading order, obtain the reading date, and determine the reading date as the date identified by the primary key.

[0084] To facilitate understanding of the process of obtaining the date and batch number of the primary key identifier from the file record, an example is given here:

[0085] For example, starting from the first line to the last line of the data file to be processed, each line of the data file to be processed is read in sequence to obtain the current 8-digit date, such as 20230508, and then obtain the batch number num of this data file, such as 1, which constitutes the first two parts of the primary key identifier.

[0086] B5: Read the batch number of the data file to be processed after initialization, and obtain the batch number identified by the primary key.

[0087] S105: Reading a target random number from the list container according to a preset reading method; wherein the preset reading method is used to ensure the uniqueness of the read target random number.

[0088] In S105, for each file record in the data file to be processed, the file record obtains a random number from the list container, and the random number is determined as a non-repeating target random number; wherein the number of times all the random numbers read in the list container are read is 1.

[0089] For example, each entry takes a random number from the List container in turn, such as 976854264, and each time it takes the next number in the List, ensuring that the number that has been taken will not be taken again, thereby achieving the uniqueness of the random number.

[0090] S106: Generate a unique primary key identifier for the file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

[0091] Each record is read with date + batch number, followed by a unique random number generated in the List container, and the three are combined as the unique primary key identifier of each record. For example, its primary key ID is 202305081976854264, which consists of date 20230508 + batch number 1 + unique random number 976854264.

[0092] The unique primary key identifier of each file record is written into the data file to be processed to complete the generation process of the primary key identifier field.

[0093] This application adopts the idea of ​​trading space for time, and generates a random number based on the total amount of data plus the date and batch to form the primary key field. This not only avoids duplication of primary key identifiers, but also speeds up the efficiency of primary key generation, meeting the uniqueness and efficiency of the primary key.

[0094] In the primary key identifier generated by this application, the first two fields are the business date and batch number. The first 9 digits of the primary key can be used to know which day and which batch each record belongs to, which can greatly facilitate business operations such as data statistics, query and modification in the background. Compared with algorithms such as the self-increment ID and the Universally Unique Identifier (UUID) generation algorithm, this solution can not only ensure uniqueness, but also the generated primary key field has unique business significance and is no longer a simple random number.

[0095] Advantages of this application:

[0096] 1. Compared with the auto-increment primary key algorithm and the traditional timestamp-based primary key identifier generation algorithm (such as the UUID algorithm or the snowflake algorithm), the primary key identifier generation method used in this application can not only save database performance, but also generate primary key identifiers in a short time for large data files without generating repeated random numbers, thereby ensuring the uniqueness, availability and efficiency of the primary key identifier.

[0097] 2. In terms of efficiency, this application adopts the idea of ​​exchanging space for time, using a larger List container to store the generated random numbers, and then taking numbers from the container to shorten the processing time, so that this method generates primary key identifiers while meeting the availability and the algorithm efficiency is also high. Tests have shown that it only takes about 200 seconds to complete the processing of 80 million data. However, using a timestamp-based method, the time interval for generating primary keys must be limited to ensure that primary keys are not repeated, and processing the same data takes several hours.

[0098] 3. The primary key identifier generated by this application is preceded by a unique field consisting of the business date and batch number, which can be used for convenient statistics and query operations. Compared with simple random number IDs and auto-incrementing integer primary keys that have no business meaning, the primary key identifier generated by this application has unique business meaning and can be queried and counted by date and batch, making subsequent queries and statistics based on the primary key very convenient.

[0099] The present application specifically relates to a unique primary key ID fusion generation method based on the total amount of data. In tasks such as data table generation and database migration, each table often requires a primary keyword as a unique identifier for row data, thereby uniquely identifying the entity and serving as an index to accelerate query efficiency. For large-scale data volumes, traditional generation methods will have problems such as keyword duplication conflicts, long generation time, and excessive storage usage. Therefore, this article proposes a primary key generation method based on the total amount of data, aiming to solve the problem of unique primary key identifier generation for large-volume data processing in high-concurrency scenarios.

[0100] This application uses a primary key identifier generation algorithm based on the total amount of data to solve the problem of generating a unique identifier primary key for each row of data in the scenario of processing large data files in a short period of time, and the unique identifier generation method based on the total amount of data described in this article can ensure the uniqueness and availability of the primary key. For the banking business scenario described in this article, it is more practical to process data for the received large data files and regenerate the primary key fields. It has high processing efficiency and can avoid duplication problems based on timestamps.

[0101] In terms of efficiency, the present application adopts the idea of ​​exchanging space for time, uses a larger List container to store the generated random numbers, and then retrieves the numbers from the container to shorten the processing time, so that the method generates primary key identifiers while meeting the availability and the algorithm efficiency is also high. Tests have shown that it only takes about 200 seconds to complete the processing of 80 million data. The algorithm of the present invention adopts the idea of ​​exchanging space for time, has high algorithm processing efficiency, and is suitable for processing massive data under high concurrency.

[0102] Compared with the random number primary key ID and the self-incrementing integer primary key algorithm that have no business meaning, the primary key ID generated by this application contains date and batch number information and thus has unique business significance, which is more conducive to subsequent business operations such as query statistics.

[0103] Beneficial effects of the embodiments of the present application: random numbers are generated based on the total amount of data, so that primary key conflicts caused by repeated timestamps due to too much data can be avoided, and the uniqueness requirement of the primary key can be met. In addition, a larger list container is used to store the generated random numbers, and then the numbers are taken from the container to shorten the processing time, so that the algorithm efficiency of the primary key identifier generated by this scheme is relatively high while meeting the availability. At the same time, the date and batch number are added to the primary key identifier, which greatly facilitates subsequent business operations such as data query and statistics, and improves business functionality. Therefore, compared with the self-incrementing primary key algorithm and the traditional timestamp-based primary key identifier generation algorithm, the primary key identifier generation method of this scheme can not only save the performance of the database, but also will not generate repeated random numbers when generating primary key identifiers for large data files in a short time, thereby ensuring the uniqueness, availability and efficiency of the generated primary key identifier, and improving the efficiency of generating unique primary key identifiers.

[0104] Based on the above embodiments Figure 1 A unique primary key identifier generation method is disclosed, and the embodiment of the present application also discloses a unique primary key identifier generation device, such as Figure 2 As shown, the unique primary key identifier generating device includes:

[0105] The first reading unit 201 is used to read the data file to be processed and obtain the total value of the number of file records of the data file to be processed;

[0106] The first generating unit 202 is used to generate a number of non-repeating initial random numbers corresponding to the total value according to a preset script;

[0107] The storage unit 203 is used to store the number of non-repeating initial random numbers corresponding to the total value into a list container;

[0108] The acquisition unit 204 is used to acquire the date of the primary key identifier and the batch number of the primary key identifier from each file record in the data file to be processed;

[0109] The second reading unit 205 is used to read the target random number from the list container according to a preset reading method; wherein the preset reading method is used to ensure the uniqueness of the read target random number;

[0110] The second generating unit 206 is used to generate a unique primary key identifier for the file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

[0111] Furthermore, the first reading unit 201 includes:

[0112] A first reading module, used for reading all rows of the data file to be processed;

[0113] The acquisition module is used to obtain the line numbers corresponding to all lines of the file to be processed through a preset built-in function;

[0114] The statistical module is used to count the number of rows corresponding to all rows of the file to be processed, and obtain the total value of the number of file records of the data file to be processed.

[0115] Furthermore, the acquisition unit 204 includes:

[0116] A definition module is used to define and initialize the variable number of the data file to be processed; wherein the variable number is used to record the batch number of the day of reading;

[0117] A first determining module, used to determine the first line of the data file to be processed as the starting line, and determine the last line of the data file to be processed as the ending line;

[0118] A second determination module is used to determine the reading order according to the start line and the end line;

[0119] A third determination module is used to read all rows of the data file to be processed according to the reading order, obtain the reading date, and determine the reading date as the date of the primary key identifier;

[0120] The second reading module is used to read the initialized batch number of the data file to be processed to obtain the batch number identified by the primary key.

[0121] Furthermore, the second reading unit 205 is specifically used to enable each file record in the data file to be processed to obtain a random number from the list container, and determine the random number as a non-repeating target random number; wherein the number of times all the random numbers read in the list container are read is 1.

[0122] Furthermore, the unique primary key identifier generating device further includes:

[0123] The writing unit is used to write the unique primary key identifier of each file record into the data file to be processed to complete the generation process of the primary key identifier field.

[0124] Beneficial effects of the embodiments of the present application: random numbers are generated based on the total amount of data, so that primary key conflicts caused by repeated timestamps due to too much data can be avoided, and the uniqueness requirement of the primary key can be met. In addition, a larger list container is used to store the generated random numbers, and then the numbers are taken from the container to shorten the processing time, so that the algorithm efficiency of the primary key identifier generated by this scheme is relatively high while meeting the availability. At the same time, the date and batch number are added to the primary key identifier, which greatly facilitates subsequent business operations such as data query and statistics, and improves business functionality. Therefore, compared with the self-incrementing primary key algorithm and the traditional timestamp-based primary key identifier generation algorithm, the primary key identifier generation method of this scheme can not only save the performance of the database, but also will not generate repeated random numbers when generating primary key identifiers for large data files in a short time, thereby ensuring the uniqueness, availability and efficiency of the generated primary key identifier, and improving the efficiency of generating unique primary key identifiers.

[0125] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned unique primary key identifier generation method.

[0126] The present application also provides an electronic device, whose structural diagram is as follows: Figure 3 As shown, it specifically includes a memory 301 and one or more instructions 302, wherein the one or more instructions 302 are stored in the memory 301 and are configured to be executed by one or more processors 303 to execute the one or more instructions 302 to perform the above-mentioned unique primary key identifier generation method.

[0127] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0128] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0129] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0130] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0131] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0132] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for generating a unique primary key identifier, characterized in that: The method comprises: Read the data file to be processed and obtain the total value of the number of file records in the data file to be processed; Generate a number of non-repeating initial random numbers corresponding to the total value according to the preset script; Store the number of non-repeating initial random numbers corresponding to the total number value into a list container; For each file record in the data file to be processed, obtaining the date of the primary key identifier and the batch number of the primary key identifier from the file record; Reading the target random number from the list container according to a preset reading method; wherein the preset reading method is used to ensure the uniqueness of the read target random number; A unique primary key identifier for the file record is generated according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

2. The method according to claim 1, characterized in that The step of reading the data file to be processed and obtaining the total value of the number of file records of the data file to be processed includes: Read all the lines of the data file to be processed; Get the line numbers corresponding to all lines of the file to be processed through the preset built-in function; The number of rows corresponding to all rows of the to-be-processed file is counted to obtain a total value of the number of file records of the to-be-processed data file.

3. The method according to claim 1, characterized in that The date and batch number of the primary key identifier are obtained from the file record, including: Define and initialize the variable number of the data file to be processed; wherein the variable number is used to record the batch number of the day of reading; The first row of the data file to be processed is determined as the start row, and the last row of the data file to be processed is determined as the end row; Determine a reading order according to the start row and the end row; Read all rows of the data file to be processed according to the reading order, obtain a reading date, and determine the reading date as the date of the primary key identifier; Read the initialized batch number of the data file to be processed to obtain the batch number identified by the primary key.

4. The method according to claim 1, characterized in that: The step of reading the target random number from the list container in a preset reading manner includes: For each file record in the data file to be processed, the file record obtains a random number from the list container, and the random number is determined as a non-repeating target random number; wherein the number of times all the random numbers read in the list container are read is 1.

5. The method according to claim 1, characterized in that After generating a unique primary key identifier for a file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number, the method further includes: The unique primary key identifier of each file record is written into the data file to be processed to complete the generation process of the primary key identifier field.

6. A unique primary key identifier generating device, characterized in that: The device comprises: A first reading unit is used to read the data file to be processed and obtain a total value of the number of file records in the data file to be processed; A first generating unit, used to generate a number of non-repeating initial random numbers corresponding to the total value according to a preset script; A storage unit, used for storing a number of non-repeating initial random numbers corresponding to the total value into a list container; An acquisition unit, for acquiring, for each file record in the data file to be processed, a date of the primary key identifier and a batch number of the primary key identifier from the file record; A second reading unit, configured to read a target random number from the list container in a preset reading manner; wherein the preset reading manner is configured to ensure the uniqueness of the read target random number; The second generating unit is used to generate a unique primary key identifier for a file record according to the date of the primary key identifier, the batch number of the primary key identifier and the target random number.

7. The device according to claim 6, characterized in that The first reading unit includes: A first reading module, used for reading all rows of the data file to be processed; The acquisition module is used to obtain the line numbers corresponding to all lines of the file to be processed through a preset built-in function; The statistical module is used to count the number of rows corresponding to all rows of the to-be-processed file to obtain the total value of the number of file records of the to-be-processed data file.

8. The device according to claim 6, characterized in that The acquisition unit comprises: A definition module is used to define and initialize the variable numbers of the data file to be processed; wherein the variable numbers are used to record the batch number of the day of reading; A first determining module, used to determine the first line of the data file to be processed as the starting line, and determine the last line of the data file to be processed as the ending line; A second determining module, used for determining a reading order according to the start row and the end row; A third determination module is used to read all rows of the data file to be processed according to the reading order, obtain a reading date, and determine the reading date as a date identified by the primary key; The second reading module is used to read the initialized batch number of the data file to be processed to obtain the batch number identified by the primary key.

9. The device according to claim 6, characterized in that The second reading unit is specifically used for: For each file record in the data file to be processed, the file record obtains a random number from the list container, and the random number is determined as a non-repeating target random number; wherein the number of times all the random numbers read in the list container are read is 1.

10. The device according to claim 6, characterized in that Also includes: The writing unit is used to write the unique primary key identifier of each file record into the data file to be processed to complete the generation process of the primary key identifier field.

Images