Association table checking method and device, storage medium and electronic equipment

Abstract

This disclosure relates to a method, apparatus, storage medium, and electronic device for verifying association tables, belonging to the field of electronic information processing technology. The method includes: controlling a first number of read threads to read data to be verified in parallel from a source data table and writing the data to be verified into a blocking queue in parallel; controlling a second number of verification threads to read the data to be verified in parallel from the blocking queue and performing parallel verification of the data to be verified against a target data table according to a preset association relationship, wherein the source data table and the target data table have an association relationship; and merging the verification results of each verification thread to obtain a total verification result corresponding to the source data table and the target data table. This disclosure, by reading data from the source data table through parallel-running read threads and performing verification through parallel-running verification threads, can effectively improve the verification speed of the association relationship between the source data table and the target data table.

Description

Technical Field

[0001] This disclosure relates to the field of electronic information processing technology, and more specifically, to a method, apparatus, storage medium, and electronic device for verifying association tables. Background Technology

[0002] With the rapid development of information technologies such as the Internet and cloud computing, the resulting data volume has also exploded. In the process of analyzing massive amounts of data, data validation is frequently required to determine whether the data meets requirements. This is especially true for multiple data tables with interrelationships, where it's necessary to validate whether the data within each table conforms to those relationships. For example, consider a student table storing information on all junior high school students in a region, and a school table storing information on all junior high schools in that region. It's necessary to validate that each student in the student table corresponds to a school in the school table, meaning each junior high school student corresponds to one junior high school. When the amount of data stored in the tables is very large, the existing data validation speed is too slow to meet practical needs. For example, with billions of data entries, a validation speed of 20,000 records / second would mean that validating 10 billion records would take nearly 6 days, making it difficult to implement in real-world applications. Summary of the Invention

[0003] The purpose of this disclosure is to provide a method, apparatus, storage medium, and electronic device for verifying association tables, in order to solve related problems existing in the prior art.

[0004] To achieve the above objectives, according to a first aspect of the present disclosure, a method for validating an association table is provided, the method comprising:

[0005] Control a first number of read threads to read the data to be verified from the source data table in parallel, and write the data to be verified into the blocking queue in parallel;

[0006] Control a second number of verification threads to read the data to be verified in parallel from the blocking queue, and perform parallel verification of the data to be verified and the target data table according to a preset association relationship, wherein the source data table and the target data table have the association relationship.

[0007] The verification results of each verification thread are merged to obtain the total verification result corresponding to the source data table and the target data table.

[0008] Optionally, each reading thread corresponds to a database connection, and each reading thread corresponds to a data group to be verified in the source data table, and each data group to be verified includes multiple data to be verified.

[0009] The control of a first number of read threads to read the data to be verified from the source data table in parallel, and to write the data to be verified into a blocking queue in parallel, includes:

[0010] Each reading thread is controlled to sequentially read one of the data to be verified from the corresponding data group through the corresponding database connection, and the data to be verified is written into the blocking queue.

[0011] The control of a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship, includes:

[0012] Each verification thread is controlled to read one piece of data to be verified sequentially from the blocking queue, and to verify the data to be verified against the target data table according to the association relationship.

[0013] Optionally, controlling a first number of read threads to read the data to be verified from the source data table in parallel, and writing the data to be verified into a blocking queue in parallel, includes:

[0014] If the blocking queue is not full, control a first number of reading threads to read the data to be verified from the source data table in parallel, and write the data to be verified into the blocking queue in parallel.

[0015] If the blocking queue is full, the first number of read threads will be blocked.

[0016] The control of a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship, includes:

[0017] If the blocking queue is not empty, control a second number of the verification threads to read the data to be verified in parallel from the blocking queue, and verify the data to be verified against the target data table according to the association relationship;

[0018] If the blocking queue is empty, block a second number of the verification threads.

[0019] Optionally, merging the verification results of each verification thread includes:

[0020] When the blocking queue is empty and each reading thread has finished reading the corresponding group of data to be verified, the verification results of each verification thread are merged to obtain the total verification result.

[0021] Optionally, each verification thread corresponds to a result cache; the method further includes:

[0022] Store the target data table into the target cache;

[0023] The parallel verification of the data to be verified and the target data table according to a preset association relationship includes:

[0024] Control each of the verification threads to read the target data table from the target cache;

[0025] The verification thread is controlled to sequentially verify whether the data to be verified and each data in the specified column of the target data table satisfy the association relationship, and the verification result is stored in the result cache corresponding to the verification thread.

[0026] Optionally, merging the verification results of each verification thread includes:

[0027] The verification results stored in the result cache corresponding to each verification thread are merged to obtain the total verification result.

[0028] To achieve the above objectives, according to a second aspect of the present disclosure, a verification apparatus for an association table is provided, the apparatus comprising:

[0029] The reading module is used to control a first number of reading threads to read the data to be verified from the source data table in parallel, and to write the data to be verified into the blocking queue in parallel.

[0030] The verification module is used to control a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship, wherein the source data table and the target data table have the association relationship.

[0031] The merging module is used to merge the verification results of each verification thread to obtain the total verification result corresponding to the source data table and the target data table.

[0032] Optionally, each reading thread corresponds to a database connection, and each reading thread corresponds to a data group to be verified in the source data table, and each data group to be verified includes multiple data to be verified.

[0033] The reading module is used for:

[0034] Each reading thread is controlled to sequentially read one of the data to be verified from the corresponding data group through the corresponding database connection, and the data to be verified is written into the blocking queue.

[0035] The verification module is used for:

[0036] Each verification thread is controlled to read one piece of data to be verified sequentially from the blocking queue, and to verify the data to be verified against the target data table according to the association relationship.

[0037] Optionally, the reading module includes:

[0038] The reading submodule is used to control a first number of reading threads to read the data to be verified from the source data table in parallel when the blocking queue is not full, and to write the data to be verified into the blocking queue in parallel.

[0039] The first blocking submodule is used to block a first number of read threads when the blocking queue is full;

[0040] The verification module includes:

[0041] The verification submodule is used to control a second number of verification threads to read the data to be verified in parallel from the blocking queue when the blocking queue is not empty, and to verify the data to be verified against the target data table according to the association relationship.

[0042] The second blocking submodule is used to block a second number of the verification threads when the blocking queue is empty.

[0043] Optionally, the merging module is used to:

[0044] When the blocking queue is empty and each reading thread has finished reading the corresponding group of data to be verified, the verification results of each verification thread are merged to obtain the total verification result.

[0045] Optionally, each verification thread corresponds to a result cache; the apparatus further includes:

[0046] Storage module, used to store the target data table into the target cache;

[0047] The verification module is used for:

[0048] Each verification thread is controlled to read the target data table from the target cache; the verification thread is controlled to sequentially verify whether the data to be verified and each data in the specified column of the target data table satisfy the association relationship, and the verification result is stored in the result cache corresponding to the verification thread.

[0049] Optionally, the merging module is used to:

[0050] The verification results stored in the result cache corresponding to each verification thread are merged to obtain the total verification result.

[0051] According to a third aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect of the present disclosure.

[0052] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0053] A memory on which computer programs are stored;

[0054] A processor is configured to execute the computer program in the memory to implement the steps of the method described in the first aspect of the present disclosure.

[0055] Through the above technical solution, this disclosure controls a first number of read threads to read the data to be verified from the source data table in parallel and write it into a blocking queue. It also controls a second number of verification threads to read the data to be verified from the blocking queue in parallel and perform parallel verification between the data to be verified and the target data table according to a preset association relationship. Finally, the verification results of each verification thread are merged to obtain the total verification result corresponding to the source data table and the target data table. This disclosure, by using parallel-running read threads to read data from the source data table and parallel-running verification threads to perform verification, can effectively improve the verification speed of the association relationship between the source data table and the target data table.

[0056] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0057] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0058] Figure 1 This is a flowchart illustrating a method for validating an association table according to an exemplary embodiment;

[0059] Figure 2 This is a schematic diagram illustrating the collaboration between a read thread and a verification thread according to an exemplary embodiment;

[0060] Figure 3 This is a flowchart illustrating another method for validating an association table according to an exemplary embodiment;

[0061] Figure 4 This is a flowchart illustrating another method for validating an association table according to an exemplary embodiment;

[0062] Figure 5 This is a block diagram illustrating a verification device for an association table according to an exemplary embodiment;

[0063] Figure 6 This is a block diagram illustrating another verification device for an association table according to an exemplary embodiment;

[0064] Figure 7 This is a block diagram illustrating another verification device for an association table according to an exemplary embodiment;

[0065] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment;

[0066] Figure 9 This is a block diagram illustrating another electronic device according to an exemplary embodiment. Detailed Implementation

[0067] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0068] Figure 1 This is a flowchart illustrating a method for validating an association table according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps:

[0069] Step 101: Control the first number of read threads to read the data to be verified in parallel from the source data table and write the data to be verified into the blocking queue in parallel.

[0070] For example, if the source and target data tables have a pre-defined association, then we can verify whether the data to be verified in the source table satisfies this association with the data in the target table. The association indicates the relationship between the data to be verified and a specified column in the target table. The association can be an existential relationship, such as every piece of data to be verified existing in the specified column. It can also be a statistical relationship, such as the sum (or maximum / minimum, average, variance, etc.) of all the data to be verified being equal to a specific data point in the specified column.

[0071] Taking an existence relation as an example, the source data table is a student table storing student information for a school, and the target data table is a class table storing class information for that school. The data to be validated could be the ID number of each student in the student table, and the specified column could be the ID number column in the class table. The existence relation indicates that every ID number in the student table exists in the ID number column of the class table. Taking a statistical relation as another example, the source data table is a sales table storing sales details for product A in a shopping mall, and the target data table is a master table storing sales information for all products in the mall. The data to be validated could be the quantity of product A sold each time in the sales table, and the specified column could be the total sales column in the master table storing the total sales volume for each product. The statistical relation indicates that the sum of all quantities in the sales table equals the total sales volume for product A in the total sales volume column of the master table.

[0072] A first set of a certain number of read threads and a blocking queue can be pre-established. A blocking queue is a thread-safe queue, meaning it supports concurrent reads and writes. The first set of a certain number can be an integer greater than 1, such as 4 or 8. The first set of a certain number of read threads can run in parallel. Each read thread reads the data to be verified from the source data table and writes it to the blocking queue. The source data table will contain multiple data items to be verified. Each of the first set of a certain number of read threads can repeatedly execute the step of reading one data item from the source data table and writing it to the blocking queue, thus achieving parallel reading and writing of data to be verified from the source data table to the blocking queue. Specifically, the multiple data items to be verified can be divided into multiple groups according to the number of read threads. Each group includes at least one data item to be verified, and each read thread reads the data items included in its corresponding group. For example, if the source data table contains 10,000 data items to be verified, and there are 8 (the first number) read threads, then the 10,000 data items can be divided into 8 groups (each group containing 1,250 data items). One read thread is responsible for reading the 1,250 data items in one group. Alternatively, the corresponding read thread can be determined based on the sequence number of each data item to be verified. For example, if there are 4 (the first number) read threads, and the 105th data item to be verified in the source data table is the sequence number modulo the first number, then the corresponding read thread can be determined by taking the sequence number modulo the first number, i.e., 105%4 = 1. Therefore, the first read thread can be determined to read the 105th data item to be verified in the source data table.

[0073] Step 102: Control the second number of verification threads to read the data to be verified in parallel from the blocking queue, and perform parallel verification of the data to be verified and the target data table according to the preset association relationship. There is an association relationship between the source data table and the target data table.

[0074] For example, a second number of verification threads can be pre-established. This second number can be an integer greater than 1, such as 4 or 8. The second number can be the same as or different from the first number; this disclosure does not impose specific limitations on this. The second number of verification threads can run in parallel. Each verification thread reads data to be verified from the blocking queue and verifies the data against the target data table according to a preset association relationship. The blocking queue may contain multiple data to be verified. Each of the second number of verification threads can repeatedly execute the steps of reading one piece of data from the blocking queue and verifying whether the data satisfies the association relationship with the target data table, thereby achieving parallel reading of data from the blocking queue and parallel verification of the data. Specifically, verifying whether the data satisfies the association relationship with the target data table can be done by determining whether the data exists in a specified column of the target data table, or by determining whether the data satisfies a preset statistical relationship with a certain data in a specified column of the target data table.

[0075] Step 103: Merge the verification results of each verification thread to obtain the total verification result corresponding to the source data table and the target data table.

[0076] For example, the verification results of each verification thread checking whether a piece of data to be verified and the target data table satisfy the association relationship can be merged to obtain the total verification result for the source and target data tables. The verification result indicates whether each piece of data to be verified satisfies the association relationship with the target data table, while the total verification result indicates whether the source and target data tables satisfy the association relationship. Specifically, each verification thread can be allocated a storage space to store the verification results. The verification result of each verification thread checking whether a piece of data to be verified and the target data table satisfy the association relationship can be stored in the corresponding storage space, and multiple verification results can be stored in the storage space. After verifying all the data to be verified in the source data table, all the verification results in the second number of storage spaces corresponding to the second number of verification threads can be merged to obtain the total verification result. Alternatively, the verification results of each verification thread checking whether a piece of data to be verified and the target data table satisfy the association relationship can be stored in the same storage space. After verifying all the data to be verified in the source data table, all the verification results stored in this storage space can be merged to obtain the total verification result.

[0077] It's important to note that to better match the number of read threads and the number of verification threads, the first and second numbers can be determined based on their respective execution speeds. For example, if the read threads execute faster than the verification threads, the first number can be less than the second, meaning more verification threads. Conversely, if the read threads execute slower than the verification threads, the first number can be greater than the second, meaning more read threads. Alternatively, the ratio of the first number to the second number can be equal to the ratio of the execution speed of the verification threads to the execution speed of the read threads.

[0078] The processing of the first number of read threads, the blocking queue, and the second number of verification threads in this embodiment can be understood as a multi-threaded concurrent cooperation mode, such as... Figure 2 As shown, a multi-threaded concurrent collaboration pattern can be, for example, a producer-consumer pattern, where the read thread is the producer, the blocking queue is the shared data area, and the verification thread is the consumer. This enables parallel verification between the source and target data tables, effectively improving verification speed to meet practical needs.

[0079] In summary, this disclosure controls a first number of read threads to read the data to be verified from the source data table in parallel and write it to a blocking queue. It also controls a second number of verification threads to read the data to be verified from the blocking queue in parallel and perform parallel verification between the data to be verified and the target data table according to a preset association relationship. Finally, the verification results of each verification thread are merged to obtain the total verification result corresponding to the source and target data tables. This disclosure, by using parallel-running read threads to read data from the source data table and parallel-running verification threads to perform verification, can effectively improve the verification speed of the association relationship between the source and target data tables.

[0080] In one implementation, each read thread corresponds to a database connection, and each read thread corresponds to a group of data to be verified in the source data table. Each group of data to be verified includes multiple data items to be verified. The source and target data tables can be stored in a database. Accordingly, to read the data to be processed from the source data table, the read thread first establishes a database connection. To further improve the verification speed, a separate database connection can be established for each read thread, allowing each thread to fully utilize its execution capabilities. Simultaneously, to ensure that the first number of read threads can cover all the data to be verified in the source data table, all the data to be verified in the source data table can be divided into a first number of groups of data to be verified, each corresponding one-to-one with one of the first number of read threads. Each group of data to be verified can include multiple data items to be verified, and the size of each group can be the same or different.

[0081] Step 101 can be implemented in the following ways:

[0082] Each read thread is controlled to read one piece of data to be verified from the corresponding data group in sequence through the corresponding database connection, and the data to be verified is written to the blocking queue.

[0083] Step 102 can be implemented in the following ways:

[0084] Each verification thread is controlled to read one piece of data to be verified sequentially from the blocking queue, and to verify the data to be verified against the target data table according to the association relationship.

[0085] Correspondingly, each read thread can sequentially read one piece of data to be verified from the corresponding data group through the corresponding database connection and write it to the blocking queue. That is, the read thread repeatedly executes the step of reading one piece of data to be verified from the corresponding data group through the corresponding database connection and writing it to the blocking queue until all the data to be verified in the corresponding data group has been read and written to the blocking queue. Similarly, each verification thread can sequentially read one piece of data to be verified from the blocking queue and verify it against the target data table according to the association relationship. That is, the verification thread repeatedly executes the step of reading one piece of data to be verified from the blocking queue and performing verification until all the data to be verified in the source data table has been verified.

[0086] Figure 3 This is a flowchart illustrating another method for validating an association table according to an exemplary embodiment, such as... Figure 3 As shown, step 101 can be achieved through the following steps:

[0087] Step 1011: If the blocking queue is not full, control the first number of read threads to read the data to be verified from the source data table in parallel, and write the data to be verified into the blocking queue in parallel.

[0088] Step 1012: If the blocking queue is full, block the first number of read threads.

[0089] Step 102 can be achieved through the following steps:

[0090] Step 1021: If the blocking queue is not empty, control a second number of verification threads to read the data to be verified in parallel from the blocking queue, and verify the data to be verified against the target data table according to the association relationship.

[0091] Step 1022: If the blocking queue is empty, block the second number of verification threads.

[0092] For example, the number of read and verification threads can be controlled based on the amount of data to be verified stored in the blocking queue. Specifically, when creating the blocking queue, its size can be preset to a specified size. During the execution of the read threads, if the blocking queue is not full (either empty or the amount of data to be verified stored in the blocking queue is less than the specified size), it means the blocking queue can continue to store data to be verified. In this case, the first number of read threads can be controlled to read the data to be verified from the source data table in parallel and write it to the blocking queue in parallel. If the blocking queue is full (i.e., the amount of data to be verified stored in the blocking queue equals the specified size), it means the blocking queue cannot continue to store data to be verified. In this case, the first number of read threads can be blocked, meaning the first number of read threads are paused.

[0093] During the execution of the verification thread, if the blocking queue is not empty (i.e., the number of data to be verified stored in the blocking queue is greater than 0 and less than or equal to the specified size), it indicates that there is still data to be verified in the blocking queue. In this case, a second number of verification threads can be controlled to read the data to be verified from the blocking queue in parallel and verify it against the target data table according to the association relationship. If the blocking queue is empty (i.e., there is no data to be verified in the blocking queue), then the second number of verification threads can be blocked, that is, the second number of verification threads can be paused. This allows the blocking queue to support concurrent reading or writing, thus achieving thread safety.

[0094] In another implementation, step 103 may include:

[0095] When the blocking queue is empty and each reading thread has finished reading its corresponding data group to be verified, the verification results of each verification thread are merged to obtain the total verification result.

[0096] For example, before merging the verification results of each verification thread, it can be determined whether the verification of all data to be verified has been completed. If it is determined that the verification of all data to be verified has been completed, the verification results of each verification thread can be merged to obtain the total verification result. If the blocking queue is empty and each read thread has finished reading its corresponding group of data to be verified, then it means that the verification of all data to be verified has been completed. Specifically, a CountDownLatch counter can be established, with the initial value of the counter being the first number. When each read thread finishes reading all the data to be verified in its corresponding group, the counter is decremented by 1. When the counter is 0 and the blocking queue is empty, it is determined that the verification of all data to be verified has been completed.

[0097] Figure 4 This is a flowchart illustrating another method for validating an association table according to an exemplary embodiment, such as... Figure 4 As shown, each verification thread corresponds to a result buffer. The method may also include the following steps:

[0098] Step 104: Store the target data table into the target cache.

[0099] For example, a target cache can be pre-established to store the target data table. Then, before the second number of verification threads verify the data to be verified, the target data table is stored in the target cache. This way, the verification threads do not need to repeatedly retrieve the target data table from the database when verifying the data, further improving the verification speed. Furthermore, only specified columns from the target data table can be stored in the target cache to reduce the space occupied by the target cache.

[0100] Accordingly, step 102 may include:

[0101] Step 1023: Control each verification thread to read the target data table from the target cache.

[0102] Step 1024: Control the verification thread to sequentially verify whether the data to be verified and each data in the specified column of the target data table satisfy the association relationship, and store the verification result in the result cache corresponding to the verification thread.

[0103] For example, a result cache can be pre-established for each verification thread, meaning there's a one-to-one correspondence between verification threads and result caches. The result cache stores the verification results of the corresponding verification thread. Specifically, each verification thread can read the target data table from the target cache, then determine whether the data to be verified satisfies the association relationship with each data in a specified column of the target data table, and store the verification result in the corresponding result cache.

[0104] Let's take a student table (as shown in Table 1) as the source data table and a class table (as shown in Table 2) as the target data table, which stores the class information of the school. The data to be validated can be the ID number of each student in the student table, and the specified column can be the column in the class table that stores the students' ID numbers (i.e., the student ID number column in Table 2).

[0105] student ID number gender age …… A 123456 male 13 …… B 123457 female 15 …… C 123458 female 13 …… …… …… …… …… ……

[0106] Table 1

[0107] Class number Class teacher's name Class of students Student ID number …… Class 1, Grade 7 a C 123458 …… Class 1, Grade 7 a D 234567 …… Class 1, Grade 7 a A 123456 …… …… …… …… …… Class 2, Grade 9 b E 345678 Class 2, Grade 9 b B 123457 …… …… …… …… ……

[0108] Table 2

[0109] Table 2 can be stored in the target cache first. Assuming the blocking queue already contains the ID numbers of students A, B, and C, namely 123456, 123457, and 123458, taking any verification thread as an example, this thread can first read Table 2 from the target cache, then read student B's ID number (123457) from the blocking queue, and then check if 123457 exists in the student ID number column. It can be seen that 123457 exists in the student ID number column (student B belongs to Class 2, Grade 9). Therefore, "exists" can be used as the verification result and stored in the result cache corresponding to this verification thread.

[0110] In another implementation, step 103 may include:

[0111] The verification results stored in the result cache corresponding to each verification thread are merged to obtain the total verification result.

[0112] For example, when merging validation results, all validation results can be retrieved from the result cache corresponding to each validation thread, and then all validation results can be merged to obtain the total validation result. The total validation result can be a collection of all validation results, or it can be determined based on all validation results. For example, if 4 validation threads correspond to 4 result caches, and the second result cache stores 100 validation results, of which 99 are "existent" and 1 is "not present," while the other 3 result caches store validation results that are all "existent," then merging the validation results in the 4 result caches will result in a total validation result of "failed," indicating that the source data table and the target data table do not satisfy the association relationship. Furthermore, the relevant information of the data to be validated with a validation result of "not present" can also be used as the total validation result. The relevant information can be, for example, the sequence number of the data to be validated in the source data table, or the data to be validated itself. For example, the data to be validated with a validation result of "not present" in the second result cache is the 57th data to be validated in the source data table, and this data to be validated is: 5555. Therefore, "The 57th data to be verified failed" can be used as the total verification result, or "Data to be verified: 5555, failed" can be used as the total verification result.

[0113] It should be noted that the execution entity in this embodiment can be a terminal or a server, and this disclosure does not specifically limit it. The terminal may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. The server may include, but is not limited to, physical servers, server clusters, or cloud servers.

[0114] In summary, this disclosure controls a first number of read threads to read the data to be verified from the source data table in parallel and write it to a blocking queue. It also controls a second number of verification threads to read the data to be verified from the blocking queue in parallel and perform parallel verification between the data to be verified and the target data table according to a preset association relationship. Finally, the verification results of each verification thread are merged to obtain the total verification result corresponding to the source and target data tables. This disclosure, by using parallel-running read threads to read data from the source data table and parallel-running verification threads to perform verification, can effectively improve the verification speed of the association relationship between the source and target data tables.

[0115] Figure 5 This is a block diagram illustrating a verification device for an association table according to an exemplary embodiment, such as... Figure 5 As shown, the device 200 may include:

[0116] The reading module 201 is used to control a first number of reading threads to read the data to be verified from the source data table in parallel and write the data to be verified into the blocking queue in parallel.

[0117] The verification module 202 is used to control a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship. There is an association relationship between the source data table and the target data table.

[0118] The merging module 203 is used to merge the verification results of each verification thread to obtain the total verification result corresponding to the source data table and the target data table.

[0119] In one implementation, each read thread corresponds to a database connection, and each read thread corresponds to a data group to be verified in the source data table. Each data group to be verified includes multiple data items to be verified.

[0120] The reading module 201 can be used for:

[0121] Each read thread is controlled to read one piece of data to be verified from the corresponding data group in sequence through the corresponding database connection, and the data to be verified is written to the blocking queue.

[0122] Verification module 202 can be used for:

[0123] Each verification thread is controlled to read one piece of data to be verified sequentially from the blocking queue, and to verify the data to be verified against the target data table according to the association relationship.

[0124] Figure 6 This is a block diagram illustrating another verification device for an association table according to an exemplary embodiment, such as... Figure 6 As shown, the reading module 201 may include:

[0125] The read submodule 2011 is used to control a first number of read threads to read the data to be verified from the source data table in parallel when the blocking queue is not full, and to write the data to be verified into the blocking queue in parallel.

[0126] The first blocking submodule 2012 is used to block a first number of read threads when the blocking queue is full.

[0127] Verification module 202 may include:

[0128] The verification submodule 2021 is used to control a second number of verification threads to read the data to be verified in parallel from the blocking queue when the blocking queue is not empty, and to verify the data to be verified against the target data table according to the association relationship.

[0129] The second blocking submodule 2022 is used to block a second number of verification threads when the blocking queue is empty.

[0130] In another implementation, the merging module 203 can be used for:

[0131] When the blocking queue is empty and each reading thread has finished reading its corresponding data group to be verified, the verification results of each verification thread are merged to obtain the total verification result.

[0132] Figure 7 This is a block diagram illustrating another verification device for an association table according to an exemplary embodiment, such as... Figure 7 As shown, each verification thread corresponds to a result buffer. The device 200 may further include:

[0133] Storage module 204 is used to store the target data table into the target cache.

[0134] Accordingly, the verification module 202 can be used for:

[0135] Control each verification thread to read the target data table from the target cache. Control each verification thread to sequentially verify whether the data to be verified satisfies the association relationship with each data in the specified column of the target data table, and store the verification result in the result cache corresponding to the verification thread.

[0136] In another implementation, the merging module 203 can be used for:

[0137] The verification results stored in the result cache corresponding to each verification thread are merged to obtain the total verification result.

[0138] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0139] In summary, this disclosure controls a first number of read threads to read the data to be verified from the source data table in parallel and write it to a blocking queue. It also controls a second number of verification threads to read the data to be verified from the blocking queue in parallel and perform parallel verification between the data to be verified and the target data table according to a preset association relationship. Finally, the verification results of each verification thread are merged to obtain the total verification result corresponding to the source and target data tables. This disclosure, by using parallel-running read threads to read data from the source data table and parallel-running verification threads to perform verification, can effectively improve the verification speed of the association relationship between the source and target data tables.

[0140] Figure 8 This is a block diagram illustrating an electronic device 300 according to an exemplary embodiment. Figure 8As shown, the electronic device 300 may include a processor 301 and a memory 302. The electronic device 300 may also include one or more of a multimedia component 303, an input / output (I / O) interface 304, and a communication component 305.

[0141] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps in the aforementioned method for verifying the association table. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 302 or transmitted via communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 305 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0142] In an exemplary embodiment, the electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method for verifying the association table.

[0143] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-described method for verifying the association table. For example, the computer-readable storage medium may be the memory 302 including the program instructions, which may be executed by the processor 301 of the electronic device 300 to complete the above-described method for verifying the association table.

[0144] Figure 9 This is a block diagram illustrating another electronic device 400 according to an exemplary embodiment. For example, electronic device 400 may be provided as a server. (Refer to...) Figure 9 The electronic device 400 includes a processor 422, which may be one or more, and a memory 432 for storing computer programs executable by the processor 422. The computer program stored in the memory 432 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processor 422 may be configured to execute the computer program to perform the aforementioned method for verifying the association table.

[0145] Additionally, the electronic device 400 may also include a power supply component 426 and a communication component 450. The power supply component 426 can be configured to perform power management of the electronic device 400, and the communication component 450 can be configured to enable communication of the electronic device 400, such as wired or wireless communication. Furthermore, the electronic device 400 may also include an input / output (I / O) interface 458. The electronic device 400 can operate on an operating system, such as Windows Server, stored in memory 432. TM Mac OSX TM Unix TM Linux TM etc.

[0146] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the above-described method for verifying the association table. For example, the non-transitory computer-readable storage medium may be the memory 432 including the program instructions, which may be executed by the processor 422 of the electronic device 400 to complete the above-described method for verifying the association table.

[0147] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described verification method of the association table when executed by the programmable device.

[0148] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0149] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0150] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for validating an association table, characterized in that, The method includes: Control a first number of read threads to read the data to be verified from the source data table in parallel, and write the data to be verified into the blocking queue in parallel; Control a second number of verification threads to read the data to be verified in parallel from the blocking queue, and perform parallel verification of the data to be verified and the target data table according to a preset association relationship, wherein the source data table and the target data table have the association relationship. The verification results of each verification thread are merged to obtain the total verification result corresponding to the source data table and the target data table; Each of the reading threads corresponds to a database connection, and each of the reading threads corresponds to a data group to be verified in the source data table. Each data group to be verified includes multiple data to be verified. The control of a first number of read threads to read the data to be verified from the source data table in parallel, and to write the data to be verified into a blocking queue in parallel, includes: Each read thread is controlled to sequentially read one of the data to be verified from the corresponding data group through the corresponding database connection, and write the data to be verified into the blocking queue.

2. The method according to claim 1, characterized in that, The control of a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship, includes: Each verification thread is controlled to read one piece of data to be verified sequentially from the blocking queue, and to verify the data to be verified against the target data table according to the association relationship.

3. The method according to claim 1, characterized in that, The control of a first number of read threads to read the data to be verified from the source data table in parallel, and to write the data to be verified into a blocking queue in parallel, includes: If the blocking queue is not full, control a first number of reading threads to read the data to be verified from the source data table in parallel, and write the data to be verified into the blocking queue in parallel. If the blocking queue is full, the first number of read threads will be blocked. The control of a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship, includes: If the blocking queue is not empty, control a second number of the verification threads to read the data to be verified in parallel from the blocking queue, and verify the data to be verified against the target data table according to the association relationship; If the blocking queue is empty, block a second number of the verification threads.

4. The method according to claim 2, characterized in that, The step of merging the verification results of each verification thread includes: When the blocking queue is empty and each reading thread has finished reading the corresponding group of data to be verified, the verification results of each verification thread are merged to obtain the total verification result.

5. The method according to claim 1, characterized in that, Each of the verification threads corresponds to a result cache; the method further includes: Store the target data table into the target cache; The parallel verification of the data to be verified and the target data table according to a preset association relationship includes: Control each of the verification threads to read the target data table from the target cache; The verification thread is controlled to sequentially verify whether the data to be verified and each data in the specified column of the target data table satisfy the association relationship, and the verification result is stored in the result cache corresponding to the verification thread.

6. The method according to claim 5, characterized in that, The step of merging the verification results of each verification thread includes: The verification results stored in the result cache corresponding to each verification thread are merged to obtain the total verification result.

7. A verification device for an association table, characterized in that, The device includes: The reading module is used to control a first number of reading threads to read the data to be verified from the source data table in parallel, and to write the data to be verified into the blocking queue in parallel. The verification module is used to control a second number of verification threads to read the data to be verified in parallel from the blocking queue, and to perform parallel verification of the data to be verified and the target data table according to a preset association relationship, wherein the source data table and the target data table have the association relationship. The merging module is used to merge the verification results of each verification thread to obtain the total verification result corresponding to the source data table and the target data table; Each of the reading threads corresponds to a database connection, and each of the reading threads corresponds to a data group to be verified in the source data table. Each data group to be verified includes multiple data to be verified. The reading module is used for: Each read thread is controlled to sequentially read each piece of data to be verified from the corresponding data group to be verified through the corresponding database connection, and write the data to be verified into the blocking queue.

8. The apparatus according to claim 7, characterized in that, The verification module is used for: Each verification thread is controlled to repeatedly read the data to be verified from the blocking queue, and to verify the data to be verified against the target data table according to the association relationship.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-6.

10. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

Images