A data page CRC check system

Abstract

The invention discloses a data page CRC verification system, comprising a sending end for sending a data page, wherein the data page is defined as a metadata page, and a discrimination module is used to obtain the value of a data page of data to be detected, and the value is compared with the value of the metadata page. If they are different, a feature extraction module is used to extract features of the data page in the data to be detected, and a feature set to be detected is generated. The intersection of the feature set to be detected and the metadata feature set is extracted, and a data page including all features of the intersection in a standby database is selected through the intersection, wherein the data page is the metadata page; an index table is generated by integrating all feature sets by making a one-to-one correspondence between feature sets and data pages, and a CRC verification formula assignment procedure is performed according to a progressive relationship in an arrangement order until the assignment matches, thereby ensuring the correctness and integrity of the data page verification.

Description

Technical Field

[0001] The invention relates to the technical field of data page CRC verification, and in particular to a data page CRC verification system. Background Art

[0002] When the frame on the link is damaged during the data page transmission process, it is easy to cause erroneous data transmission. The use of CRC check in server network communication can correct errors in the information transmission process at a high rate, complete the calculation of the data check code in a very short time, and quickly complete the error correction process. The automatic retransmission of data packets greatly improves the communication speed of the server, providing guarantees for communication efficiency and security.

[0003] However, if only a single CRC check or retransmission data page operation is used, it is easy to lead to low transmission efficiency, which in turn causes redundant cyclic redundancy check processes in the CRC check calculation, and the same errors are prone to occur in the single retransmission data transmission path, resulting in the inability to provide complete and correct data pages, and thus the inability to guarantee the quality of communication data transmission; for this reason, we propose a data page CRC check system and method. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the present invention proposes a data page CRC check system to provide a complete data page CRC check program and flexibly back up and resend instructions for damaged data pages to be repaired.

[0005] To achieve the above object, according to an embodiment of the first aspect of the present invention, a data page CRC check system is provided, comprising:

[0006] The standby database has data pages stored internally.

[0007] A sending end, used for sending a data page, wherein the data page is defined as a metadata page, extracting a feature portion of the metadata page, and generating a metadata feature set;

[0008] A feature extraction module is used to extract metadata pages, extract feature parts of metadata pages, and generate metadata feature sets;

[0009] The discrimination module obtains the value of the metadata page through CRC check, and integrates it with the metadata page and metadata feature set into the initial transmission data;

[0010] At the receiving end, the initially sent data arrives at the receiving end via the data transmission channel. At this time, the data received by the receiving end is defined as the data to be detected. The assignment of the data page of the data to be detected is obtained through the discrimination module, and compared with the assignment of the metadata page. If they are different, the feature extraction module is used to extract features from the data page in the data to be detected, generate a feature set to be detected, extract the intersection of the feature set to be detected and the metadata feature set, and select the backup data page in the standby database that includes all the features of the intersection through the intersection. The backup data page is the metadata page.

[0011] Furthermore, the value assigned to the data page of the data to be detected is compared with the value assigned to the metadata page. If they are the same, it is determined that the backup data page in the data to be detected is a metadata page.

[0012] Furthermore, if there are multiple data pages in the standby database selected by intersection that include all the features of the intersection, the values ​​are obtained by CRC check respectively, and compared with the values ​​of the metadata page by the discrimination module, and the identical ones are the metadata pages.

[0013] Furthermore, all data pages in the standby database are extracted, and the feature parts of each data page are extracted by the feature extraction module to generate several feature sets, which correspond to data pages one by one, and all feature sets are integrated to generate an index table.

[0014] Furthermore, in the process of selecting data pages in the standby database including all the features of the intersection through the intersection, the target feature set is selected through the index table, the intersection is a subset of the target feature set, and the backup data page corresponding to the target feature set is extracted, and the backup data page is the metadata page.

[0015] Furthermore, a target feature set is selected through an index table, and a non-intersecting subset of the feature set to be detected and the metadata feature set is extracted through a feature extraction module, which is defined as a damage feature. The damage feature is integrated with the metadata page and the metadata feature set into a backup database, and the metadata page corresponding to the target feature set is marked as a damaged data page.

[0016] Furthermore, the backup database arranges the number of intersections containing the most target feature sets in descending order according to the number of intersections. When the value assigned to the backup data page containing the most intersections using the CRC check formula does not match the value assigned to the sent data page, the data page in the next order of arrangement is determined as the data page containing the most intersections and the CRC check formula assignment procedure is continued until the assignment matches. The backup database determines the backup data page with the matching assignment and replaces the backup data page with the backup data page to repair the damaged data page to be repaired.

[0017] Furthermore, when the CRC check formula is assigned to repeatedly check more than two backup data pages for two times, a non-intersection subset of the backup data pages is identified, and the backup data pages for which no damage features are identified are selected as metadata pages.

[0018] Furthermore, the standby database may back up the undamaged data pages that were initially transmitted.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] 1. The present invention extracts the intersection of the feature set to be detected and the metadata feature set, and generates an index table by integrating all feature sets through one-to-one correspondence between feature sets and data pages. The CRC check formula assignment procedure is performed in a progressive relationship according to the arrangement order until the assignment matches, thereby ensuring the correctness and completeness of the data page check.

[0021] 2. In the present invention, a feature extraction module is used to extract a non-intersection subset of the feature set to be detected and the metadata feature set, which is defined as a damage feature. The damage feature is integrated with the metadata page and the metadata feature set into a spare database. The present invention uses the spare database to back up damaged data pages and undamaged data pages in the metadata page, thereby preventing recognition errors during the intersection recognition process, ensuring the success rate of intersection recognition between the metadata feature set and the feature set of the backup data page, and reducing the number of CRC checks. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A system block diagram of a data page CRC check system of the present invention;

[0024] Figure 2 The present invention is a flow chart of a data page CRC verification method. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0026] The technical solution involved in the present invention is mainly used for data page backup and data page CRC verification; with the application of CRC to correct errors in information transmission process for a high proportion of data such as computer files, how to provide a complete data page CRC verification program and flexibly backup and resend instructions for damaged data pages to be repaired is a problem that needs to be solved urgently.

[0027] See also Figure 1-2 The present invention provides a data page CRC verification system. The prior art uses a configuration file of data object verification rules to perform a validity check on the data received on the page. The data page CRC verification system provided by the present invention performs a polynomial calculation on the data and attaches the result to the back of the frame. The receiving device also executes a similar algorithm to ensure the correctness and integrity of data transmission.

[0028] The present invention provides a data page CRC check system, comprising:

[0029] The standby database has data pages stored internally.

[0030] A sending end, used for sending a data page, wherein the data page is defined as a metadata page, extracting a feature portion of the metadata page, and generating a metadata feature set;

[0031] A feature extraction module is used to extract metadata pages, extract feature parts of metadata pages, and generate metadata feature sets;

[0032] The discrimination module obtains the value of the metadata page through CRC check, and integrates it with the metadata page and metadata feature set into the initial transmission data;

[0033] At the receiving end, the initially sent data arrives at the receiving end via the data transmission channel. At this time, the data received by the receiving end is defined as the data to be detected. The assignment of the data page of the data to be detected is obtained through the discrimination module, and compared with the assignment of the metadata page. If they are different, the feature extraction module is used to extract features from the data page in the data to be detected, generate a feature set to be detected, extract the intersection of the feature set to be detected and the metadata feature set, and select the backup data page in the standby database that includes all the features of the intersection through the intersection. The backup data page is the metadata page.

[0034] The server may be a central processing unit (CPU), other general-purpose servers, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose server may be a micro server or any conventional server, etc. The server is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire user terminal.

[0035] In a specific embodiment, the value assigned to the data page of the data to be detected is compared with the value assigned to the metadata page. If they are the same, it is determined that the backup data page in the data to be detected is a metadata page.

[0036] If there are multiple data pages in the standby database selected by the intersection that include all the features of the intersection, the values ​​are obtained by CRC check respectively, and compared with the values ​​of the metadata page by the discrimination module, and the same ones are the metadata pages;

[0037] Extract all data pages in the standby database, and extract the feature parts of each data page respectively through the feature extraction module to generate several feature sets, wherein the feature sets correspond to the data pages one by one, and integrate all the feature sets to generate an index table;

[0038] The data page is the basic unit for managing storage space in innoDB, and the page that stores data is called the index page. In the process of selecting the data page in the standby database that includes all the features of the intersection through the intersection, the target feature set is selected through the index table, and the intersection is a subset of the target feature set. The backup data page corresponding to the target feature set is extracted, and the backup data page is the metadata page.

[0039] In a specific embodiment, a target feature set is selected through an index table, and a non-intersecting subset of the feature set to be detected and the metadata feature set is extracted through a feature extraction module and defined as a damage feature. The damage feature is integrated with the metadata page and the metadata feature set into a backup database, and the metadata page corresponding to the target feature set is marked as a damaged data page.

[0040] The backup database arranges the number of intersections containing the most target feature sets in descending order according to the number of intersections. When the value assigned to the backup data page containing the most intersections using the CRC check formula does not match the value assigned to the sent data page, the data page in the next position in the arrangement order is determined as the data page containing the most intersections and the CRC check formula assignment procedure is continued until the assignment matches. The backup database determines the backup data page with the matching assignment and replaces the backup data page with the backup data page to repair the damaged data page to be repaired.

[0041] The corruption features that are not easy to identify in data pages include the unused space in the page; the page directory, the relative position of some records in the page, and the size of the space they occupy is uncertain;

[0042] In a specific embodiment, when the CRC check formula is assigned to repeatedly check more than two backup data pages for two times, a non-intersection subset in the backup data page is identified, and the backup data page with no identified damage features is selected as the metadata page. The damaged data page and the undamaged data page in the metadata page are backed up together using the backup database, thereby preventing recognition errors in the intersection recognition process, ensuring the success rate of the intersection recognition of the metadata feature set and the feature set of the backup data page, and reducing the number of CRC checks.

[0043] The standby database can back up the undamaged data pages that were initially transferred;

[0044] Specifically, the standby database performs data backup through the operation mechanism of the primary node database, generates a unique standby database control file by establishing a backup file of the primary database, and uses the archived log files and backup files of the primary database to rebuild the primary database on the standby database. When the standby database is in recovery mode, the standby database is not activated. Once the primary database is down, the standby database is activated to become the primary database so that the system can continue to run. This embodiment uses the standby database to back up damaged data pages and undamaged data pages in the metadata page, thereby preventing recognition errors in the intersection recognition process, ensuring the intersection recognition success rate of the metadata feature set and the feature set of the backup data page, and thereby reducing the number of CRC checks.

[0045] The working principle of the present invention is as follows: the value of the data page of the data to be detected is obtained by the discrimination module, and compared with the value of the metadata page. If they are different, the feature extraction module performs feature extraction on the data page in the data to be detected, generates a feature set to be detected, extracts the intersection of the feature set to be detected and the metadata feature set, and selects a backup data page in the standby database that includes all the features of the intersection through the intersection, and the backup data page is the metadata page;

[0046] The value assigned to the data page of the data to be detected is compared with the value assigned to the metadata page. If they are the same, it is determined that the backup data page in the data to be detected is a metadata page;

[0047] If there are multiple data pages in the standby database selected by the intersection that include all the features of the intersection, the values ​​are obtained respectively through CRC check, and compared with the values ​​of the metadata page through the discrimination module, and the same ones are the metadata pages; all the data pages in the standby database are extracted, and the feature parts of each data page are extracted respectively through the feature extraction module to generate several feature sets, the feature sets correspond to the data pages one by one, and all the feature sets are integrated to generate an index table; the index table is generated based on the integration of all the feature sets, so as to facilitate the CRC check formula assignment procedure according to the progressive relationship of the arrangement order until the assignment matches, thereby ensuring the correctness and integrity of the data page check;

[0048] The present invention uses a standby database to back up damaged data pages and undamaged data pages in metadata pages, thereby preventing recognition errors during intersection recognition, ensuring the intersection recognition success rate of the metadata feature set and the feature set of the backup data page, and reducing the number of CRC checks.

[0049] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0050] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A data page CRC check system, characterized in that: include: Backup database, which stores data pages internally; A sending end, used for sending a data page, wherein the data page is defined as a metadata page; A feature extraction module is used to extract the feature part of the metadata page and generate a metadata feature set; extract all the data pages in the backup database, and extract the feature part of each data page respectively through the feature extraction module to generate a number of feature sets, the feature sets correspond to the data pages one by one, integrate all the feature sets to generate an index table, select the target feature set through the index table, extract the non-intersection subset of the feature set to be detected and the metadata feature set through the feature extraction module, define it as a damage feature, integrate the damage feature with the metadata page and the metadata feature set into the backup database, and mark the metadata page corresponding to the target feature set as a damaged data page; The discrimination module, the backup database arranges the number of intersection sets containing the most target feature sets in descending order according to the number of intersections. When the value assigned to the backup data page containing the most intersections by the CRC check formula does not match the value assigned to the sending data page, the data page in the next order of arrangement is determined as the backup data page containing the most intersections and the CRC check formula assignment procedure is continued until the assignment matches. The backup database determines the backup data page with the matching assignment and replaces the backup data page with the damaged data page to be repaired. Through the CRC check, the assignment of the metadata page is obtained, and it is integrated with the metadata page and the metadata feature set into the initial sending data. When the CRC check formula assignment is repeated for more than 2 backup data pages, the non-intersection subset in the backup data page is identified, and the backup data page without the identified damaged feature is selected as the metadata page; At the receiving end, the initially sent data arrives at the receiving end via the data transmission channel. At this time, the data received by the receiving end is defined as the data to be detected. The assignment of the data page in the data to be detected is obtained through the discrimination module, and compared with the assignment of the metadata page. If they are different, the feature extraction module is used to extract features of the data page in the data to be detected, generate a feature set to be detected, extract the intersection of the feature set to be detected and the metadata feature set, and select the backup data page in the backup database that includes all the features of the intersection through the intersection. The backup data page is the metadata page.

2. A data page CRC check system according to claim 1, characterized in that: The value assigned to the data page of the data to be detected is compared with the value assigned to the metadata page. If they are the same, it is determined that the backup data page in the data to be detected is a metadata page.

3. A data page CRC check system according to claim 1, characterized in that: If there are multiple data pages in the backup database selected by intersection that include all the features of the intersection, the values ​​are obtained by CRC check respectively, and compared with the values ​​of the metadata page through the discrimination module, and the identical ones are the metadata pages.

4. A data page CRC check system according to claim 1, characterized in that: In the process of selecting the data page including all the features of the intersection in the backup database through the intersection, the target feature set is selected through the index table, the intersection is a subset of the target feature set, and the backup data page corresponding to the target feature set is extracted, and the backup data page is the metadata page.

Images