Data storage management method for long-period operation automatic test system

Abstract

A data storage management method of a long-period operation automatic test system comprises the following steps: S1, before a test is started, a system control computer performs data storage capability assessment, and assesses whether a storage space of a storage device meets a demand or not according to task characteristic parameters; s2, starting the test, performing storage real-time monitoring by the system control computer, during the execution period of a test task, after the test data of each test execute write-in operation and are stored in the storage space of the storage equipment, executing instruction interaction by the system control computer, and checking whether the test data are normally stored or not; and S3, the test is finished, the system controls the computer to carry out cloud backup, and data stored in the storage device is backed up to a cloud server. According to the test data of the long-period running automatic test system, the storage capability can be evaluated before the test task starts, the data storage monitoring in the test process and the data cloud backup after the test is completed can be realized, and the safety of the stored data is ensured.

Description

Technical Field

[0001] The invention relates to the field of measurement and control technology, and in particular to a data storage management method for a long-cycle running automatic test system. Background Art

[0002] With the development of computer technology, bus and interface technology, and software technology, the automation level and measurement and control capabilities of automatic test systems are gradually improving. As a production method that can replace manual labor and release labor, it has been increasingly used in defense, military, and civilian production.

[0003] The essence of the automatic test system is to use computer software to control the system hardware resources to perform test tasks according to the set process, and the test process is automatic without human intervention. The current application status of most automatic test systems is to automatically run for several hours or dozens of hours or even days without supervision, and automatically store the data generated during the test. Test data is usually used for later data statistical analysis and product troubleshooting. In order to ensure the integrity of test data, a reliable data storage method is particularly important. Summary of the invention

[0004] The purpose of the present invention is to provide a data storage management method for a long-period running automatic test system to ensure the integrity of stored test data.

[0005] In order to achieve the above-mentioned purpose, the present invention provides a data storage management method for a long-cycle automatic test system, comprising the following steps: S1. Before the test starts, the system controls a computer to evaluate the data storage capacity, and evaluates whether the storage space of the storage device meets the demand based on the task characteristic parameters; if yes, go to S2; if no, back up the original test data in the storage device to the local space, and clear the memory occupied by the storage device, and go to S2; S2. At the start of the test, the system controls a computer to perform real-time storage monitoring. During the execution of the test task, after the test data of each test is written and stored in the storage space of the storage device, the system controls the computer to execute instruction interaction to check whether the test data is stored normally, and at the same time check the usage of the storage space at fixed time intervals; S3. At the end of the test, the system controls the computer to perform cloud backup, and backs up the data stored in the storage device to the cloud server.

[0006] Preferably, in said S1, said data storage capacity evaluation comprises a three-layer computing system, namely, a filtering and extraction layer, a computing layer and an execution layer.

[0007] Preferably, the filtering and extraction layer filters and extracts data tables from the database, and quickly clusters multiple data tables, and then packages the table attributes, table contents and other data information of each data table according to the transmission protocol rules and transmits them to the computing layer; the computing layer analyzes and calculates the data throughput of each data table based on the data information packaged and sent by the filtering and extraction layer, and summarizes the calculation to obtain the data throughput of the entire test task, and then passes it to the execution layer; the execution layer obtains the current storage resource status of the storage device, and compares it with the data throughput of the entire test task packaged and sent by the computing layer, so as to evaluate the feasibility of completing the test task.

[0008] Preferably, if the remaining storage space of the current storage device does not meet the storage conditions of the test task data throughput, the storage data in the current storage device is backed up locally according to the data table, and the data backup is stored in the form of a file, and then the storage space of the storage device is cleared to complete the current test task; if the remaining storage capacity of the current storage device is greater than the test task data throughput and the redundant storage space meets the danger threshold, it is determined that the test task can be completed.

[0009] Preferably, the danger threshold is the storage space required to complete the 2-hour test task execution.

[0010] Preferably, in said S1, the data tables extracted from the database by the filtering and extraction layer include: a product information table, an equipment configuration table, an operation information table and a test data table.

[0011] Preferably, in S2, the real-time storage monitoring is performed during the execution of the test task, and a streaming feature data anomaly detection method is used to monitor the data storage status of the storage device in real time to determine whether the data can be stored normally.

[0012] Preferably, a fault processing lookup table is provided in the system control computer. When a fault occurs, a table lookup method is used to search for a corresponding processing method in the fault processing lookup table according to the error content in the feedback characteristic data to handle the current system fault.

[0013] Preferably, the cloud server is provided with an upload data node table, which stores the ID number of the most recently uploaded data table, and the table resume node number value is updated after each data upload, and the next resume node starts from the record number in the table.

[0014] Preferably, the cloud backup adopts polling-based node data retransmission, that is, when the polling interval expires, the data of this test task is triggered to be retransmitted to the cloud backup server at one time after the last upload termination sequence number, and the cloud backup server and local data storage table format remain consistent and are still stored in table form.

[0015] To sum up, compared with the prior art, the data storage management method of a long-period automatic test system provided by the present invention adopts a dual data backup method, which includes not only local backup of data, but also cloud backup of data; at the same time, the present invention is aimed at the test data of the long-period automatic test system, and can evaluate the storage capacity before the start of the test task, monitor the data storage during the test, and back up the data in the cloud after the test is completed, thereby ensuring the security of the stored data and having great practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flow chart of the data storage management method of the medium and long-term operation automatic test system of the present invention;

[0017] Figure 2 This is a schematic diagram of the data structure of the filtering and extraction layer in the present invention. DETAILED DESCRIPTION

[0018] The following is combined with Figures 1-2 , the present invention is further explained by describing a preferred specific embodiment in detail.

[0019] It should be noted that the drawings are in a very simplified form and use non-precise proportions. They are only used to conveniently and clearly assist in explaining the embodiments of the present invention, and are not used to limit the conditions for the implementation of the present invention. Therefore, they have no substantive technical significance. Any structural modification, change in proportional relationship or adjustment of size, without affecting the effects and purposes that can be achieved by the present invention, should still fall within the scope of the technical content disclosed by the present invention.

[0020] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only the elements explicitly listed, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0021] like Figure 1 As shown, the present invention provides a data storage management method for a long-term automatic test system, which specifically includes the following steps:

[0022] S1. Before the test begins, the system controls the computer to evaluate the data storage capacity and evaluates whether the storage space of the storage device meets the requirements based on the task characteristic parameters; if yes, go to S2; if no, back up the original test data in the storage device to the local space and clear the memory occupied by the storage device to ensure that the storage space meets the requirements of this task, then go to S2;

[0023] S2, the test starts, the system controls the computer to perform real-time storage monitoring, during the test task execution, after the test data of each test is written and stored in the storage space of the storage device, the system controls the computer to execute instruction interaction to check whether the test data is stored normally, and at the same time check the usage of the storage space at fixed time intervals;

[0024] S3. After the test is completed, the system controls the computer to perform cloud backup and back up the data stored in the storage device to the cloud server.

[0025] Furthermore, in the above S1, the data storage capacity evaluation includes a three-layer computing system, namely, a filtering and extraction layer, a computing layer, and an execution layer. The filtering and extraction layer quickly filters and extracts data tables from the database according to the task feature parameters, and quickly clusters the multiple data tables involved in the test task, and then uses the feature extraction method to package the table attributes, table contents and other data information of each data table according to the transmission protocol rules and transmits them to the computing layer; the computing layer analyzes and calculates the data throughput of each data table at different granularities for the data information packaged and sent by the filtering and extraction layer, and then summarizes and calculates the multi-source data to obtain the data throughput of the entire test task and passes it to the execution layer; the execution layer obtains the current storage resource status of the storage device, and compares it with the data throughput of the entire test task packaged and sent by the computing layer to evaluate the feasibility of completing the test task.

[0026] If the remaining storage space of the current storage device does not meet the storage conditions of the test task data throughput, the storage data in the current storage device is backed up locally according to the data table, and the data backup is stored in the form of a file, and then the storage space of the storage device is cleared to complete the current test task; if the remaining storage capacity of the current storage device is greater than the test task data throughput and the redundant storage space meets the danger threshold, it is considered that the test task can be completed. Among them, the danger threshold is determined according to the test task data throughput under actual conditions. In this embodiment, the danger threshold is the storage space required for the execution of a 2-hour test task.

[0027] Specifically, it can be understood that in the above S1, a test task usually corresponds to multiple data tables, and the data types of each column in the table are different, the data lengths are different, and the storage space occupied is different. During the execution of the test task, row data will be continuously written to the corresponding test data table, and the data generation rate of each test data table is different. Therefore, before the test task starts, it is first necessary to quickly filter and extract the data table information according to the test task attributes, find the multi-source table information involved in this test task, and quickly perform cluster analysis to extract the table attributes, data types, data volume and other information of each table.

[0028] Furthermore, in S1, the data tables extracted from the database by the filtering and extraction layer include: a product information table, an equipment configuration table, an operation information table and a test data table.

[0029] Specifically, the table attribute of the data table, that is, the row data generation rate of the data table, can be obtained to obtain the table code. For example, the product information table is a data table that records data only once at the beginning of the task, and only one row of data is recorded for each task; the equipment configuration table is a data table that records data only once at the beginning of the task, but the number of records varies according to the task attributes; the operation information table is recorded at the end of the task and has only one row of data; the test data table is a continuous storage record table, which maintains continuous and uninterrupted data recording at a time interval of 1s throughout the entire task duration.

[0030] Furthermore, the data table content information includes information such as the data type and data volume of the table data record content. If the data type stored in a column of a table is double, the table data occupies 8 bytes; if the data type stored in a column of a table is varchar(255), the table data occupies a maximum of 255 characters. The storage space occupied by a row of data is the sum of the storage space occupied by all table data in the row.

[0031] Specifically, the transmission protocol rules adopted by the filtering and extraction layer are as follows: Figure 2 As shown, data information is packaged and transmitted in the form of a data stream, and the data stream includes a table code of a data table, and several connected data types and the number of the data types.

[0032] Furthermore, the test task is started after the data storage capacity is evaluated. In S2, the real-time monitoring of storage is performed during the execution of the test task. It adopts a streaming feature data anomaly detection method to monitor the data storage status of the storage device in real time to determine whether the data can be stored normally.

[0033] Specifically, the streaming feature data anomaly detection method can interactively monitor the changes in the storage capacity of each data table and the data writing status through real-time instructions of the system control computer. Specifically, during the execution of the test task, the system control computer continuously and uninterruptedly sends query instructions at a certain time interval, and monitors the feature data feedback when writing data. Among them, the feature data is a data block composed of error code, error content and error status, etc. The feature data can be fed back to the system control computer as an interactive parameter, and the system control computer can analyze the system error status according to the received feature data.

[0034] It should be noted that a fault handling lookup table is set in the system control computer. When a fault occurs, the corresponding processing method can be searched in the fault handling lookup table according to the error content in the feedback feature data using the table lookup method, and the current system fault is processed according to the processing method. If the data write timeout occurs, the test task can be notified to stop through the global alarm variable; if a dangerous threshold is set in the fault handling lookup table, the dangerous threshold is set as the system alarm start condition. When the storage capacity is monitored to reach the dangerous threshold, a warning is sent through the global alarm variable, and a storage space scan is started every 1 minute until the storage space is exhausted to notify the test task to stop.

[0035] It is very important that if the long-cycle automatic test system stops testing due to a storage failure, the test data written into the storage device that has been tested needs to be uploaded to the cloud server for backup.

[0036] Furthermore, when the test task starts, the system control computer will generate a completion flag, and the completion flag is a false value 0 at this time; when the test task is completed or the long-cycle automatic test system has a storage failure and stops testing, the completion flag will change from a false value 0 to a true value 1; the completion flag can trigger instructions to extract the test data that has been written to the storage device and upload it to the cloud server for backup.

[0037] Among them, the cloud server is provided with an upload data node table, which can store the ID number of the data table uploaded most recently, and the table resume node number value will be updated after each data upload, and the next resume node will start from the record number in the table; wherein, the uploaded backup is uploaded in the format of the data table.

[0038] Among them, the cloud backup adopts the polling node data continuation method, that is, when the polling interval time is reached, the data of this test task is triggered to be transferred to the cloud backup server at one time after the last upload termination sequence number, and the cloud backup server and local data storage table format are consistent, and are still stored in table format. Specifically, before the data upload starts, the last upload node is queried from the log file, and when the upload starts, the data is transferred from the upload node and uploaded to the cloud backup server in the form of table data. The server data storage table format is consistent with the local, and the system automatically records the continuation node sequence number of each test table and writes it into the log file.

[0039] It can be understood that by adopting polling-type node data transmission, the data table involved in this test task can be uploaded, and it can start from the starting point of the data storage of the current test task, so as to ensure the integrity of the test data.

[0040] It can be understood that the present invention adopts a dual data backup method, which includes not only local backup of data, but also cloud backup of data; wherein, local backup is performed at the beginning of the task to evaluate whether the current storage space meets the requirements, and if not, the data is backed up locally; and cloud backup is performed at the end of the test task, when the completion mark triggers data extraction and upload to the cloud backup.

[0041] To sum up, the present invention provides a data storage management method for a long-period running automatic test system. For the test data of the long-period running automatic test system, it can evaluate the storage capacity before the test task starts, monitor the data storage during the test, and back up the data in the cloud after the test is completed, thereby ensuring the security of the stored data and having great practical value.

[0042] Although the content of the present invention has been described in detail through the above preferred embodiments, it should be appreciated that the above description should not be considered as a limitation of the present invention. After reading the above content, it will be apparent to those skilled in the art that various modifications and substitutions of the present invention will occur. Therefore, the protection scope of the present invention should be limited by the appended claims.

Claims

1. A data storage management method for a long-term automatic test system, characterized in that: The following steps are involved: S1. Before the test starts, the system controls the computer to evaluate the data storage capacity and evaluates whether the storage space of the storage device meets the requirements according to the task characteristic parameters; if yes, go to S2; if no, back up the original test data in the storage device to the local space and clear the memory occupied by the storage device, go to S2; S2, the test starts, the system controls the computer to perform real-time storage monitoring, during the test task execution, after the test data of each test is written and stored in the storage space of the storage device, the system controls the computer to execute instruction interaction to check whether the test data is stored normally, and at the same time check the usage of the storage space at fixed time intervals; S3. After the test is completed, the system controls the computer to perform cloud backup and back up the data stored in the storage device to the cloud server.

2. The data storage management method for a long-period automatic test system according to claim 1, characterized in that: In the above-mentioned S1, the data storage capacity evaluation includes a three-layer computing system, namely, a filtering and extraction layer, a computing layer, and an execution layer.

3. The data storage management method for a long-period automatic test system according to claim 2, characterized in that: The filtering and extraction layer filters and extracts data tables from the database, and quickly clusters multiple data tables, and then packages the table attributes, table contents and other data information of each data table according to the transmission protocol rules and transmits them to the computing layer; The computing layer analyzes the data information sent by the filtering and extraction layer to calculate the data throughput of each data table, and summarizes the calculation to obtain the data throughput of the entire test task, and then passes it to the execution layer; The execution layer obtains the current storage resource status of the storage device, and compares it with the data throughput of the entire test task packaged and sent by the calculation layer to evaluate the feasibility of completing the test task.

4. The data storage management method for a long-period automatic test system according to claim 3, characterized in that: If the remaining storage space of the current storage device does not meet the storage conditions of the test task data throughput, the storage data in the current storage device is backed up locally according to the data table, and the data backup is stored in the form of a file, and then the storage space of the storage device is cleared to complete the current test task; If the remaining storage capacity of the current storage device is greater than the test task data throughput and the redundant storage space meets the danger threshold, it is determined that the test task can be completed.

5. The data storage management method for the long-term operation automatic completion test system according to claim 4, characterized in that: The danger threshold is the storage space required to complete the 2-hour test task execution.

6. The data storage management method for a long-period automatic test system according to claim 1, characterized in that: In the above S1, the data tables extracted from the database by the filtering and extraction layer include: a product information table, an equipment configuration table, an operation information table and a test data table.

7. The data storage management method for a long-period automatic test system according to claim 1, characterized in that: In the above-mentioned S2, the real-time storage monitoring is performed during the execution of the test task, and a streaming feature data anomaly detection method is used to monitor the data storage status of the storage device in real time to determine whether the data can be stored normally.

8. The data storage management method for a long-period automatic test system according to claim 1, characterized in that: A fault handling lookup table is provided in the system control computer. When a fault occurs, a table lookup method is used to search for a corresponding processing method in the fault handling lookup table according to the error content in the feedback characteristic data to handle the current system fault.

9. The data storage management method for a long-period automatic test system according to claim 1, characterized in that: The cloud server is provided with an upload data node table, which stores the ID number of the most recently uploaded data table, and the table resume node number value is updated after each data upload, and the next resume node starts from the record number in the table.

10. The data storage management method for a long-period automatic test system according to claim 9, characterized in that: The cloud backup adopts polling-based node data transmission, that is, when the polling interval expires, the data of this test task is triggered to be transmitted to the cloud backup server at one time after the last upload termination sequence number, and the cloud backup server and local data storage table format are consistent and are still stored in table form.

Images