Data reliability auditing method and device, electronic equipment and storage medium

Abstract

The invention discloses a data reliability auditing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring audited data in a scientific research work process; wherein the data comprises a data file, a data directory and data stored in a database; monitoring the data to generate monitoring data; and analyzing the monitoring data, and if the analysis shows that the reliability of the monitored data has a problem, giving an alarm. According to the technical scheme, whether the scientific research data is abnormally operated or modified can be detected in time, so that the reliability of the scientific research data is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data auditing, and in particular, to a method and device for auditing data reliability, an electronic device, and a storage medium. Background Art

[0002] Daily scientific research activities are always accompanied by the generation, processing, and application of a large amount of data. Currently, the data generated or processed in scientific research activities is usually stored and managed in the form of computer files or databases. On the one hand, a large amount of data is obtained through the use of various measuring instruments in scientific research activities. In some fields, there are also robot scientists who can conduct a large number of experiments in parallel, generating high-throughput data. On the other hand, in recent years, with the transformation of the scientific research paradigm, computer simulation and emulation have become a common means in many scientific research fields, and a large amount of data will also be generated. Processing and analyzing the above experimental data or simulation data will obtain a large amount of secondary data, such as graphs, tables, animations, videos, etc. Currently, most scientific research activities directly or indirectly revolve around various types of data. It can be seen from this that the reliability of data is a very important and non-negligible issue in scientific research activities. Data reliability includes data accuracy, integrity, and availability. Data accuracy means that the data content is accurate and error-free, without any difference from the content at the time of its generation. Data integrity means that no part of the data file is lost. Data availability means that the expected data can be obtained at the expected location. It goes without saying that data reliability is one of the most basic requirements in scientific research activities, and it is a prerequisite for preventing academic misconduct and ensuring the correctness and repeatability of scientific research results. It is necessary to adopt technical means to ensure the data reliability in scientific research activities.

[0003] In scientific research activities, the full life cycle of data includes stages such as data generation, data storage, data analysis, and data application. As Figure 1 shown, data should be audited at each stage to ensure data reliability. The characteristics of the data generated during the scientific research process include being massive, heterogeneous, multi-source, having a complex directory structure, and multiple storage methods. The data auditing system needs to be developed according to these characteristics. The goal of the system is to target the full life cycle of data in scientific research activities. Once the data content is tampered with accidentally or intentionally, or abnormal operations are performed on data files and directories at the computer file level, it can be detected.

[0004] After retrieval, among the published domestic and foreign papers, there are researches on preventing web page files in Web (World Wide Web) servers from being tampered with, and researches on using hash values to digitally sign air detection result files to prevent the files from being tampered with, etc. There is no report on the research of data reliability in the process of scientific research work. After retrieval, among the publicly available domestic and foreign patents, there are many patents related to data auditing, but quite a lot of them are aimed at the data auditing and data traceability problems of relational databases. Select patents that have a certain relevance to this application, such as CN113158229A, CN110928930A, CN111404888A, CN113010494A, CN202111224933, CN202210157609, CN201910196358, CN201811235171, CN201910207627, CN202010421601, CN201610560815, US7200777B1, US10360203B2, US2005097149A1. There is no patent related to the data reliability audit of the entire life cycle of data in the process of scientific research work, and there is only an algorithm for encrypting files using hash values to verify data authenticity. Summary of the Invention

[0005] The present invention provides a data reliability audit method, device, electronic device and storage medium, which can timely detect whether there are abnormal operations or modifications to scientific research data, thereby ensuring the reliability of scientific research data.

[0006] According to one aspect of the present invention, there is provided a data reliability audit method, the method comprising:

[0007] Obtain the data to be audited in the process of scientific research work; wherein, the data includes data files, data directories and data stored in a database;

[0008] Monitor the data to generate monitoring data;

[0009] Analyze the monitoring data, and if the analysis shows that there is a problem with the reliability of the monitored data, then give an alarm.

[0010] According to another aspect of the present invention, there is provided a data reliability audit device, the device comprising:

[0011] A data acquisition unit for acquiring the data to be audited in the process of scientific research work; wherein, the data includes data files, data directories and data stored in a database;

[0012] A monitoring data generation unit for monitoring the data to generate monitoring data;

[0013] A reliability analysis unit for analyzing the monitored data and alarming if the analysis indicates a problem with the reliability of the monitored data.

[0014] According to another aspect of the present invention, there is provided an electronic device, which includes:

[0015] At least one processor; and

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute a data reliability auditing method according to any embodiment of the present invention.

[0018] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement a data reliability auditing method according to any embodiment of the present invention when executed.

[0019] The technical solution of the embodiment of the present invention obtains the data to be audited during the scientific research process, then monitors the data to generate monitored data, and analyzes the monitored data. If the analysis indicates a problem with the reliability of the monitored data, an alarm is given. This technical solution can timely detect whether there are abnormal operations or modifications in scientific research data, thereby ensuring the reliability of scientific research data.

[0020] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0022] Figure 1 is a schematic diagram of the full life cycle of data in scientific research activities provided by an embodiment of the present application;

[0023] Figure 2 is a flowchart of a data reliability auditing method according to Embodiment 1 of the present invention;

[0024] Figure 3 It is a data reliability audit framework diagram provided in the first embodiment of the present application;

[0025] Figure 4 It is a flowchart of the initial stage of file event monitoring provided in the first embodiment of the present application;

[0026] Figure 5 It is a flowchart of the inspection stage of file event monitoring provided in the first embodiment of the present application;

[0027] Figure 6 It is a flowchart of the initial stage of data fingerprint provided in the first embodiment of the present application;

[0028] Figure 7 It is a flowchart of the inspection stage of data fingerprint provided in the first embodiment of the present application;

[0029] Figure 8 It is a flowchart of the data reliability audit method for measurement results and simulation results provided in the second embodiment of the present invention;

[0030] Figure 9 It is a flowchart of the data reliability audit method for thesis data with simulation calculation and data analysis as the research content provided in the third embodiment of the present invention;

[0031] Figure 10 It is a schematic structural diagram of a data reliability audit device provided in the fourth embodiment of the present invention;

[0032] Figure 11 It is a schematic structural diagram of an electronic device for implementing a data reliability audit method provided in the fifth embodiment of the present invention. Detailed implementation manners

[0033] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] It should be noted that the term "target" and the like in the description, claims and the above drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 2 It is a flowchart of a data reliability auditing method provided according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of auditing data reliability. This method can be executed by a data reliability auditing device, which can be implemented in the form of hardware and / or software, and the data reliability auditing device can be configured in a device. For example, the device can be a device with communication and computing capabilities such as a background server. As Figure 2 shown, the method includes:

[0037] S210. Obtain the data to be audited during the scientific research process; wherein, the data includes data files, data directories, and data stored in a database.

[0038] In this solution, data is one of the core elements of scientific research. It refers to the original data generated during the scientific research process, as well as various data derived from processing and analyzing the original data. The reliability of these data is very important for scientific research, which is reflected in ensuring the correctness of scientific research results, the repeatability of scientific research processes, and preventing scientific misconduct in various aspects. Among them, the data sources during the scientific research process are relatively diverse, including those obtained through experimental measurements, computer calculations, manual or computer program analysis and processing of data, etc. Currently, these data are usually stored in readable and writable computer storage media (such as hard disks, disk arrays, etc.) and stored in the form of computer files or databases. The data types during the scientific research process are diverse and heterogeneous, including structured data and unstructured data, generally editable text data, and non-editable binary data, etc.

[0039] Furthermore, the reliability of data first means that during the entire scientific research process, all data can maintain the state when it was originally generated, and its content has not been modified intentionally or unintentionally; secondly, it is ensured that the attributes of the data file have not been modified intentionally or unintentionally, where the data file attributes refer to the author of the data file, the generation time, the data format, etc.; finally, the availability of the data is guaranteed, that is, the correct storage path of the data can be known, and the correct data can be obtained at any time on this path.

[0040] In this embodiment, the data reliability audit covers the entire life cycle of data during the scientific research process, including the data generation stage, the data storage stage, the data analysis stage, and the data application stage.

[0041] In this solution, the audited data during the scientific research process can be obtained through data acquisition technology. For example, the audited data during the scientific research process can be read from the database.

[0042] S220. Monitor the data to generate monitoring data.

[0043] In this solution, Figure 3 is the data reliability audit framework diagram provided in the first embodiment of this application. As Figure 3 shown, the data reliability audit consists of a data monitoring module, an audit database module, a reliability analysis module, and an alarm module. Based on the data monitoring module, the data is monitored to generate monitoring data.

[0044] S230. Analyze the monitoring data. If the analysis shows that there is a problem with the reliability of the monitored data, an alarm is issued.

[0045] In this embodiment, as Figure 3 shown, based on the reliability analysis module, the monitoring data is analyzed. If the analysis shows that there is a problem with the reliability of the monitored data, an alarm is issued based on the alarm module.

[0046] Optionally, the data reliability audit consists of a data monitoring module, an audit database module, a reliability analysis module, and an alarm module;

[0047] Among them, the data monitoring module is used to monitor data files, data directories, and data stored in the database, generate monitoring data, and store the monitoring data in the audit database module; the monitoring data includes data file operation logs, data directory operation logs, and various data fingerprints;

[0048] The audit database module is used to store and manage various monitoring data during the data reliability audit process; the various monitoring data includes data file operation logs, data directory operation logs, and various data fingerprints;

[0049] The reliability analysis module is used to analyze the monitoring data in the audit database module regularly or irregularly, in real time or non-real time, and determine whether there are problems with the reliability of the monitored data.

[0050] The alarm module is used to give an alarm when there are problems with the reliability of the monitored data. The alarm methods include sending alarm text messages or emails to users or managers, popping up an alarm window on the computer terminal, and making the computer emit an alarm sound.

[0051] In this solution, the data monitoring module monitors data files, data directories, or data stored in the database, generates monitoring data such as operation logs of data files and data directories, and various data fingerprints, and then transfers and stores them in the audit database module. The audit database module stores and manages various monitoring data in the data reliability audit process, including data file operation logs, data directory operation logs, and various data fingerprints. The reliability analysis module analyzes the monitoring data in the audit database module regularly or irregularly, in real time or non-real time, and determines whether there are problems with the reliability of the monitored data files, data directories, or data stored in the database. According to the results of the reliability analysis module, if there are problems with the reliability of the monitored data files, data directories, or data stored in the database, it is transferred to the alarm module for alarm. The alarm methods include but are not limited to sending alarm text messages or emails to users or managers, popping up an alarm window on the computer terminal, and making the computer emit an alarm sound, etc.

[0052] Aiming at the characteristics of the data generated in the scientific research process, such as massive volume, heterogeneity, multiple sources, complex directory structure, and multiple storage methods, it can timely detect whether there are abnormal operations or modifications to the scientific research data, so as to ensure the reliability of the scientific research data.

[0053] Optionally, monitoring the data includes:

[0054] For abnormal operations on the data files and data directories, file event monitoring technology is used for monitoring; and for modifications to the data files stored in the computer file system and the data stored in the database, data fingerprint technology is used for monitoring.

[0055] In this solution, data monitoring is the core technology and is implemented using two technologies. For abnormal operations on data files or data directories, file event monitoring technology is used; for modifications to data files and data stored in the database, data fingerprint technology is used. In specific application scenarios, both technologies can be used simultaneously or only one of them can be used to complete data monitoring.

[0056] Optionally, the monitoring process includes an initialization phase and an inspection phase;

[0057] Correspondingly, using file event monitoring technology for monitoring includes:

[0058] In the initialization phase, a detailed list of the data files and data directories is provided to the data monitoring module and stored in the monitoring target data table of the audit database module;

[0059] In the inspection phase, the data monitoring module reads the detailed list of the data files and data directories from the audit database module, and monitors the data files and data directories according to the detailed list. If a file operation occurs on the data files and data directories, the operation log information of the file operation event is stored in the audit database module for use by the reliability analysis module; wherein, the file operations include opening, editing, renaming, deleting, moving, and changing attributes; the operation log information includes the operation occurrence time, operation object, operation type, and the user who performs the operation;

[0060] Based on the reliability analysis module reading the operation log information from the audit database module regularly or irregularly, in real time or non-real time, for analysis, if the file operation event belongs to a pre-determined operation event, no action will be taken; if the file operation event does not belong to a pre-determined operation event, the file anomaly information is recorded in the audit database module and the alarm module is submitted for alarm.

[0061] In this solution, Figure 4 is the flowchart of the initialization phase of file event monitoring provided in Embodiment 1 of the present application. As Figure 4 shown, for file event monitoring, in the initialization phase, a detailed list of the data files or data directories to be monitored is provided to the data monitoring module and stored in the monitoring target data table of the audit database module.

[0062] Furthermore, Figure 5 is the flowchart of the inspection phase of file event monitoring provided in Embodiment 1 of the present application. As Figure 5As shown, after initialization is completed, the system enters the inspection phase. The data monitoring module reads the detailed list of data files or data directories to be monitored from the audit database module, and continuously monitors these data files or data directories. If any file operation event occurs on the data files or data directories, including but not limited to opening, editing, renaming, deleting, moving, changing attributes, etc., the data monitoring module will detect it and store the operation log information such as the time of the file operation event, the operation object, the operation type, and the user who performed the operation into the audit database module. The reliability analysis module periodically or aperiodically, in real-time or non-real-time, retrieves the operation log information from the audit database module for analysis to determine the nature of the file operation event. If the file operation event belongs to the pre-determined operation event, no action will be taken; if the file operation event does not belong to the pre-determined operation event, the reliability analysis module will record the file anomaly information into the audit database module and submit it to the alarm module to trigger an alarm.

[0063] Optionally, data fingerprint technology is used for monitoring, including:

[0064] In the initialization phase, the data monitoring module calculates the data fingerprint of the data file or the data stored in the database, and stores the data fingerprint and the full path of the data file or the database address of the data stored in the database into the data fingerprint data table in the audit database module; wherein, the data fingerprint is generated using MD5, SHA-2 series hash algorithms; the database address is composed of the database data table name and the index value in the data table;

[0065] In the inspection phase, based on the data fingerprint data table in the audit database module, obtain the full path of the data file or the database address of the data stored in the database, recalculate the data fingerprint of the data file or the data stored in the database to obtain the target data fingerprint, and compare the target data fingerprint with the data fingerprint stored in the audit database module. If the target data fingerprint is the same as the data fingerprint, no action will be taken; if the target data fingerprint is different from the data fingerprint, based on the reliability analysis module, record the file anomaly information into the audit database module and submit it to the alarm module for alarm;

[0066] Among them, the comparison between the target data fingerprint and the data fingerprint is based on binary data and is performed byte by byte; when all bytes of the target data fingerprint are exactly the same as the data fingerprint, the target data fingerprint is the same as the data fingerprint.

[0067] In this solution, Figure 6 is the flowchart of the data fingerprint initialization phase provided in the first embodiment of the present application, as Figure 6As shown in the figure, for data fingerprints, in the initialization phase, calculate the data fingerprints of the data files to be monitored or the data stored in the database, and store the data fingerprints together with the full path of the source data files or the database addresses of the source data in the data fingerprint data table of the audit database module. The data fingerprints can be generated using hash algorithms such as MD5, SHA-2 series (including SHA-224, SHA-256, SHA-384, SHA-512), etc. The database address is composed of the database data table name and the index value in the data table;

[0068] Furthermore, Figure 7 is the flowchart of the data fingerprint check phase provided in the first embodiment of this application. As Figure 7 shown in the figure, in the check phase, the reliability analysis module checks the data fingerprints regularly or irregularly, in real time or non-real time. The checking method is to first obtain the data files to be monitored or the data stored in the database based on the data fingerprint data table of the audit database module, recalculate its data fingerprint, and compare it with the fingerprint data stored in the audit database. If the two are the same, it means that the content of the data file or the data stored in the database has not been modified, and no action will be taken; if the two are different, it means that the content of the data file or the data stored in the database has been modified. At this time, the reliability analysis module records the file movement information into the audit database module and submits it to the alarm module to trigger an alarm.

[0069] Specifically, the comparison of data fingerprints is based on binary data and is performed byte by byte. It is considered that the two are the same only when all bytes of the two data fingerprints are exactly the same.

[0070] Optionally, monitoring the data further includes:

[0071] Monitoring the file operations of the data files or data directories based on the data fingerprint technology; wherein, the data files include a data file, multiple data files in the same directory, all data files in one or more directories and all their subdirectories at all levels; the data directories include one or more directories and all their subdirectories at all levels.

[0072] Optionally, monitoring the file operations of the data files or data directories based on the data fingerprint technology includes:

[0073] In the initialization phase, use the commands of the computer operating system to transfer the attribute information of the data file or data directory to an attribute file, calculate the data fingerprint of the attribute file, and store the data fingerprint of the attribute file in the audit database module;

[0074] In the inspection stage, based on the reliability analysis module, obtain the attribute information of the data file or data directory, transfer the attribute information of the data file or data directory to the target attribute file, and calculate the data fingerprint of the target attribute file;

[0075] Compare the data fingerprint of the target attribute file with the data fingerprint of the attribute file. If the data fingerprint of the target attribute file is the same as the data fingerprint of the attribute file, no action will be taken; if the data fingerprint of the target attribute file is different from the data fingerprint of the attribute file, the reliability analysis module will record the file change information into the audit database module and submit it to the alarm module for alarm;

[0076] Among them, the attribute information includes file name, size, owner, read and write permissions, storage full path, creation time, and modification time.

[0077] Specifically, the data fingerprint technology can also monitor the file operations of a data file. In the initialization stage, use the commands of the computer operating system to transfer the attribute information of a data file to be monitored to an attribute file, calculate the data fingerprint of the attribute file and store it in the audit database module. When it is necessary to judge the data reliability, the reliability analysis module re-obtains the attribute information of the same data file and stores it in a new attribute file, calculates the data fingerprint of the new attribute file, and takes out the data fingerprint in the audit database module for comparison. If the two are the same, it means that the data file has not been abnormally operated, and no action will be taken; if the two are different, it means that the data file has been abnormally operated. At this time, the reliability analysis module will record the file change information into the audit database module and submit it to the alarm module to trigger an alarm. Among them, the file attributes include file name, size, owner, read and write permissions, storage full path, creation time, modification time, etc.

[0078] Furthermore, the data fingerprint technology can also monitor the file operations of multiple data files in the same directory. In the initialization stage, use the commands of the computer operating system to transfer the list of all attribute information of multiple files in the same directory to be monitored to an attribute file, calculate the data fingerprint of the attribute file and store it in the audit database module. When it is necessary to judge the data reliability, the reliability analysis module re-obtains the list of all attribute information of multiple data files in the same directory to be monitored and transfers it to a new attribute file, calculates the data fingerprint of the new attribute file, and extracts the data fingerprint in the audit database module for comparison. If the two are the same, it means that the multiple data files in the same directory to be monitored have not been abnormally operated, and no action will be taken; if the two are different, it means that the multiple data files in the same directory to be monitored have been abnormally operated. At this time, the reliability analysis module records the file movement information in the audit database module and submits it to the alarm module to trigger an alarm. Among them, the file attributes include file name, size, owner, read / write permission, full storage path, creation time, modification time, etc.

[0079] In this solution, the data fingerprint technology can also monitor the file operations of all data files in one or more directories and all their subdirectories at all levels. In the initialization stage, use the commands of the computer operating system to transfer the list of all attribute information of all data files in one or more directories and all their subdirectories at all levels to be monitored to an attribute file, calculate the data fingerprint of the attribute file and store it in the audit database module. When it is necessary to judge the data reliability, the reliability analysis module re-obtains the list of all attribute information of all data files in one or more directories and all their subdirectories at all levels to be monitored and transfers it to a new attribute file, calculates the data fingerprint of the new attribute file, and extracts the data fingerprint in the audit database module for comparison. If the two are the same, it means that all data files in one or more directories and all their subdirectories at all levels to be monitored, including all relevant directories, have not been abnormally operated, and no action will be taken; if the two are different, it means that the data files in one or more directories and their subdirectories at all levels to be monitored, including relevant directories, have been abnormally operated. At this time, the reliability analysis module records the file movement information in the audit database module and submits it to the alarm module to trigger an alarm. Among them, the file attributes include file name, size, owner, read / write permission, full storage path, creation time, modification time, etc.

[0080] Optionally, the data reliability audit is applicable to files stored on a computer disk or data stored in a database;

[0081] Among them, the file or data is obtained by means of commercial software, free software, open-source software, programming, scanning, recording, and video recording;

[0082] Among them, for the data generated by the open-source software, the process of the generated data is tracked using the log information internally generated during the operation of the open-source software, to ensure that the data is not modified during the internal transfer and processing within the software.

[0083] Optionally, tracking the process of the generated data using the log information internally generated during the operation of the open-source software includes:

[0084] Modify the open-source software to transfer or append the operation information of the open-source software on the data to an external log file in real time;

[0085] Define abnormal data operations when using the open-source software to process data in the audit database module;

[0086] When conducting a reliability audit on the data generated by the open-source software, compare the external log file of the open-source software according to the abnormal data operations. If an abnormal data operation is matched, record the abnormal data operation into the audit database module based on the reliability analysis module, and submit it to the alarm module for alarm.

[0087] Specifically, data reliability audit is applicable to files stored on a computer disk and data stored in a database, and there is no restriction on the source of the files or data. Whether the files or data are obtained using commercial software, free software, open-source software, or self-programming, or acquired through scanning, recording, video recording, etc., their reliability can be audited.

[0088] Furthermore, for the data generated by the open-source software, in addition to the above steps, the log information internally generated during its operation can also be used to track the process of the generated data, to ensure that the data is not modified during the internal transfer and processing within the open-source software. Specifically, modify the open-source software so that its operation information on the data can be dumped or appended to an external log file in real time; define what abnormal data operations are when using this open-source software to process data in the audit database module; when a reliability audit needs to be conducted on the data generated by the open-source software, compare the external log file of the open-source software according to the abnormal data operations defined in the above steps. If a matching abnormal data operation is found, the reliability analysis module records the abnormal data operation into the audit database module and submits it to the alarm module to trigger an alarm.

[0089] Optionally, the data reliability audit is applicable to various data files generated and derived during the scientific research process, and is also applicable to other types of files; among them, the other types of files include source code files, various configuration files, executable code files, and program library files;

[0090] Among them, the data reliability audit is applicable to the scientific research field, the engineering field, and the social field; the data reliability audit can be integrated or embedded into other systems.

[0091] Specifically, the data reliability audit is applicable not only to various types of narrow-sense data files generated and derived during the scientific research process, but also to all other types of files, such as source code files, various configuration files, executable code files, program library files, etc., provided that these files are stored on a computer storage medium in the form of computer files. The data reliability audit can be implemented not only in the scientific research field, but also in other fields, such as engineering, social, etc. The data reliability audit can also be used as a module and integrated or embedded into other systems, such as R & D management systems, office automation systems, enterprise resource planning systems, etc.

[0092] The technical solution of the embodiment of the present invention obtains the data to be audited during the scientific research process, then monitors the data, generates monitoring data, and analyzes the monitoring data. If the analysis shows that there is a problem with the reliability of the monitored data, an alarm is issued. By implementing this technical solution, in view of the characteristics of the data generated during the scientific research process, such as massive volume, heterogeneity, multiple sources, complex directory structures, and multiple storage methods, file event monitoring technology and data fingerprint technology are used to detect whether there are abnormal operations or modifications to the scientific research data in a timely manner, thereby ensuring the reliability of the scientific research data.

[0093] Embodiment 2

[0094] Figure 8 It is a flowchart of the data reliability audit method for the measurement results and simulation results provided by the second embodiment of the present invention. The relationship between this embodiment and the above-mentioned first embodiment is a detailed description of the data reliability audit process. As Figure 8 shown, the method includes:

[0095] S810. Calculate the data fingerprints of the measurement results and the simulation results.

[0096] In this solution, the measurement result is the state data of the experimental system measured by the experimental system at a specific location or moment, and the simulation result is the simulation instance obtained by the simulation system applying one or more models or simulation methods to run one or more experimental systems.

[0097] Among them, the experimental system can be composed of one or more experimental devices with different scales, structures, and functions, and each experimental device can operate in a coupled manner or independently. Specifically, in the experimental system of this embodiment, the experimental device is a gas-solid circulating fluidized bed, and the measurement result is the real-time data during the experiment collected by installing measurement equipment on the fluidized bed.

[0098] In this embodiment, the simulation object of the simulation system is the experimental system. Specifically, in the simulation system of this embodiment, the simulation instance is to perform real-time simulation of the gas-solid fluid flow in the fluidized bed of the experimental device using simulation software on a supercomputer platform, and the simulation result is the calculation result of the above simulation instance. The data reliability audit system for the measurement results and simulation results provided in this embodiment can be executed by a data reliability audit device, which can be implemented in the form of hardware and / or software, and the data reliability audit device can be configured in an electronic device.

[0099] Specifically, the measurement results are the data during the experimental process collected by the measurement devices on the fluidized bed, and the simulation results are the data generated by performing real-time simulation of the gas-solid fluid flow in the fluidized bed using simulation software. Both the measurement results and the simulation results are stored in the computer file system in the form of files, and the metadata describing the measurement results and the metadata describing the simulation results are stored in the metadata database. There is no limitation on the simulation software in this embodiment, and the simulation software can be commercial software, free software, open-source software, or self-developed simulation software.

[0100] Furthermore, the measurement results include flow rate, pressure difference, concentration distribution and other flow data, which are composed of the physical quantities collected by 33 data acquisition points. During the operation of the experimental device, the measurement data acquisition frequency is 1 time per second, and the collected measurement data is stored in the form of a file on the computer disk. The metadata describing the measurement results includes the data acquisition point number, the meaning of the physical quantity corresponding to the acquisition point, the value range of the physical quantity, the meaning of each field in the data file, the storage location of the data file, etc.

[0101] Specifically, the simulation results are the relevant data obtained by simulating the gas-solid fluid flow in the fluidized bed using simulation software, and the data is stored in the computer file system in the form of a file. The metadata describing the simulation results includes the simulation software used in the simulation calculation, the storage directory of the simulation result data, the start time of the simulation calculation, the end time of the simulation calculation, etc.

[0102] In this solution, the data reliability audit system reads the measurement results and simulation results from the computer file system, and reads the metadata information of the measurement results and simulation results from the metadata database. Use the commands of the computer operating system to transfer the list of all attribute information of the measurement result file to an attribute file A respectively, and transfer the list of all attribute information of the simulation result file to an attribute file B respectively. Calculate the data fingerprint of file A, and use the data fingerprint of file A as the data fingerprint of the measurement results. Calculate the data fingerprint of file B, and use the data fingerprint of file B as the data fingerprint of the simulation results.

[0103] Specifically, the data fingerprint is generated using the Message Digest Algorithm 5 (MD5). The MD5 algorithm processes the specified input data to generate a unique fixed-length 128-bit hash value, which is used to verify the correctness of the data. Because even a slight change in the input data will cause a significant change in the corresponding hash value.

[0104] Optionally, the data fingerprint can also be generated using algorithms such as the SHA-2 series (including SHA-224, SHA-256, SHA-384, SHA-512).

[0105] Optionally, all the data of the measurement result is directly used as the input data of the MD5 algorithm to calculate the data fingerprint of the measurement result, and all the data of the simulation result is directly used as the input data of the MD5 algorithm to calculate the data fingerprint of the simulation result.

[0106] Optionally, the measurement result is not stored in the computer file system but in a database. The data reliability audit system reads the measurement result data from the database and transfers it to a JSON file in key-value form, calculates the data fingerprint of the JSON file, and uses the data fingerprint of the JSON file as the data fingerprint of the measurement result.

[0107] The S820, data fingerprint, data index, and the agreed data operation type are written into the audit database.

[0108] Among them, the data index is used to access the measurement result and the simulation result; the data fingerprint is the calculated data fingerprint of the measurement result and the simulation result; the agreed file operation types for the measurement result and the simulation result are used to complete the reliability analysis. The audit database includes a monitoring target data table, a data fingerprint table, a file operation dictionary table, a file system operation log table, and a user information table.

[0109] Specifically, the measurement result is stored in the computer file system in the form of a file, and the full path of the measurement result in the file system is used as the data index of the measurement result. The simulation result is stored in the computer file system in the form of a file, and the full path of the simulation result in the file system is used as the data index of the simulation result.

[0110] Furthermore, the full path of the measurement result in the file system and the full path of the simulation result in the file system are written into the monitoring target data table of the audit database. The data fingerprint of the measurement result and the data fingerprint of the simulation result are written into the data fingerprint table of the audit database. The agreed file operation types for the measurement result and the simulation result are new, open, and copy, and the above operation types are written into the file operation dictionary table of the audit database to complete the reliability analysis.

[0111] In this embodiment, after the data fingerprint, data index, and the agreed file operation type are written into the audit database, the data monitoring initialization is completed.

[0112] Optionally, the measurement results are not stored in the computer file system, but in the database, and the combination of the data table name where the measurement results are located and their ID values in the data table is used as the data index.

[0113] S830. Monitoring of file operation events.

[0114] Among them, the monitoring of file operation events is to monitor any file operation events on the measurement results and simulation results, including but not limited to operations such as access, modification, and deletion.

[0115] Specifically, the measurement results and simulation results are stored in the computer file system in the form of files, and the monitoring of the measurement results and simulation results is realized through file event monitoring. File event monitoring first obtains the list of data files to be monitored based on the monitored target data table in the audit database, that is, the detailed list of the data files or data directories to be monitored, and continuously monitors these data files or data directories. If any operation events occur on the data files or data directories, including but not limited to opening, editing, renaming, deleting, moving, changing attributes, etc., the data monitoring system will detect them and store the log information such as the time when the file operation event occurs, the operation object, the operation type, and the user who performs the operation into the audit database.

[0116] Specifically, the file system event monitoring mechanism inotify provided by the Linux operating system is used to monitor file operations. The data reliability audit system registers the full paths of the measurement results and simulation results into the inotify monitoring list. Inotify continuously monitors the measurement results and simulation results. If any operation events occur on the above data files or data directories, including but not limited to opening, editing, renaming, deleting, moving, changing attributes, etc., inotify will detect them and store the log information such as the time when the operation event occurs, the operation object, the operation type, and the user who performs the operation into the file system operation log table of the audit database.

[0117] S840. Reliability analysis.

[0118] In this embodiment, the reliability analysis is to analyze the monitored data in the audit database regularly or irregularly, in real time or non-real time, and judge whether there are problems with the reliability of the monitored measurement result data or simulation result data by comparing the data fingerprint and reviewing the file operation events.

[0119] Specifically, the comparison of data fingerprints includes: First, read the monitored target data table in the audit database to obtain a data index; then, access the data file stored in the computer file system or the data stored in the database to be monitored according to the data index, recalculate its data fingerprint using the method described in S810 above, and compare it with the original data fingerprint previously calculated and stored in the audit database. If the two are the same, it indicates that the content of the data file stored in the computer file system or the data stored in the database has not been modified, and no action will be taken; if the two are different, it indicates that the content of the data file stored in the computer file system or the data stored in the database has been modified. At this time, record the file movement information in the audit database and submit an alarm request to trigger an alarm.

[0120] Furthermore, the comparison of data fingerprints is based on binary data and is performed byte by byte. It is considered that the two are the same only when all bytes of the two data fingerprints are exactly the same.

[0121] Specifically, review the file operation event, that is, review the log information of the user operation file written into the audit database. The data reliability audit system reads the file operation log from the file system operation log table in the audit database for analysis to determine the nature of the operation event. If the event belongs to the pre-agreed operation, no action will be taken; if the operation event does not belong to the pre-agreed operation, the data reliability audit system will record the file movement information in the audit database and submit an alarm request to trigger an alarm. Among them, the pre-agreed file operation events include opening, moving, and copying. The data reliability audit system traverses the operation dictionary table of the audit database, reads the pre-agreed legal operations, and when querying operation records including deletion, etc., since deletion does not belong to the pre-agreed legal file operation, an alarm request is submitted to trigger an alarm.

[0122] S850, Alarm.

[0123] In this solution, when the data content of the measurement result and the simulation result changes, or when an abnormal file operation event occurs, the data reliability audit system notifies the user or administrator in the form of an alarm.

[0124] Specifically, the ways of alarm include but are not limited to sending alarm text messages or emails to the user or administrator, popping up an alarm window on the computer terminal, making the computer emit an alarm sound, etc.

[0125] S860, Write the reliability analysis result into the audit database.

[0126] Among them, the reliability analysis result includes the time of reliability analysis, the object of reliability analysis, the result of reliability analysis, whether to alarm, etc.

[0127] Specifically, information such as the ID in the monitoring target data table of the audit database for the measurement results or simulation results of this reliability analysis, whether an alarm is issued, and the time when the reliability analysis occurred is written into the audit database.

[0128] Embodiment III

[0129] Figure 9 FIG. is a flowchart of a method for auditing the reliability of paper data with simulation calculation and data analysis as the research content provided by Embodiment III of the present invention. The relationship between this embodiment and Embodiment I above is a detailed description of the data reliability audit process. As Figure 9 shown, the method includes:

[0130] S910. Generate simulation data using simulation software.

[0131] In this solution, the audit of the reliability of paper data with simulation calculation and data analysis as the research content is applicable to the audit of the reliability of paper data with simulation calculation and data analysis as the research content and the continuous work links such as simulation calculation, data analysis, chart drawing, and insertion carried out for writing this paper. Specifically, it is to audit the data reliability of each link in the continuous work links. If the audit result of the data reliability of any link fails, the audit result of the paper data reliability fails.

[0132] In this embodiment, the simulation data includes simulation result data and simulation software configuration data. The simulation result data refers to the relevant data calculated by the simulation software; the simulation software configuration data refers to the data input and selected by the user when using the simulation software for simulation calculation, including device models, flow field settings, calculation nodes, or calculation time, etc. Among them, the simulation software can be free software, open-source software, commercial software, or simulation software developed independently.

[0133] Furthermore, the simulation data is stored in the computer file system in the form of files, and the metadata describing the simulation is stored in the metadata database. The metadata includes: simulation software name, simulation result data storage directory, simulation software configuration data storage directory, simulation start time, simulation end time, etc.

[0134] S920. Data monitoring.

[0135] Among them, data monitoring is to monitor any operation events on the simulation data, including but not limited to operations such as access, modification, movement, and deletion.

[0136] Specifically, the simulation data is stored in the computer file system in the form of files, and the monitoring of the simulation results is achieved through file event monitoring.

[0137] Further, before monitoring the simulated data, it is necessary to initialize the monitoring data, including writing the simulated data access index to the monitoring target data table in the audit database, writing the data fingerprint of the simulated data to the data fingerprint table in the audit database, and writing the audit rules to the audit rule data table in the audit database. The simulated data access index, the data fingerprint of the simulated data, and the audit rules are all used to complete the data reliability audit. The audit database includes a monitoring target data table, a data fingerprint table, a computer file system operation log table, a data relationship table, a user information table, a software operation log table, a research result table, a research result and chart relationship table, and an audit result table.

[0138] Among them, the full path of the simulated data in the computer file system is used as the access index of the simulated data. Specifically, the data monitoring system needs to read the full path of the simulated data from the computer file system, obtain a list of all files in the directory through the operating system command, and store the full paths of all files in the list into the monitoring target data table in the audit database.

[0139] Among them, the data fingerprint of the simulated data is generated using the MD5 (Message-Digest Algorithm 5) algorithm. The MD5 algorithm processes the specified input data to generate a unique fixed-length 128-bit hash value, which is used to verify the correctness of the data. Because even if the input data changes slightly, its corresponding hash value will change significantly. The data monitoring system calculates the data fingerprint of the simulated data and stores the data fingerprint together with the full path of its source data file into the data fingerprint data table in the audit database.

[0140] Specifically, the audit rules refer to the list of legal file system operations, the data fingerprint comparison granularity, the alarm method, etc. used when the reliability analysis system analyzes data reliability.

[0141] Further, after the monitoring data initialization is completed, the data monitoring system continuously monitors the files or directories in the monitoring target data table of the audit database. If any file operations on the data files or data directories occur, including but not limited to opening, editing, renaming, deleting, moving, changing attributes, etc., the data monitoring system will detect them and store the log information such as the time when the file operation event occurred, the operation object, the operation type, and the user who performed the operation into the file system operation log table in the audit database for reliability analysis.

[0142] Specifically, the inotify file system event monitoring mechanism provided by the Linux operating system is used to monitor file operations.

[0143] S930. Analyze the simulated data through data analysis software to generate analysis data.

[0144] Among them, the analysis data includes analysis result data and analysis software configuration data. The analysis result data refers to the relevant data obtained by analyzing the simulation result data through the data analysis software; the analysis software configuration data refers to the data input and selected by the user when using the data analysis software for data analysis. The analysis data is stored in the computer file system in the form of files, and the metadata describing the analysis is stored in the metadata database. The metadata includes: the name of the analysis software, the storage directory of the analysis result data, the storage directory of the analysis software configuration data, the start time of the analysis, the end time of the analysis, etc.

[0145] Specifically, the simulation data is imported into the data analysis software for analysis. After the data analysis is completed, the analysis result data file and its corresponding analysis configuration file are stored in the computer file system.

[0146] Among them, there is no restriction on the source of the analysis software. The analysis software can be commercial software, free software, open-source software, or self-developed data analysis software.

[0147] Optionally, for the data generated by open-source software or self-developed software, the internal log information generated during its operation can also be used to track the process of generating the data, so as to ensure that the data is not modified during the process of flowing and being processed within the software. If open-source software is used, first modify the open-source software so that the operation information of the data can be dumped or appended to an external log file in real time; if self-developed software is used, add the function of dumping or appending the operation information of the data to an external log file in real time when developing the software. Define in the audit database what is an abnormal data operation when using this open-source software or self-developed software to process data; finally, when it is necessary to audit the reliability of the data generated by the open-source software or self-developed software, compare the log files of the open-source software or self-developed software according to the abnormal data operations defined in the audit database. If a matching abnormal data operation is found, the reliability analysis system records the abnormal data operation in the audit database.

[0148] S940. Data monitoring and updating for the analysis data.

[0149] Specifically, after the analysis data is written into the computer file system and the metadata describing the analysis is written into the metadata database, the data index of the above analysis data is written into the monitoring target data table of the audit database to update the monitoring target, calculate the data fingerprint of the above analysis data and write it into the data fingerprint table of the audit database, and write the association relationship between the analysis data and the simulation data into the data relationship table of the audit database.

[0150] Further, after the operator closes the data analysis software, the data monitoring system saves the operation logs of the data analysis software and the operation logs of the simulation result data file directory into the audit database, and adds the storage path of the analysis data in the computer file system as a data index to the monitoring target data table of the audit database.

[0151] Specifically, the data fingerprint of the analysis data is generated using the MD5 algorithm.

[0152] Among them, the association relationship between the analysis data and the simulation data is realized by associating the IDs of the above analysis result data and simulation result data in the monitoring target data table of the audit database. First, determine the ID of the simulation data to be analyzed in the monitoring target data table of the audit database; then, write the ID of the analysis data in the monitoring target data table of the audit database and the ID of the simulation data in the monitoring target data table of the audit database into the data relationship table of the audit database.

[0153] S950. The user draws graphs and tables for the analysis data through the data processing software.

[0154] In this embodiment, the data processing software is used to analyze the analysis data, and then graph and table files are drawn. The graphs and tables are stored in the computer file system in the form of files, and the metadata describing the graphs and tables is stored in the metadata database. The metadata includes: the name of the data processing software, the storage directory of the software processing result data, the storage directory of the data processing software configuration data, the data processing start time, the data processing end time, etc.

[0155] Specifically, the analysis data directory is imported into the data processing software for processing. After the graphs and tables are generated, the graph and table files are stored in the computer file system, and the configuration files of the corresponding data processing software are stored in the computer file system.

[0156] S960. Data monitoring update for the graphs and tables.

[0157] Specifically, after the generated graph and table files are written into the computer file system and the metadata describing the graphs and tables is written into the metadata database, the indexes of the above graph and table files are written into the monitoring target data table of the audit database to update the monitoring target, calculate the data fingerprints of the above graph and table files and write them into the data fingerprint table of the audit database, and write the relationship between the graph and table files and the analysis data into the data relationship table.

[0158] Further, after the operator closes the data processing software, the data monitoring system saves the operation logs of the data processing software and the operation logs of the analysis result file directory into the audit database, and adds the storage path of the graph and table files in the computer file system as a data index to the monitoring target data table of the audit database.

[0159] Specifically, the data fingerprints of the figure and table files are generated using the MD5 algorithm.

[0160] Specifically, the association relationship between the figure and table files and the analysis data is realized by associating the IDs of the above analysis result data and the figure and table files in the monitoring target data table of the audit database. First, determine the ID of the analysis data for which figures and tables need to be drawn in the monitoring target data table of the audit database. Then, write the ID of the figure and table files in the monitoring target data table of the audit database and the ID of the analysis data for which figures and tables need to be drawn in the monitoring target data table of the audit database into the data relationship table of the audit database.

[0161] S970. Insert the generated figures and tables into the paper.

[0162] Among them, when inserting the figures and tables into the paper, the data monitoring system writes the relevant information of the paper into the research result table of the audit database.

[0163] Specifically, the data monitoring system adds the ID of the above figure and table files in the monitoring target table of the audit database and the ID of the paper in the research result table of the audit database to the research result and chart relationship table of the audit database for reliability auditing.

[0164] S980. Conduct reliability auditing on the paper data.

[0165] Specifically, as Figure 9 shown, the administrator initiates the auditing of the above paper data. The reliability analysis system finds the ID of the data related to the paper in the monitoring target data table of the audit database through the paper ID provided by the administrator, and reads the pre-set audit rules in the audit rule table of the audit database to perform reliability judgment on the data in the monitoring target data table of the audit database. The reliability judgment includes judging the nature of the operation events stored in the audit database and verifying the data fingerprints of the target data.

[0166] Optionally, the administrator can specify audit rules to adjust the audit strictness.

[0167] Specifically, the reliability analysis system finds the ID of the figure and table files related to the specified paper ID in the monitoring target data table of the audit database from the research result and chart relationship table of the audit database, finds the ID of the analysis data related to the above figure and table files in the monitoring target data table of the audit database from the data relationship table of the audit database, and finds the ID of the simulation data related to the above analysis data in the monitoring target data table of the audit database from the data relationship table of the audit database.

[0168] Specifically, for operation event auditing, the reliability analysis system obtains the event information corresponding to the monitoring target from the computer file system operation log table in the audit database. The reliability analysis system reads the action list from the audit rule table in the audit database. If the event belongs to the pre-agreed actions, no action will be taken. If the event does not belong to the pre-agreed actions, the reliability analysis system records the file change information in the audit database. Among them, the pre-agreed legal file operation events include creation, moving, and copying. When an operator deletes a file, the data monitoring system obtains this operation record and stores the record in the audit database. The reliability analysis system traverses the audit database. When it searches for the operation record of deleting a file, since the deletion operation does not belong to the pre-agreed legal file operations, the reliability analysis system submits an alarm request to the alarm system to trigger an alarm and stores the audit result in the audit result data table in the audit database. Among them, the monitoring target can refer to the figures, table files, analysis data, and simulation data related to the audited thesis found above.

[0169] Specifically, for data fingerprints, the reliability analysis system first obtains the list of monitored data files based on the monitoring target data table in the audit database, recalculates the data fingerprints of the above data files, and compares them with the original data fingerprints stored in the audit database. If the two are the same, it means that the target data file has not been modified and no action will be taken. If the two are different, it means that the data file has been modified. At this time, the reliability analysis system records the file change in the audit database, submits an alarm request to the alarm system to trigger an alarm, and stores the audit result in the audit result data table in the audit database.

[0170] Among them, the comparison of the above data fingerprints is based on binary data and is carried out byte by byte. All bytes of the two data fingerprints need to be exactly the same for them to be the same.

[0171] Optionally, the data fingerprint verification operation can be automatically performed at regular intervals, and the time interval can be specified by the system administrator or project leader.

[0172] Optionally, the data fingerprint verification operation can be automatically performed when the system computing resources are idle. For example, the system resource usage is obtained through the Linux system command top. If the system CPU resource usage rate is lower than 50%, the reliability analysis system performs the data fingerprint verification operation.

[0173] Optionally, the data fingerprint verification operation time can be specified immediately by the system administrator or project leader.

[0174] Optionally, the data monitoring system can use data fingerprints to monitor the file operations of a file.

[0175] Optionally, the data monitoring system can use data fingerprints to monitor the file operations of multiple files in the same directory file.

[0176] Optionally, the data monitoring system can use data fingerprints to monitor the file operations of all data files in the same directory file and all its subdirectories at all levels.

[0177] Specifically, if it is necessary to audit the reliability of a file, multiple files in the same directory file, or all data files in the same directory file and all its subdirectories at all levels using data fingerprints, during the data monitoring initialization phase, the file attribute information is stored in text file A, and then the data fingerprint of text file A is calculated and stored in the audit database. When it is necessary to judge the data reliability, the reliability analysis module stores the attribute information of the file to be audited in text file B again, then calculates the data fingerprint of text file B and compares it with the data fingerprint of text file A stored in the audit database. If the two are the same, it means that the file to be audited has not been abnormally operated, and no action will be taken; if the two are different, it means that the file has been abnormally operated. At this time, the reliability analysis system records the file movement information in the audit database, submits an alarm request to the alarm system to trigger an alarm, and stores the audit result in the audit result data table in the audit database.

[0178] Among them, the alarm methods include, but are not limited to, sending alarm text messages or emails to users or system administrators, popping up an alarm window on the computer terminal, making the computer emit an alarm sound, etc.

[0179] Specifically, the administrator designates that the audit rules for simulated data and analysis data are not modifiable and not deletable, and the audit rules for graph and table files are not modifiable and deletable, and writes the above audit rules into the audit rule table in the audit database. When the user adds data to a certain table file, at this time, the reliability analysis system detects that the file has been modified through data fingerprint comparison, and the audit fails; when the user deletes a generated image file, at this time, the reliability analysis module detects the deletion operation through the computer file system operation log, but since it conforms to the above audit rules, the audit passes.

[0180] Furthermore, if both the data fingerprint audit and the operation log audit results are passed, the reliability analysis system sends a message through the front-end interface, and the audit passes. If the data fingerprint audit or the operation log audit fails, the reliability analysis system sends a message through the front-end interface, and the audit fails.

[0181] Embodiment 4

[0182] Figure 10 It is a schematic structural diagram of a data reliability audit device provided by Embodiment 4 of the present invention. As Figure 10 shown, the device includes:

[0183] A data acquisition unit 1010, configured to acquire the data to be audited during the scientific research process; wherein, the data includes data files, data directories, and data stored in a database;

[0184] A monitoring data generation unit 1020, configured to monitor the data and generate monitoring data;

[0185] A reliability analysis unit 1030, configured to analyze the monitoring data, and if the analysis shows that there is a problem with the reliability of the monitored data, then an alarm is issued.

[0186] Optionally, the data reliability audit covers the entire life cycle of data during the scientific research process, including the data generation stage, the data storage stage, the data analysis stage, and the data application stage;

[0187] Wherein, the data refers to various measurement data and calculation result data directly obtained during the scientific research process, or data obtained after at least one analysis and processing of various measurement data and calculation result data; the data is stored in a readable and writable computer storage medium, and is stored in the form of computer files or databases; the types of the data include structured data and unstructured data, editable text data, and non-editable binary data; the sources of the data include those obtained through experimental measurements, computer calculations, manual operations, or the use of computer programs to analyze and process data;

[0188] Wherein, the data reliability means that the data can maintain its state when it is generated, the content has not been modified; the file attributes of the data have not been modified; the data is stored in the form of computer files, and its correct storage location can be known.

[0189] Optionally, the data reliability audit consists of a data monitoring module, an audit database module, a reliability analysis module, and an alarm module;

[0190] Wherein, the data monitoring module is configured to monitor data files, data directories, and data stored in a database, generate monitoring data, and store the monitoring data in the audit database module; the monitoring data includes data file operation logs, data directory operation logs, and various data fingerprints;

[0191] The audit database module is configured to store and manage various monitoring data during the data reliability audit process; the various monitoring data includes data file operation logs, data directory operation logs, and various data fingerprints;

[0192] The reliability analysis module is used to analyze the monitoring data in the audit database module regularly or irregularly, in real time or non-real time, and determine whether there are problems with the reliability of the monitored data;

[0193] The alarm module is used to give an alarm when there are problems with the reliability of the monitored data; the alarm methods include sending alarm text messages or emails to users or managers, popping up an alarm window on the computer terminal, and making the computer emit an alarm sound.

[0194] Optionally, the monitoring data generation unit 1020 is specifically used for:

[0195] Monitor the abnormal operations of the data files and data directories by using file event monitoring technology; and monitor the modification of the data files stored in the computer file system and the data stored in the database by using data fingerprint technology.

[0196] Optionally, the monitoring process includes an initialization stage and an inspection stage;

[0197] Correspondingly, the monitoring data generation unit 1020 is also used for:

[0198] In the initialization stage, provide the detailed list of the data files and data directories to the data monitoring module and store it in the monitoring target data table of the audit database module;

[0199] In the inspection stage, the data monitoring module reads the detailed list of the data files and data directories from the audit database module, monitors the data files and data directories according to the detailed list, and if file operations occur on the data files and data directories, store the operation log information of the file operation events into the audit database module for use by the reliability analysis module; where the file operations include opening, editing, renaming, deleting, moving, and changing attributes; the operation log information includes the operation occurrence time, operation object, operation type, and the user who performs the operation;

[0200] Based on the reliability analysis module reading and analyzing the operation log information from the audit database module regularly or irregularly, in real time or non-real time, if the file operation event belongs to a pre-determined operation event, no action will be taken; if the file operation event does not belong to a pre-determined operation event, record the file anomaly information into the audit database module and submit it to the alarm module for alarm.

[0201] Optionally, the monitoring data generation unit 1020 is also used for:

[0202] In the initialization stage, the data monitoring module calculates the data fingerprint of the data file or the data stored in the database, and stores the data fingerprint and the full path of the data file or the database address of the data stored in the database into the data fingerprint data table in the audit database module; wherein, the data fingerprint is generated by using MD5, SHA-2 series hash algorithms; the database address is composed of the database data table name and the index value in the data table;

[0203] In the inspection stage, based on the data fingerprint data table in the audit database module, obtain the full path of the data file or the database address of the data stored in the database, recalculate the data fingerprint of the data file or the data stored in the database to obtain the target data fingerprint, and compare the target data fingerprint with the data fingerprint stored in the audit database module. If the target data fingerprint is the same as the data fingerprint, no action will be taken; if the target data fingerprint is different from the data fingerprint, based on the reliability analysis module, record the file anomaly information into the audit database module and submit it to the alarm module for alarm;

[0204] Among them, the comparison between the target data fingerprint and the data fingerprint is based on binary data and byte by byte; when all bytes of the target data fingerprint and the data fingerprint are exactly the same, the target data fingerprint is the same as the data fingerprint.

[0205] Optionally, the monitoring data generation unit 1020 is further configured to:

[0206] Monitor the abnormal operations of the data file or data directory based on the data fingerprint technology; wherein, the data file includes a data file, multiple data files in the same directory, all data files in one or more directories and all their subdirectories at all levels.

[0207] Optionally, the monitoring data generation unit 1020 is further configured to:

[0208] In the initialization stage, use the commands of the computer operating system to transfer the attribute information of the data file or data directory to an attribute file, calculate the data fingerprint of the attribute file, and store the data fingerprint of the attribute file into the audit database module;

[0209] In the inspection stage, based on the reliability analysis module, obtain the attribute information of the data file or data directory, transfer the attribute information of the data file or data directory to a target attribute file, and calculate the data fingerprint of the target attribute file;

[0210] Compare the data fingerprint of the target attribute file with that of the attribute file. If they are the same, no action will be taken. If they are different, the reliability analysis module will record the file change information into the audit database module and submit it to the alarm module for alarm.

[0211] Among them, the attribute information includes file name, size, owner, read / write permission, full storage path, creation time, and modification time.

[0212] Optionally, the data reliability audit applies to files stored on a computer disk or data stored in a database.

[0213] Among them, the file or data is obtained by means of commercial software, free software, open-source software, programming, scanning, recording, and video recording.

[0214] Among them, for the data generated by the open-source software, the internal log information generated when the open-source software operates is used to track the process of the generated data to ensure that the data is not modified during the process of flowing and being processed within the software.

[0215] Optionally, using the internal log information generated when the open-source software operates to track the process of the generated data includes:

[0216] Modify the open-source software to save or append the operation information of the open-source software on the data to an external log file in real time.

[0217] Define abnormal data operations when using the open-source software to process data in the audit database module.

[0218] When conducting a reliability audit on the data generated by the open-source software, compare the external log file of the open-source software according to the abnormal data operations. If an abnormal data operation is matched, based on the reliability analysis module, record the abnormal data operation into the audit database module and submit it to the alarm module for alarm.

[0219] Optionally, the data reliability audit applies to various data files generated and derived during the scientific research process, and also applies to other types of files; among them, the other types of files include source code files, various configuration files, executable code files, and library files.

[0220] Among them, the data reliability audit applies to the scientific research field, the engineering field, and the social field; the data reliability audit can be integrated or embedded into other systems.

[0221] An apparatus for auditing data reliability provided by an embodiment of the present invention can execute a method for auditing data reliability provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution of the method.

[0222] Embodiment 5

[0223] Figure 11 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0224] As Figure 11 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0225] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0226] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a data reliability auditing method.

[0227] In some embodiments, a data reliability auditing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data reliability auditing method described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute a data reliability auditing method by any other suitable means (e.g., by means of firmware).

[0228] The various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special or general-purpose programmable processor, that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0229] The computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to the processors of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the computer programs are executed by the processors, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0230] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0231] For providing interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used for providing interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0232] The systems and techniques described herein can be implemented in a computing system that includes backend components (such as, for example, a data server), or a computing system that includes middleware components (such as, for example, an application server), or a computing system that includes frontend components (such as, for example, a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (such as, for example, a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0233] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0234] It should be understood that various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0235] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A data reliability auditing method, characterized in that Including: Obtaining the data to be audited during the scientific research process; wherein, the data includes data files, data directories, and data stored in a database; Monitoring the data to generate monitoring data; Analyzing the monitoring data, and if the analysis shows that there are problems with the reliability of the monitored data, an alarm is issued.

2. The method according to claim 1, characterized in that, The data reliability audit covers the entire life cycle of data during the scientific research process, including the data generation stage, data storage stage, data analysis stage, and data application stage; Wherein, the data refers to various measurement data and calculation result data directly obtained during the scientific research process, or data obtained after at least one analysis and processing of various measurement data and calculation result data; the data is stored in a readable and writable computer storage medium in the form of computer files or databases; the types of the data include structured data and unstructured data, editable text data, and non-editable binary data; the sources of the data include those obtained through experimental measurements, computer calculations, manual or computer program analysis and processing of data; Wherein, the data reliability means that the data can maintain its state when it is generated, its content has not been modified; the file attributes of the data have not been modified; the data is stored in the form of a computer file and its correct storage location can be known.

3. The method according to claim 1, wherein The data reliability audit consists of a data monitoring module, an audit database module, a reliability analysis module, and an alarm module; Wherein, the data monitoring module is used to monitor data files, data directories, and data stored in a database, generate monitoring data, and store the monitoring data in the audit database module; the monitoring data includes data file operation logs, data directory operation logs, and various data fingerprints; The audit database module is used to store and manage various monitoring data during the data reliability audit process; the various monitoring data includes data file operation logs, data directory operation logs, and various data fingerprints; The reliability analysis module is used to analyze the monitoring data in the audit database module regularly or irregularly, in real time or non-real time, and judge whether there are problems with the reliability of the monitored data; The alarm module is used to issue an alarm when there are problems with the reliability of the monitored data; the alarm methods include sending alarm text messages or emails to users or managers, popping up an alarm window on a computer terminal, and making the computer emit an alarm sound.

4. The method according to claim 3, characterized in that, Monitoring the data includes: Monitoring abnormal operations of the data files and data directories using file event monitoring technology; and monitoring the modification of data files stored in the computer file system and the data stored in the database using data fingerprint technology.

5. The method according to claim 4, characterized in that, The monitoring process includes an initialization stage and an inspection stage; Correspondingly, monitoring using file event monitoring technology includes: In the initialization stage, providing a detailed list of the data files and data directories to the data monitoring module and storing it in the monitoring target data table of the audit database module; In the inspection stage, the data monitoring module reads the detailed list of the data files and data directories from the audit database module, monitors the data files and data directories according to the detailed list, and if file operations occur on the data files and data directories, stores the operation log information of the file operation events into the audit database module for use by the reliability analysis module; wherein, the file operations include opening, editing, renaming, deleting, moving, and changing attributes; the operation log information includes the operation occurrence time, the operation object, the operation type, and the user who performs the operation. Based on the reliability analysis module periodically or aperiodically, in real-time or non-real-time, reads the operation log information from the audit database module for analysis. If the file operation event belongs to a pre-determined operation event, no action will be taken; if the file operation event does not belong to a pre-determined operation event, records the file anomaly information into the audit database module and submits it to the alarm module for alarming.

6. The method according to claim 5, characterized in that, The monitoring is carried out using data fingerprint technology, including: In the initialization stage, the data monitoring module calculates the data fingerprint of the data file or the data stored in the database, and stores the data fingerprint and the full path of the data file or the database address of the data stored in the database into the data fingerprint data table in the audit database module; wherein, the data fingerprint is generated using MD5, SHA-2 series hash algorithms; the database address is composed of the database data table name and the index value in the data table. In the inspection stage, based on the data fingerprint data table in the audit database module, obtains the full path of the data file or the database address of the data stored in the database, recalculates the data fingerprint of the data file or the data stored in the database to obtain the target data fingerprint, and compares the target data fingerprint with the data fingerprint stored in the audit database module. If the target data fingerprint is the same as the data fingerprint, no action will be taken; if the target data fingerprint is different from the data fingerprint, based on the reliability analysis module, records the file anomaly information into the audit database module and submits it to the alarm module for alarming. Among them, the comparison between the target data fingerprint and the data fingerprint is based on binary data and byte by byte; when all bytes of the target data fingerprint and the data fingerprint are exactly the same, the target data fingerprint is the same as the data fingerprint.

7. The method according to claim 4, wherein The monitoring of the data also includes: Monitoring the file operations of the data file or data directory based on data fingerprint technology; wherein, the data file includes a single data file, multiple data files in the same directory, all data files in one or more directories and all their subdirectories at all levels; the data directory includes one or more directories and all their subdirectories at all levels.

8. The method according to claim 7, wherein Monitoring the file operations of the data file or data directory based on data fingerprint technology, including: In the initialization stage, the attribute information of the data file or data directory is transferred to an attribute file using the commands of the computer operating system, the data fingerprint of the attribute file is calculated, and the data fingerprint of the attribute file is stored in the audit database module; In the inspection stage, based on the reliability analysis module, the attribute information of the data file or data directory is obtained, the attribute information of the data file or data directory is transferred to a target attribute file, and the data fingerprint of the target attribute file is calculated; The data fingerprint of the target attribute file is compared with the data fingerprint of the attribute file. If the data fingerprint of the target attribute file is the same as the data fingerprint of the attribute file, no action will be taken; if the data fingerprint of the target attribute file is different from the data fingerprint of the attribute file, the reliability analysis module will record the file change information in the audit database module and submit it to the alarm module for alarming; Among them, the attribute information includes file name, size, owner, read and write permissions, full storage path, creation time, and modification time.

9. The method according to claim 1, characterized in that, The data reliability audit is applicable to files stored on a computer disk or data stored in a database; Among them, the file or data is obtained by means of commercial software, free software, open-source software, programming, scanning, recording, and video recording; Among them, for the data generated by the open-source software, the internal log information generated during the operation of the open-source software is used to track the process of the generated data to ensure that the data is not modified during the process of flowing and being processed within the software.

10. The method according to claim 9, wherein Using the internal log information generated during the operation of the open-source software to track the process of the generated data includes: Modifying the open-source software to transfer or append the operation information of the open-source software on the data to an external log file in real time; Defining abnormal data operations when using the open-source software to process data in the audit database module; When performing a reliability audit on the data generated by the open-source software, the external log file of the open-source software is compared according to the abnormal data operations. If an abnormal data operation is matched, the abnormal data operation is recorded in the audit database module based on the reliability analysis module and submitted to the alarm module for alarming.

11. The method according to claim 1, wherein The data reliability audit is applicable to various data files generated and derived during the scientific research process, and is also applicable to other types of files; among them, the other types of files include source code files, various configuration files, executable code files, and library files; Among them, the data reliability audit is applicable to the scientific research field, the engineering field, and the social field; the data reliability audit can be integrated or embedded into other systems.

12. A data reliability auditing device, characterized in that, Including: A data acquisition unit for acquiring the data to be audited during the scientific research process; among them, the data includes data files, data directories, and data stored in a database; A monitoring data generation unit for monitoring the data and generating monitoring data; A reliability analysis unit for analyzing the monitoring data. If the analysis shows that there is a problem with the reliability of the monitored data, an alarm will be issued.

13. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, enables the at least one processor to execute a data reliability auditing method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for implementing a data reliability auditing method according to any one of claims 1-11 when executed by a processor.

Images