Storage data security management system adapted to software development

Abstract

The invention belongs to the field of data storage management, relates to a data processing technology, and is used for solving the problem that a storage mode cannot be allocated to software data by adopting a dynamic hierarchical discrimination basis in the prior art, in particular to a storage data security management system adaptive to software development. Comprising a hierarchical storage module, an access monitoring module, an optimization processing module and a fault monitoring module which are connected in sequence, the hierarchical storage module is used for performing hierarchical storage processing on the storage data of software development: marking the storage data generated in the software development process as a processing object, generating a processing period, and marking a storage mode of the processing object through the access frequency value; according to the method, the corresponding storage mode is matched for the storage data according to the access frequency of the storage data, so that the data with relatively high access frequency can be stored through the database, and the access efficiency is ensured by utilizing the characteristics of high read-write speed and low delay of the database.

Description

Technical Field

[0001] The present invention belongs to the field of data storage management and relates to data processing technology, in particular to a storage data security management system suitable for software development. Background Art

[0002] There are many ways to save software development data, including databases, local file systems, etc. Among them, databases are the most commonly used method because they provide structured data storage and efficient query functions. Databases are one of the main ways to store and manage data, and are particularly suitable for applications that require efficient query and data integrity.

[0003] The invention patent with announcement number CN117992031A discloses a software development management platform and management method based on data sharing, which divides the software development process into different stages. At the same time, software developers carry out development work in combination with operating procedures, and cooperate with data sharing to enable software developers to focus on development itself without being affected by work on data resources, thereby reducing repetitive data conversion and transmission work, improving software development efficiency, and reducing development costs; however, the management platform is unable to monitor the data access efficiency of the database. On the basis of adopting the "cold and hot data tiering" mechanism, it is impossible to use dynamic tiering judgment basis to allocate storage methods for software data, resulting in the data access efficiency of the in-memory database cannot be guaranteed.

[0004] In response to the above technical problems, this application proposes a solution. Summary of the Invention

[0005] The purpose of the present invention is to provide a storage data security management system suitable for software development, which is used to solve the problem that the existing technology cannot use a dynamic hierarchical judgment basis to allocate storage methods for software data;

[0006] The technical problem to be solved by the present invention is: how to provide a storage data security management system that is suitable for software development and can use a dynamic hierarchical judgment basis to allocate storage mode for software data.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] A storage data security management system suitable for software development includes a hierarchical storage module, an access monitoring module, an optimization processing module, and a fault monitoring module connected in sequence;

[0009] The hierarchical storage module is used to perform hierarchical storage processing on the storage data of software development: the storage data generated during the software development process is marked as a processing object, a processing cycle is generated, the access frequency of the processing object in the last L1 days is obtained within the processing cycle and marked as an access frequency value, and the storage method of the processing object is marked according to the access frequency value;

[0010] The access monitoring module is used to monitor and analyze the data access efficiency of the memory database: the processing cycle is divided into a number of processing time periods, and at the end of the processing time period, the read and write speed values of the memory database when processing the data access task are obtained and marked as the read speed value, and the memory occupancy value of the memory database at the end of the processing time period is marked as the occupancy data; the read speed values and occupancy data of all processing time periods are processed to obtain the occupancy critical value ZL;

[0011] The optimization processing module is used to optimize the data access efficiency of the memory database;

[0012] The fault monitoring module is used to perform fault monitoring and analysis on the distributed file storage system.

[0013] Furthermore, the specific process of marking the storage method of the processing object includes: comparing the access frequency value with the preset access frequency threshold: if the access frequency value is less than the access frequency threshold, the storage method of the processing object is marked as distributed file storage, and the processing object is stored in the distributed file storage system; if the access frequency value is greater than or equal to the access frequency threshold, the storage method of the processing object is marked as database storage, and the processing object is stored in the memory database.

[0014] Furthermore, the process of obtaining the occupancy optimization value ZY includes: at the end of the processing cycle, a reading speed set is formed by the reading speed values of all processing time periods, the variance of the reading speed set is calculated to obtain a performance stability value, and the performance stability value is compared with the performance stability threshold: if the performance stability value is greater than or equal to the performance stability threshold, the maximum element and the minimum element in the reading speed set are eliminated, and then the performance stability value is recalculated, and so on, until the performance stability value is less than the performance stability threshold; if the performance stability value is less than the performance stability threshold, the maximum value of the occupancy data of the processing time period corresponding to the retained element in the reading speed set is marked as the occupancy critical value ZL.

[0015] Furthermore, the specific process of the optimization processing module optimizing the data access efficiency of the memory database includes: obtaining the occupancy optimization value ZY through the formula ZY=t1×ZL, where t1 is the proportional coefficient, and 0.75≤t1≤0.85; at the end of the processing cycle, the data packets in the memory database are transferred to each distributed file storage system in order of access frequency value from small to large, until the memory occupancy value of the memory database is less than the occupancy optimization value ZY, and the minimum value of the access frequency value of the data packets retained in the memory database is used as the access frequency threshold of the next processing cycle and sent to the hierarchical storage module.

[0016] Furthermore, the specific process of the fault monitoring module performing fault monitoring and analysis on the distributed file storage system includes: marking the hard disk in the distributed file storage system as a monitoring object, obtaining the percentage of bad sectors of the monitoring object at the end of the processing period and marking it as a fault probability value, and marking the monitoring object as a normal object or an abnormal object according to the fault probability value; transferring the storage data in the abnormal object to the normal object, and generating a fault processing signal at the same time and sending the fault processing signal to the administrator's mobile terminal.

[0017] Furthermore, the specific process of marking a monitored object as a normal object or an abnormal object includes: comparing the fault probability value with a preset fault probability threshold: if the fault probability value is less than the fault probability threshold, the corresponding monitored object is marked as a normal object; if the fault probability value is greater than or equal to the fault probability threshold, the corresponding monitored object is marked as an abnormal object.

[0018] Furthermore, the working method of the storage data security management system adapted for software development includes the following steps:

[0019] Step 1: Perform tiered storage processing on the storage data of software development;

[0020] Step 2: Monitor and analyze the data access efficiency of the in-memory database;

[0021] Step 3: Optimize and analyze the data access efficiency of the memory database;

[0022] Step 4: Perform fault monitoring and analysis on the distributed file storage system.

[0023] The present invention has the following beneficial effects:

[0024] 1. The tiered storage module can be used to perform tiered storage processing on the storage data of software development. The corresponding storage method is matched to the access frequency of the stored data, so that data with high access frequency can be stored in the database, and the high read and write speed and low latency characteristics of the database are used to ensure access efficiency.

[0025] 2. The access monitoring module can monitor and analyze the data access efficiency of the in-memory database. The reading speed and occupancy data of each processing period are counted in a periodic and time-divided monitoring manner. The occupancy threshold is marked by combining the reading speed and occupancy data. The occupancy threshold is used to provide feedback on the collapse characteristics of the in-memory database in data access efficiency, providing data support for optimizing the processing and analysis process.

[0026] 3. The optimization processing module can optimize and analyze the data access efficiency of the memory database. After numerical processing of the critical occupancy value, the optimized occupancy value is obtained. Data is transferred based on the optimized occupancy value, reducing the storage burden of the memory database and ensuring its data access efficiency.

[0027] 4. The fault monitoring module can be used to perform fault monitoring and analysis on the distributed file storage system, collect statistics on the bad sector parameters of the hard disk in the distributed file storage system and generate a failure probability value. The failure risk of the monitored object can be evaluated by the failure probability value. When the failure risk does not meet the requirements, data can be transferred in time to avoid data loss caused by hard disk failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 This is a system block diagram of Embodiment 1 of the present invention;

[0030] Figure 2 This is a flow chart of the method of embodiment 2 of the present invention. DETAILED DESCRIPTION

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0032] Example 1: Figure 1 As shown, the storage data security management system adapted to software development includes a hierarchical storage module, an access monitoring module, an optimization processing module and a fault monitoring module connected in sequence, and the optimization processing module is communicatively connected to the hierarchical processing module.

[0033] The tiered storage module is used to perform tiered storage processing on the storage data of software development: the storage data generated during the software development process is marked as a processing object, a processing cycle is generated, the access frequency of the processing object in the last L1 days is obtained within the processing cycle and marked as the access frequency value, and the access frequency value is compared with the preset access frequency threshold: if the access frequency value is less than the access frequency threshold, the storage method of the processing object is marked as distributed file storage, and the processing object is stored in the distributed file storage system; if the access frequency value is greater than or equal to the access frequency threshold, the storage method of the processing object is marked as database storage, and the processing object is stored in the memory database; the storage data of software development is subjected to tiered storage processing, and the corresponding storage method is matched to it according to the access frequency of the storage data, so that data with higher access frequency can be stored through the database, and the high read and write speed and low latency characteristics of the database are used to ensure access efficiency.

[0034] The access monitoring module is used to monitor and analyze the data access efficiency of the memory database: the processing cycle is divided into several processing time periods, and the read and write speed values of the memory database when processing data access tasks are obtained at the end of the processing time period and marked as the read speed value, and the memory occupancy value of the memory database at the end of the processing time period is marked as occupied data; at the end of the processing cycle, the read speed values of all processing time periods are used to form a read speed set, and the variance of the read speed set is calculated to obtain a performance stability value, and the performance stability value is compared with the performance stability threshold: if the performance stability value is greater than or equal to the performance stability threshold, the maximum element in the read speed set is compared with the minimum element Eliminate them, then recalculate the performance stability value, and so on, until the performance stability value is less than the performance stability threshold; if the performance stability value is less than the performance stability threshold, mark the maximum value of the occupancy data of the processing period corresponding to the retained elements in the reading speed set as the occupancy critical value ZL; monitor and analyze the data access efficiency of the memory database, and perform statistics on the reading speed value and occupancy data of each processing period in a periodic time period monitoring manner, mark the occupancy critical value in combination with the reading speed value and occupancy data, and use the occupancy critical value to feedback the collapse characteristics of the memory database in the data access efficiency, so as to provide data support for the optimization of the processing and analysis process.

[0035] The optimization processing module is used to optimize and analyze the data access efficiency of the memory database: the occupancy optimization value ZY is obtained through the formula ZY=t1×ZL, where t1 is the proportional coefficient, and 0.75≤t1≤0.85; at the end of the processing cycle, the data packets in the memory database are transferred to each distributed file storage system in order of access frequency value from small to large, until the memory occupancy value of the memory database is less than the occupancy optimization value ZY, and the minimum access frequency value of the data packet retained in the memory database is used as the access frequency threshold of the next processing cycle and sent to the hierarchical storage module; the data access efficiency of the memory database is optimized and analyzed, and the occupancy optimization value is obtained after numerical processing of the occupancy critical value. The data is transferred based on the occupancy optimization value to reduce the storage burden of the memory database and ensure its data access efficiency.

[0036] The fault monitoring module is used to perform fault monitoring and analysis on the distributed file storage system: mark the hard disk in the distributed file storage system as a monitoring object, obtain the proportion of the number of bad sectors of the monitoring object at the end of the processing period and mark it as a fault probability value, and compare the fault probability value with the preset fault probability threshold: if the fault probability value is less than the fault probability threshold, the corresponding monitoring object is marked as a normal object; if the fault probability value is greater than or equal to the fault probability threshold, the corresponding monitoring object is marked as an abnormal object; the storage data in the abnormal object is transferred to the normal object, and a fault processing signal is generated and sent to the administrator's mobile phone terminal; perform fault monitoring and analysis on the distributed file storage system, count the bad sector parameters of the hard disk in the distributed file storage system and generate a fault probability value, evaluate the failure risk of the monitoring object through the failure probability value, and transfer data in time when the failure risk does not meet the requirements to avoid data loss caused by hard disk failure.

[0037] Example 2: Figure 2 As shown, the storage data security management method adapted for software development includes the following steps:

[0038] Step 1: Perform tiered storage processing on the storage data generated during software development: Mark the storage data generated during the software development process as processing objects. During the processing cycle, obtain the access frequency of the processing objects in the last L1 days and mark it as the access frequency value. The access frequency value is used to mark the storage method of the processing objects.

[0039] Step 2: Monitor and analyze the data access efficiency of the in-memory database: Divide the processing cycle into several processing time periods. At the end of each processing time period, obtain the read and write speed values of the in-memory database when processing data access tasks and mark them as read speed values. Calculate the read speed values of all processing time periods to obtain the performance stability value, and use the performance stability value to mark the critical occupation value ZL.

[0040] Step 3: Optimize and analyze the data access efficiency of the in-memory database: Calculate the critical occupancy value to obtain the optimized occupancy value ZY. Use the optimized occupancy value ZY to transfer the data packets in the in-memory database to various distributed file storage systems.

[0041] Step 4: Perform fault monitoring and analysis on the distributed file storage system: Mark the hard disks in the distributed file storage system as monitoring objects, obtain the percentage of bad sectors in the monitoring objects at the end of the processing period and mark it as the failure probability value, mark the monitoring objects as normal objects or abnormal objects based on the failure probability value, and transfer the storage data in the abnormal objects to normal objects.

[0042] A storage data security management system adapted for software development, when in operation, marks the storage data generated during the software development process as a processing object, obtains the access frequency of the processing object in the last L1 days within the processing cycle and marks it as an access frequency value, and marks the storage method of the processing object by the access frequency value; divides the processing cycle into several processing time periods, obtains the read and write speed values of the memory database when processing data access tasks at the end of the processing time period and marks them as read speed values, numerically calculates the read speed values of all processing time periods to obtain a performance stability value, and marks the occupancy critical value ZL by the performance stability value; numerically calculates the occupancy critical value to obtain an occupancy optimization value ZY, and transfers the data packets in the memory database to various distributed file storage systems by the occupancy optimization value ZY; marks the hard disk in the distributed file storage system as a monitoring object, obtains the proportion of the number of bad sectors of the monitoring object at the end of the processing time period and marks it as a failure probability value, marks the monitoring object as a normal object or an abnormal object by the failure probability value, and transfers the storage data in the abnormal object to a normal object.

[0043] The above content is merely an example and explanation of the structure of the present invention. Those skilled in the art may make various modifications or additions to the described specific embodiments or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the scope of protection of the present invention.

[0044] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0045] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to specific embodiments. Obviously, many modifications and variations are possible based on the contents of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A storage data security management system adapted for software development, characterized in that: It includes a hierarchical storage module, an access monitoring module, an optimization processing module and a fault monitoring module connected in sequence; The hierarchical storage module is used to perform hierarchical storage processing on the storage data of software development: the storage data generated during the software development process is marked as a processing object, a processing cycle is generated, the access frequency of the processing object in the last L1 days is obtained within the processing cycle and marked as an access frequency value, and the storage method of the processing object is marked according to the access frequency value; The access monitoring module is used to monitor and analyze the data access efficiency of the memory database: the processing cycle is divided into a number of processing time periods, and at the end of the processing time period, the read and write speed values of the memory database when processing the data access task are obtained and marked as the read speed value, and the memory occupancy value of the memory database at the end of the processing time period is marked as the occupied data; The reading speed values and occupancy data of all processing periods are processed to obtain the occupancy critical value ZL; The optimization processing module is used to optimize the data access efficiency of the memory database; The fault monitoring module is used to perform fault monitoring and analysis on the distributed file storage system.

2. The storage data security management system adapted for software development according to claim 1, characterized in that: The specific process of marking the storage method of the processing object includes: comparing the access frequency value with the preset access frequency threshold: if the access frequency value is less than the access frequency threshold, the storage method of the processing object is marked as distributed file storage, and the processing object is stored in the distributed file storage system; if the access frequency value is greater than or equal to the access frequency threshold, the storage method of the processing object is marked as database storage, and the processing object is stored in the memory database.

3. The storage data security management system adapted for software development according to claim 2, characterized in that: The process of obtaining the occupancy optimization value ZY includes: at the end of the processing cycle, the reading speed values of all processing time periods are used to form a reading speed set, the variance of the reading speed set is calculated to obtain a performance stability value, and the performance stability value is compared with the performance stability threshold: if the performance stability value is greater than or equal to the performance stability threshold, the maximum element and the minimum element in the reading speed set are eliminated, and then the performance stability value is recalculated, and so on, until the performance stability value is less than the performance stability threshold; if the performance stability value is less than the performance stability threshold, the maximum value of the occupancy data of the processing time period corresponding to the retained element in the reading speed set is marked as the occupancy critical value ZL.

4. The storage data security management system adapted for software development according to claim 3, characterized in that: The specific process of the optimization processing module optimizing and analyzing the data access efficiency of the memory database includes: obtaining the occupancy optimization value ZY through the formula ZY=t1×ZL, where t1 is the proportional coefficient, and 0.75≤t1≤0.85; at the end of the processing cycle, the data packets in the memory database are transferred to each distributed file storage system in the order of access frequency value from small to large, until the memory occupancy value of the memory database is less than the occupancy optimization value ZY, and the minimum value of the access frequency value of the data packets retained in the memory database is used as the access frequency threshold of the next processing cycle and sent to the hierarchical storage module.

5. The storage data security management system adapted for software development according to claim 4, characterized in that: The specific process of the fault monitoring module performing fault monitoring and analysis on the distributed file storage system includes: marking the hard disk in the distributed file storage system as a monitoring object, obtaining the percentage of bad sectors of the monitoring object at the end of the processing period and marking it as a fault probability value, and marking the monitoring object as a normal object or an abnormal object based on the fault probability value; transferring the storage data in the abnormal object to the normal object, and generating a fault processing signal at the same time and sending the fault processing signal to the administrator's mobile terminal.

6. The storage data security management system adapted for software development according to claim 5, characterized in that: The specific process of marking a monitored object as a normal object or an abnormal object includes: comparing the fault probability value with a preset fault probability threshold: if the fault probability value is less than the fault probability threshold, the corresponding monitored object is marked as a normal object; if the fault probability value is greater than or equal to the fault probability threshold, the corresponding monitored object is marked as an abnormal object.

7. The storage data security management system adapted for software development according to any one of claims 1 to 6, characterized in that: The working method of the storage data security management system adapted for software development comprises the following steps: Step 1: Perform tiered storage processing on the storage data of software development; Step 2: Monitor and analyze the data access efficiency of the in-memory database; Step 3: Optimize and analyze the data access efficiency of the memory database; Step 4: Perform fault monitoring and analysis on the distributed file storage system.

Images