Distributed data storage and access software development method based on cloud computing

Abstract

The invention provides a distributed data storage and access software development method based on cloud computing, and aims to construct an efficient, safe and extensible data storage system. The method comprises seven key steps of demand analysis and system design, cloud resource configuration and initialization, distributed data storage algorithm development, data processing layer construction, access control layer development, user interface and interaction design and system test and optimization. By accurately calculating storage resource requirements, optimizing cloud resource configuration and designing a consistent Hash data fragmentation algorithm and a data redundancy strategy, high availability and fault tolerance of data are ensured. Meanwhile, the AES-256 encryption technology and the LZ4 compression technology are adopted to guarantee the data security and the transmission efficiency. According to the method disclosed by the invention, a high-performance, extensible, safe and reliable distributed data storage and access system can be constructed, so that the increasing data storage and access requirements are met, and the user experience is improved.

Description

Technical Field

[0001] The invention belongs to the field of software development methods, and specifically discloses a distributed data storage and access software development method based on cloud computing. Background Art

[0002] With the rapid development of information technology, data has become a core resource in modern society. Traditional data storage methods, such as local storage or centralized storage, can no longer meet the growing demand for data storage and efficient access requirements. Especially in the era of big data, the amount of data is growing explosively, and data access patterns are becoming increasingly complex and diverse, which places higher demands on the scalability, performance, and security of data storage systems.

[0003] In the prior art, distributed data storage systems are widely used due to their good scalability and fault tolerance. However, these systems still face many challenges. On the one hand, traditional distributed storage systems often lack flexibility in resource allocation, and it is difficult to dynamically adjust storage and computing resources according to actual needs, resulting in low resource utilization or waste of resources. On the other hand, the unreasonable design of data sharding and redundancy strategies often leads to increased data access delays and reduced system performance. In addition, data security and privacy protection are also urgent issues to be solved in existing distributed storage systems. The inadequacy of data encryption and compression technology makes data vulnerable to attacks during transmission and storage, and occupies a large amount of storage space.

[0004] At the same time, the access control mechanism of existing distributed storage systems is often too simple to meet the fine-grained permission management requirements in complex application scenarios. The lack of effective access auditing functions also makes it difficult for the system to trace and monitor access operations, posing security risks. In addition, the shortcomings of user interface and interaction design also affect the user's operating experience. Especially when processing large-scale data, users often face problems such as cumbersome operations and slow responses. Summary of the invention

[0005] In view of the above problems, the present invention proposes a distributed data storage and access software development method based on cloud computing to solve the problems existing in the prior art. The method of the present invention can build a high-performance, scalable, secure and reliable distributed data storage and access system to meet the growing data storage and access needs and improve user experience.

[0006] The present invention includes the following technical solutions:

[0007] A distributed data storage and access software development method based on cloud computing, characterized in that the method comprises the following seven steps in sequence:

[0008] S1, conduct demand analysis and system design, in which the overall architecture of the system is designed by analyzing the user's data storage needs, access patterns, security requirements and performance indicators, and the required storage resources are calculated using the formula R = D*(1+G), where D is the current data volume, G is the annual data growth rate, and R is the expected storage resource demand. At the same time, data sharding and redundancy strategies are designed to ensure high data availability and fault tolerance;

[0009] S2, configure and initialize cloud resources. According to the system design results, select and configure computing, storage and network resources on the cloud platform. Use the formula N = C / (P*E) to calculate the number of virtual machines required, where C is the total computing power requirement, P is the computing power of a single virtual machine, and E is the efficiency coefficient. At the same time, set the security policy and access rights of the cloud service.

[0010] S3, develop a distributed data storage algorithm, design a data sharding algorithm based on consistent hashing, map data objects to storage nodes, the formula is index = hash (key) % N, where key is the data object identifier, N is the number of storage nodes, and implement a data redundancy algorithm, such as setting a replication factor F to ensure that each data slice has a copy on F nodes;

[0011] S4, build the data processing layer, develop the data encryption algorithm, use AES-256 encryption, the encryption process is expressed as Cipher = Encrypt (Plain, Key), where Plain is plaintext, Key is the key, and Cipher is the ciphertext; at the same time, develop the data compression algorithm, use LZ4 compression, and the compression ratio is calculated as Ratio = Original / Compressed, where Original is the original data size and Compressed is the compressed data size;

[0012] S5, develop the access control layer, design and implement the role-based access control RBAC model, define roles, permissions and role-permission relationships; at the same time, implement the access audit function, record each access operation, including user, time, operation type and accessed data object, and the audit record is expressed as Log = (User, Time, Action, Object);

[0013] S6, design and implement user interface and interaction, develop intuitive and easy-to-use Web interface, support data upload, download, query and management; provide API interface, allow third-party applications to interact with the system through HTTP request, API call is expressed as Response = API (Endpoint, Method, Params), where Endpoint is API address, Method is request method, Params is request parameters;

[0014] S7, conduct system testing and optimization, conduct comprehensive functional testing, performance testing and security testing on the system to ensure that the system meets the design requirements; according to the test results, use the formula Opt = Org-Imp to calculate the optimization space, where Org is the original performance indicator, Imp is the improved performance indicator, and Opt is the optimization space; based on the optimization space, optimize the system architecture, algorithm and code to improve system performance and user experience.

[0015] Furthermore, the present invention discloses a computer storage device for executing the above-mentioned distributed data storage and access software development method based on cloud computing, including demand analysis, resource allocation, algorithm development, data processing, access control, interface design and system testing optimization modules to realize data storage and access services.

[0016] Furthermore, the present invention also discloses a data processing server, comprising the above-mentioned computer storage device.

[0017] The present invention has the following beneficial effects:

[0018] The distributed data storage and access software development method based on cloud computing proposed in the present invention ensures that the system can meet the diverse data storage and access needs of users through refined demand analysis and system design, and at the same time realizes an efficient, secure and scalable system architecture. The scientific storage resource demand calculation method and reasonable data sharding and redundancy strategy are adopted to effectively improve the utilization rate of storage resources and the availability of data. By developing an advanced data processing layer, efficient encryption and compression of data are achieved, data security is guaranteed and storage space occupancy is reduced. The role-based access control model and access audit function provide the system with fine-grained permission management and operation traceability capabilities, significantly enhancing the security of the system. The intuitive and easy-to-use Web interface and API interface not only improve the user experience, but also promote data sharing and application integration. Comprehensive system testing and optimization steps ensure the stability and performance of the system in actual operation, and provide strong support for the continuous optimization and upgrading of the system. In summary, the development method described in the present invention shows significant advantages in improving data storage and access efficiency, ensuring data security, enhancing system scalability and user experience, etc., and provides an efficient and reliable solution for distributed data storage and access in a cloud computing environment, with broad application prospects and important practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The main flow chart for the development of the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Example

[0022] A distributed data storage and access software development method based on cloud computing, the method sequentially comprises the following steps: Figure 1 The following seven steps are shown:

[0023] S1, conduct demand analysis and system design:

[0024] S11) Analysis of user data storage needs: Analyze and clarify user data storage needs through questionnaires, interviews, and workshops, including but not limited to data type, scale, current data volume D, expected growth, access mode, read and write frequency, query complexity, and data update strategy;

[0025] S12 User security requirements analysis: Analyze and determine user security requirements for data storage and access, including data confidentiality, integrity, availability, and audit and traceability capabilities;

[0026] S13 Performance indicator determination: negotiate with users and determine the key performance indicators of the system, including but not limited to response time, throughput, number of concurrent users, data recovery time objective (RTO) and data recovery point objective (RPO);

[0027] S14 System overall architecture design: Based on the above analysis, the overall architecture of the system is designed, which includes but is not limited to the front-end user interface, back-end data processing layer, storage layer, access control layer, and monitoring and operation and maintenance layer. The microservice architecture is used to promote the scalability, maintainability and high availability of the system;

[0028] S15 Storage resource demand calculation: The formula R = D*(1+G) is used to estimate storage resource demand, where D represents the current data volume, G represents the annual growth rate of data, and R represents the expected storage resource demand. The storage resource demand is calculated by further considering data redundancy, storage efficiency, nonlinear factors of data growth, and future development of storage technology.

[0029] S16 Data sharding and redundancy strategy design: Design data sharding and redundancy strategies, and use consistent hashing algorithm technology to achieve uniform distribution and efficient access to data; set a reasonable replication factor to ensure that when a single storage node fails, data can be quickly restored from other nodes to ensure high availability and fault tolerance of the system; at the same time, formulate data backup and recovery strategies to respond to catastrophic events and ensure business continuity.

[0030] S2, configure and initialize cloud resources:

[0031] S21 Cloud resource selection: Based on the system design results of step S1, the required computing, storage and network resource specifications are clarified, a cloud platform service provider that meets the requirements is selected, and the types of cloud services to be used are determined, including but not limited to virtual machines, object storage, block storage, database services, and network services;

[0032] S22 Determination of total computing power requirements: Based on factors such as business processing requirements, data processing algorithm complexity, and expected number of concurrent users in system design, the total computing power requirements C of the system are comprehensively evaluated and determined. This requirement is expressed in quantitative indicators such as the number of operations that can be executed per second, the amount of data processed per second, or the number of CPU cores required;

[0033] S23 Evaluation of computing power of a single virtual machine: Conduct a detailed evaluation of the types of virtual machines provided by the cloud platform to determine the computing power of a single virtual machine P, including key parameters such as the number of CPU cores, memory size, and disk I / O performance, as well as the specific impact of these parameters on system performance;

[0034] S24 Determination of efficiency coefficient: Based on the actual operating environment of the system, the cost of virtualization technology, the service quality assurance (SLA) of the cloud platform and other factors, the efficiency coefficient E is reasonably set. This coefficient is used to reflect the actual utilization rate or efficiency of the virtual machine computing capacity relative to its theoretical value in actual operation;

[0035] S25 Calculation of the number of virtual machines required: Use the refined formula N = C / (P*E) to accurately calculate the number of virtual machines required, where N is the total number of virtual machines required, C is the total computing power requirement, P is the computing power of a single virtual machine, and E is the efficiency coefficient; this calculation takes into account the efficiency loss in actual operation to ensure that the configured virtual machine resources can meet the computing power requirements of the system;

[0036] S26 Cloud Resource Configuration: Based on the calculated number of virtual machines and other resource requirements, configure the corresponding computing resources, storage resources, and network resources on the cloud platform, including selecting the instance type of the virtual machine, configuring storage volumes, setting the network topology, allocating IP addresses, etc.

[0037] S27 Cloud service security policy settings: Set strict security policies for configured cloud services, including but not limited to network access control, firewall rules, data encryption policies, authentication and authorization mechanisms, to ensure the security of cloud services and data protection;

[0038] S28 Access permission configuration: Configure appropriate access permissions for different users or user groups based on user roles, business requirements, and security policies to ensure that only authorized users can access and operate resources in cloud services.

[0039] S3, develops distributed data storage algorithms:

[0040] S31 Design a data sharding algorithm based on consistent hashing: Design and implement a data sharding algorithm based on consistent hashing function, which is used to evenly map data objects to storage nodes; the algorithm uses the formula index = hash (key) % N for calculation, where key is the unique identifier of the data object, hash (key) is the hash value obtained by applying the hash function to the key, N is the total number of current storage nodes, % represents the modulo operation, and index is the calculated storage node index; the hash function has good hashability and low collision probability to achieve uniform distribution of data objects;

[0041] S32 implements data redundancy algorithm: To improve the high availability and fault tolerance of data, design and implement data redundancy algorithm; set the replication factor F to ensure that each data slice has a copy on F different storage nodes; when writing data, store the data slice on the primary node according to the index calculated by the data sharding algorithm, and select F-1 other nodes as replica nodes according to the replication factor F, and copy the data slice to these nodes; when reading data, first try to read the data slice from the primary node. If the primary node is unavailable, try to read from the replica nodes in turn;

[0042] S33 algorithm optimization and implementation: optimize the data sharding algorithm and data redundancy algorithm based on consistent hashing to improve the execution efficiency and reliability of the algorithm; consider the dynamic changes of storage nodes and design corresponding node joining and exiting mechanisms; implement the algorithm code and conduct sufficient testing, including unit testing, integration testing and performance testing;

[0043] S34 Algorithm Integration and Verification: Integrate the designed and implemented data sharding algorithm and data redundancy algorithm into the distributed data storage and access software system; verify and test the algorithm by simulating actual business scenarios and data access patterns to ensure the effectiveness and reliability of the algorithm in actual operation.

[0044] S4, build the data processing layer:

[0045] S41 develops data encryption algorithm: To achieve data security during storage and transmission, the AES-256 encryption algorithm is used to encrypt data; the encryption process is expressed as Cipher = Encrypt (Plain, Key), where Plain represents the plaintext data to be encrypted, Key represents the encryption key, and Cipher represents the ciphertext data obtained after encryption; the secure storage and management of the key, as well as the correctness and efficiency of the encryption process are achieved;

[0046] S42 develops data compression algorithm: To reduce storage space usage and improve data transmission efficiency, the LZ4 compression algorithm is used to compress data; the compression ratio is calculated as Ratio = Original / Compressed, where Original represents the size of the original data and Compressed represents the size of the compressed data; the efficiency of the compression algorithm and the rationality of the compression ratio are achieved, while ensuring the recoverability of the compressed data;

[0047] S43 Data Processing Layer Construction: Integrate the developed data encryption algorithm and data compression algorithm into the data processing layer; before writing data into the storage node, encrypt and compress the data and store it in ciphertext and compressed form; when reading data, first obtain the ciphertext and compressed data from the storage node, then decrypt and decompress it to restore the original data; realize that the data processing layer can efficiently and securely complete the encryption, compression, decryption and decompression operations of data;

[0048] S44 Algorithm testing and optimization: Conduct sufficient testing on the developed data encryption algorithm and data compression algorithm, including functional testing, performance testing, and security testing; optimize and adjust the algorithm based on the test results to achieve its reliability and efficiency in practical applications; at the same time, continue to pay attention to and research new encryption algorithms and compression algorithms so as to timely update and upgrade the data processing layer.

[0049] S5, develop access control layer:

[0050] S51 designs a role-based access control (RBAC) model: To achieve fine-grained access control and permission management, the Role-Based Access Control model is designed and implemented; roles, permissions, and the relationship between roles and permissions in the system are clearly defined; each role has appropriate permissions to meet the access requirements of different users or user groups to system resources; at the same time, a flexible role and permission management mechanism is provided to enable dynamic adjustments based on business needs;

[0051] S52 implements access audit function: To achieve the traceability and security of system access operations, develop access audit function; record relevant information of each access operation, including the user who performs the access operation (User), the time when the access operation occurs (Time), the type of access operation (Action), and the data object accessed (Object); the audit record is expressed as Log = (User, Time, Action, Object); realize the integrity, accuracy and non-tamperability of the audit record, so as to conduct audit analysis, troubleshooting or security audit when necessary;

[0052] S53 Access control layer integration and testing: Integrate the designed RBAC model and the implemented access audit function into the access control layer; conduct comprehensive testing on the access control layer, including functional testing, performance testing, and security testing; ensure that the access control layer can correctly and efficiently perform access control and audit operations while meeting the security and reliability requirements of the system;

[0053] S54 Access control policy optimization and update: According to the actual operation of the system and business needs, the access control policy is optimized and updated regularly; the definition of roles and permissions, as well as the relationship between roles and permissions, is adjusted; at the same time, the access audit records are analyzed to identify potential access risks or abnormal behaviors, and corresponding security measures are taken in a timely manner to prevent and respond.

[0054] S6, design and implement user interface and interaction:

[0055] S61 Develop an intuitive and easy-to-use Web interface: Design and implement a user-friendly Web interface to provide convenient data management operations; the interface should support multiple functions such as data upload, download, query and management, so that users can easily complete data operations; use intuitive design elements and interactive methods, including clear navigation menus, convenient operation buttons, and real-time feedback prompts to enhance the user experience;

[0056] S62 provides API interface to support third-party application interaction: To expand the application scope of the system and promote data sharing, an API interface is developed to allow third-party applications to interact with the system through HTTP requests; API calls are expressed as Response = API (Endpoint, Method, Params), where Endpoint represents the address of the API, Method represents the request method, and Params represents the request parameters; the design of the API interface is concise, standardized, easy to understand and use, and detailed API documentation and sample code are provided to facilitate third-party developers to integrate and develop;

[0057] S63 user interface and interaction integration and testing: Integrate the developed Web interface and API interface into the system and conduct comprehensive testing; the test content includes the functional integrity of the interface, the smoothness of the interaction, the availability, response speed and security of the API interface; ensure that the user interface and interaction can meet the needs of users and provide stable and reliable services;

[0058] Optimization and update of S64 user interface and interaction: Regularly optimize and update the user interface and interaction based on user feedback and business needs; adjust the layout and style of the interface to improve the aesthetics and usability of the interface; add or improve functional modules to meet new user needs; at the same time, monitor and analyze the API interface to promptly discover and fix potential problems to ensure the stability and security of the API interface.

[0059] S7, perform system testing and optimization:

[0060] S71 Comprehensive System Test: Perform comprehensive functional tests, performance tests, and security tests on the system to verify whether the system meets the design requirements; functional tests ensure that all functions of the system operate normally, performance tests evaluate the performance of the system under different loads, and security tests check the system's defense capabilities against potential security threats;

[0061] S72 Optimization space evaluation: Based on the test results, the optimization space is calculated using the formula Opt = Org-Imp, where Org represents the original performance index, Imp represents the currently achieved performance index, and Opt represents the potential optimization space; quantifying the possibility of improving system performance and providing a clear direction for subsequent optimization work;

[0062] S73 system optimization implementation: Based on the optimization space obtained through evaluation, the system architecture, algorithms and codes are optimized; the system architecture design is adjusted to improve the overall performance, the algorithm is optimized to improve the data processing efficiency, and the code is streamlined to reduce resource consumption.

[0063] It can be seen from the above embodiments that the present invention can build a high-performance, scalable, secure and reliable distributed data storage and access system through precise demand analysis and system design, flexible cloud resource configuration and initialization, efficient data sharding and redundancy strategy, powerful data processing and security protection mechanism, fine-grained access control and audit function, and intuitive and easy-to-use user interface and interaction design. The method described in the present invention can meet the growing data storage and access needs and improve the user experience.

[0064] The above are only a few preferred embodiments of the present invention, which are described in a more specific and detailed manner, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A distributed data storage and access software development method based on cloud computing, characterized in that: The method includes the following seven steps in sequence: S1, conduct demand analysis and system design, in which the overall architecture of the system is designed by analyzing the user's data storage needs, access patterns, security requirements and performance indicators, and the required storage resources are calculated using the formula R = D*(1+G), where D is the current data volume, G is the annual data growth rate, and R is the expected storage resource demand. At the same time, data sharding and redundancy strategies are designed to ensure high data availability and fault tolerance; S2, configure and initialize cloud resources. According to the system design results, select and configure computing, storage and network resources on the cloud platform. Use the formula N = C / (P*E) to calculate the number of virtual machines required, where C is the total computing power requirement, P is the computing power of a single virtual machine, and E is the efficiency coefficient. At the same time, set the security policy and access rights of the cloud service. S3, develop a distributed data storage algorithm, design a data sharding algorithm based on consistent hashing, map data objects to storage nodes, the formula is index = hash (key) % N, where key is the data object identifier, N is the number of storage nodes, and implement a data redundancy algorithm, such as setting a replication factor F to ensure that each data slice has a copy on F nodes; S4, build the data processing layer, develop the data encryption algorithm, use AES-256 encryption, the encryption process is expressed as Cipher = Encrypt (Plain, Key), where Plain is plaintext, Key is the key, and Cipher is the ciphertext; at the same time, develop the data compression algorithm, use LZ4 compression, and the compression ratio is calculated as Ratio = Original / Compressed, where Original is the original data size and Compressed is the compressed data size; S5, develop the access control layer, design and implement the role-based access control RBAC model, define roles, permissions and role-permission relationships; at the same time, implement the access audit function, record each access operation, including user, time, operation type and accessed data object, and the audit record is expressed as Log = (User, Time, Action, Object); S6, design and implement user interface and interaction, develop intuitive and easy-to-use Web interface, support data upload, download, query and management; provide API interface, allow third-party applications to interact with the system through HTTP request, API call is expressed as Response = API (Endpoint, Method, Params), where Endpoint is API address, Method is request method, Params is request parameters; S7, conduct system testing and optimization, conduct comprehensive functional testing, performance testing and security testing on the system to ensure that the system meets the design requirements; according to the test results, use the formula Opt = Org-Imp to calculate the optimization space, where Org is the original performance indicator, Imp is the improved performance indicator, and Opt is the optimization space; based on the optimization space, optimize the system architecture, algorithm and code to improve system performance and user experience.

2. The distributed data storage and access software development method based on cloud computing according to claim 1 is characterized in that: The step S1 includes the following specific steps: S11) Analysis of user data storage needs: Analyze and clarify user data storage needs through questionnaires, interviews, and workshops, including but not limited to data type, scale, current data volume D, expected growth, access mode, read and write frequency, query complexity, and data update strategy; S12 User security requirements analysis: Analyze and determine user security requirements for data storage and access, including data confidentiality, integrity, availability, and audit and traceability capabilities; S13 Performance indicator determination: negotiate with users and determine the key performance indicators of the system, including but not limited to response time, throughput, number of concurrent users, data recovery time objective (RTO) and data recovery point objective (RPO); S14 System overall architecture design: Based on the above analysis, the overall architecture of the system is designed, which includes but is not limited to the front-end user interface, back-end data processing layer, storage layer, access control layer, and monitoring and operation and maintenance layer. The microservice architecture is used to promote the scalability, maintainability and high availability of the system; S15 Storage resource demand calculation: The formula R = D*(1+G) is used to estimate storage resource demand, where D represents the current data volume, G represents the annual growth rate of data, and R represents the expected storage resource demand. The storage resource demand is calculated by further considering data redundancy, storage efficiency, nonlinear factors of data growth, and future development of storage technology. S16 Data sharding and redundancy strategy design: Design data sharding and redundancy strategies, and use consistent hashing algorithm technology to achieve uniform distribution and efficient access to data; Set a reasonable replication factor to ensure that when a single storage node fails, data can be quickly restored from other nodes to ensure high availability and fault tolerance of the system; at the same time, formulate a data backup and recovery strategy to deal with catastrophic events and ensure business continuity.

3. The distributed data storage and access software development method based on cloud computing according to claim 1 is characterized in that: The step S2 comprises the following specific steps: S21 Cloud resource selection: Based on the system design results of step S1, the required computing, storage and network resource specifications are clarified, a cloud platform service provider that meets the requirements is selected, and the types of cloud services to be used are determined, including but not limited to virtual machines, object storage, block storage, database services, and network services; S22 Determination of total computing power requirements: Based on factors such as business processing requirements, data processing algorithm complexity, and expected number of concurrent users in system design, the total computing power requirements C of the system are comprehensively evaluated and determined. This requirement is expressed in quantitative indicators such as the number of operations that can be executed per second, the amount of data processed per second, or the number of CPU cores required; S23 Evaluation of computing power of a single virtual machine: Conduct a detailed evaluation of the types of virtual machines provided by the cloud platform to determine the computing power of a single virtual machine P, including key parameters such as the number of CPU cores, memory size, and disk I / O performance, as well as the specific impact of these parameters on system performance; S24 Determination of efficiency coefficient: Based on the actual operating environment of the system, the cost of virtualization technology, the service quality assurance (SLA) of the cloud platform and other factors, the efficiency coefficient E is reasonably set. This coefficient is used to reflect the actual utilization rate or efficiency of the virtual machine computing capacity relative to its theoretical value in actual operation; S25 Calculation of the number of virtual machines required: Use the refined formula N = C / (P*E) to accurately calculate the number of virtual machines required, where N is the total number of virtual machines required, C is the total computing power requirement, P is the computing power of a single virtual machine, and E is the efficiency coefficient; this calculation takes into account the efficiency loss in actual operation to ensure that the configured virtual machine resources can meet the computing power requirements of the system; S26 Cloud Resource Configuration: Based on the calculated number of virtual machines and other resource requirements, configure the corresponding computing resources, storage resources, and network resources on the cloud platform, including selecting the instance type of the virtual machine, configuring storage volumes, setting the network topology, allocating IP addresses, etc. S27 Cloud service security policy settings: Set strict security policies for configured cloud services, including but not limited to network access control, firewall rules, data encryption policies, authentication and authorization mechanisms, to ensure the security of cloud services and data protection; S28 Access permission configuration: Configure appropriate access permissions for different users or user groups based on user roles, business requirements, and security policies to ensure that only authorized users can access and operate resources in cloud services.

4. The distributed data storage and access software development method based on cloud computing according to claim 1 is characterized in that: The step S3 comprises the following specific steps: S31 Design a data sharding algorithm based on consistent hashing: Design and implement a data sharding algorithm based on consistent hashing function to evenly map data objects to storage nodes; The algorithm uses the formula index = hash (key) % N for calculation, where key is the unique identifier of the data object, hash (key) is the hash value obtained by applying the hash function to key, N is the total number of current storage nodes, % represents the modulo operation, and index is the calculated storage node index; this makes the hash function have good hashability and low collision probability to achieve uniform distribution of data objects; S32 implements data redundancy algorithm: To improve the high availability and fault tolerance of data, design and implement data redundancy algorithm; set the replication factor F to ensure that each data slice has a copy on F different storage nodes; when writing data, store the data slice on the primary node according to the index calculated by the data sharding algorithm, and select F-1 other nodes as replica nodes according to the replication factor F, and copy the data slice to these nodes; when reading data, first try to read the data slice from the primary node. If the primary node is unavailable, try to read from the replica nodes in turn; S33 algorithm optimization and implementation: optimize the data sharding algorithm and data redundancy algorithm based on consistent hashing to improve the execution efficiency and reliability of the algorithm; consider the dynamic changes of storage nodes and design corresponding node joining and exiting mechanisms; implement the algorithm code and conduct sufficient testing, including unit testing, integration testing and performance testing; S34 Algorithm Integration and Verification: Integrate the designed and implemented data sharding algorithm and data redundancy algorithm into the distributed data storage and access software system; verify and test the algorithm by simulating actual business scenarios and data access patterns to ensure the effectiveness and reliability of the algorithm in actual operation.

5. The distributed data storage and access software development method based on cloud computing according to claim 1 is characterized in that: The step S4 comprises the following specific steps: S41 develops data encryption algorithm: To achieve data security during storage and transmission, the AES-256 encryption algorithm is used to encrypt data; the encryption process is expressed as Cipher = Encrypt (Plain, Key), where Plain represents the plaintext data to be encrypted, Key represents the encryption key, and Cipher represents the ciphertext data obtained after encryption; Achieve secure storage and management of keys, as well as correctness and efficiency of encryption process; S42 develops data compression algorithm: To reduce storage space usage and improve data transmission efficiency, the LZ4 compression algorithm is used to compress data; the compression ratio is calculated as Ratio = Original / Compressed, where Original represents the size of the original data and Compressed represents the size of the compressed data; the efficiency of the compression algorithm and the rationality of the compression ratio are achieved, while ensuring the recoverability of the compressed data; S43 Data Processing Layer Construction: Integrate the developed data encryption algorithm and data compression algorithm into the data processing layer; before writing data into the storage node, encrypt and compress the data and store it in ciphertext and compressed form; when reading data, first obtain the ciphertext and compressed data from the storage node, then decrypt and decompress it to restore the original data; The data processing layer can efficiently and securely complete data encryption, compression, decryption and decompression operations; S44 Algorithm testing and optimization: Conduct sufficient testing on the developed data encryption algorithm and data compression algorithm, including functional testing, performance testing, and security testing; Based on the test results, the algorithm is optimized and adjusted to achieve its reliability and efficiency in practical applications; at the same time, we continue to pay attention to and research new encryption algorithms and compression algorithms so that the data processing layer can be updated and upgraded in a timely manner.

6. The distributed data storage and access software development method based on cloud computing according to claim 1, characterized in that: The step S5 comprises the following specific steps: S51 Design a role-based access control (RBAC) model: To achieve fine-grained access control and permission management, design and implement a Role-Based Access Control model; clearly define the roles, permissions, and the relationship between roles and permissions in the system; Ensure that each role has appropriate permissions to meet the access needs of different users or user groups to system resources; at the same time, provide a flexible role and permission management mechanism to enable dynamic adjustments based on business needs; S52 implements access audit function: To achieve the traceability and security of system access operations, develop access audit function; record relevant information of each access operation, including the user who performs the access operation (User), the time when the access operation occurs (Time), the type of access operation (Action), and the data object accessed (Object); the audit record is expressed as Log = (User, Time, Action, Object); realize the integrity, accuracy and non-tamperability of the audit record, so as to conduct audit analysis, troubleshooting or security audit when necessary; S53 Access control layer integration and testing: Integrate the designed RBAC model and the implemented access audit function into the access control layer; conduct comprehensive testing on the access control layer, including functional testing, performance testing, and security testing; The access control layer can correctly and efficiently perform access control and audit operations while meeting the security and reliability requirements of the system; S54 Access control policy optimization and update: According to the actual operation of the system and business needs, the access control policy is optimized and updated regularly; the definition of roles and permissions, as well as the relationship between roles and permissions, is adjusted; at the same time, the access audit records are analyzed to identify potential access risks or abnormal behaviors, and corresponding security measures are taken in a timely manner to prevent and respond.

7. The method for developing distributed data storage and access software based on cloud computing according to claim 1, characterized in that: The step S6 comprises the following specific steps: S61 Develop an intuitive and easy-to-use Web interface: Design and implement a user-friendly Web interface to provide convenient data management operations; the interface should support multiple functions such as data upload, download, query and management, so that users can easily complete data operations; use intuitive design elements and interactive methods, including clear navigation menus, convenient operation buttons, and real-time feedback prompts to enhance the user experience; S62 provides API interface to support third-party application interaction: To expand the application scope of the system and promote data sharing, an API interface is developed to allow third-party applications to interact with the system through HTTP requests; API calls are expressed as Response = API (Endpoint, Method, Params), where Endpoint represents the address of the API, Method represents the request method, and Params represents the request parameters; the design of the API interface is concise, standardized, easy to understand and use, and detailed API documentation and sample code are provided to facilitate third-party developers to integrate and develop; Integration and testing of S63 user interface and interaction: Integrate the developed web interface and API interface into the system and conduct comprehensive testing; The test content includes the functional integrity of the interface, the fluency of interaction, the availability, response speed and security of the API interface; ensuring that the user interface and interaction can meet the needs of users and provide stable and reliable services; Optimization and update of S64 user interface and interaction: Regularly optimize and update the user interface and interaction based on user feedback and business needs; adjust the layout and style of the interface to improve the aesthetics and usability of the interface; add or improve functional modules to meet new user needs; at the same time, monitor and analyze the API interface to promptly discover and fix potential problems to ensure the stability and security of the API interface.

8. The method for developing distributed data storage and access software based on cloud computing according to claim 1, characterized in that: The step S7 comprises the following specific steps: S71 Comprehensive System Test: Perform comprehensive functional tests, performance tests, and security tests on the system to verify whether the system meets the design requirements; functional tests ensure that all functions of the system operate normally, performance tests evaluate the performance of the system under different loads, and security tests check the system's defense capabilities against potential security threats; S72 Optimization space evaluation: Based on the test results, the optimization space is calculated using the formula Opt = Org-Imp, where Org represents the original performance index, Imp represents the currently achieved performance index, and Opt represents the potential optimization space; Quantify the possibility of improving system performance and provide a clear direction for subsequent optimization work; S73 system optimization implementation: based on the optimization space obtained through evaluation, the system architecture, algorithm and code are optimized; Adjust the system architecture design to improve overall performance, optimize algorithms to improve data processing efficiency, and streamline codes to reduce resource consumption.

9. A computer storage device for executing the distributed data storage and access software development method based on cloud computing as described in any one of claims 1 to 8, including demand analysis, resource allocation, algorithm development, data processing, access control, interface design and system testing optimization modules to realize data storage and access services.

10. A data processing server, characterized in that: Comprising the computer storage device of claim 9.

Images