A system design method suitable for storing multiple types of files

By training and regulating the mapping relationship between the migration platform and the target platform, the data migration instability caused by the similarity between the target platform and the platform to be replaced is solved, the stability and security of the migration are improved, and more efficient data migration is achieved .

CN118568065BActive Publication Date: 2024-11-15北京科杰科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411034576.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-15
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

In the prior art, the similarity between the target platform and the platform to be replaced leads to data leakage during the migration process, resulting in a problem of degradation of data security and a decrease in stability of replacement adaptation optimization.

Method used

By training the mapping relationship between the migration platform and the target platform, output the platform replacement model, modify the service code of the migration platform, and recover according to the proportion of access times that failed to pass the access verification, the byte difference before and after the data migration, and the function recovery The proportion of errors is adjusted to adjust the similar number of target platforms in mapping training, the storage migration ratio of data, and the collection granularity of migration operation logs to improve migration stability.

Benefits of technology

Reduces the impact of data migration instability caused by the close similarity between the target platform and the platform to be replaced, improves the stability and security of data migration, avoids leakage and loss of data during the migration process, and enhances the migration operation log accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118568065B_ABST
    Figure CN118568065B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data processing technology, and in particular to a system design method for adapting to multiple types of file storage, comprising the following steps: training the mapping relationship between a platform to be migrated and a target platform to output a platform replacement model, modifying the service code of the platform to be migrated according to the platform replacement model; migrating the data of the platform to be migrated to the target platform and recording the operations of the migration process in a log, and performing access verification on the services accessing the target platform; judging the stability of replacement adaptation optimization based on the proportion of the number of accesses that failed the access verification; if it is determined that the stability does not meet the requirements, adjusting the similarity number of the target platform in the mapping training; if it is determined that the stability of the data migration does not meet the requirements, adjusting the storage migration ratio of the data. The present invention achieves an improvement in the stability of replacement adaptation optimization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a system design method suitable for storing multiple types of files. Background Art

[0002] In the existing technology, the file storage system is a common third-party component in current project development, such as Huawei's obs, Amazon's s3, open source minio, etc. Different file storage systems usually have their own independent APIs. In the ToB field, different customers will deploy different file systems according to their actual demand scenarios, which requires ToB products to flexibly adapt to customers' existing file systems. Current large projects are usually composed of multiple service modules. If each module is adapted independently for different customers, there will be repeated reinvention, low efficiency, and confusion in adaptation modes. For the above scenarios, can the process be simplified and efficiency improved to achieve the effect of unified adaptation everywhere?

[0003] Chinese Patent Publication No.: CN114385233A discloses a cross-platform adaptive data processing workflow system and method, the method includes: the client constructs a workflow based on application programming interface calls and canvas dragging, and sends the constructed workflow to the server through the Istio security gateway; based on Kubernetes technology, the server optimizes and adapts the computing environment of the logical operators of the received workflow; according to the obtained optimal computing environment adaptation result, cross-platform communication is performed by binding a cross-platform address through a cross-platform communication data transmission channel to achieve cross-platform data processing. It can be seen that the cross-platform adaptive data processing workflow system and method have the problem that due to the close similarity between the target platform and the platform to be replaced, data leakage occurs during the migration process, resulting in a decrease in data security and thus a decrease in the stability of the replacement adaptation optimization. Summary of the invention

[0004] To this end, the present invention provides a system design method that is adapted to storage of multiple types of files, so as to overcome the problem in the prior art that due to the close similarity between the target platform and the platform to be replaced, data leakage occurs during the migration process, resulting in a decrease in data security and thus a decrease in the stability of the replacement adaptation optimization.

[0005] To achieve the above-mentioned purpose, the present invention provides a system design method that is adaptable to storage of multiple types of files, comprising the following steps: training the mapping relationship between a platform to be migrated and a target platform to output a platform replacement model, and modifying the service code of the platform to be migrated according to the platform replacement model; migrating the data of the platform to be migrated to the target platform and recording the operations of the migration process in a log, and performing access verification on the services accessing the target platform; obtaining the number of accesses that failed the access verification in the target platform; judging the stability of the replacement adaptation optimization based on the proportion of the number of accesses that failed the access verification; if it is determined that the stability does not meet the requirements, adjusting the similarity number of the target platform in the mapping training or executing a process for determining the stability of data migration; if it is determined that the stability of the data migration does not meet the requirements, adjusting the storage migration ratio of the data or adjusting the collection granularity of the migration operation log based on the proportion of the number of recovery errors of the functions in the target platform; wherein, the similarity number of the target platform in the mapping training is the similarity number of services in the target platform and the services in the platform to be migrated in the mapping training.

[0006] Further, adjusting the similarity number of the target platform in the mapping training includes:

[0007] Based on the proportion of access times that failed access verification, it is determined that the stability of replacement adaptation optimization does not meet the requirements;

[0008] If the proportion of the number of accesses that failed the access verification meets the first proportion condition or the second proportion condition, it is determined that the stability of the replacement adaptation optimization does not meet the requirements;

[0009] If the proportion of the number of accesses that failed the access verification only meets the second proportion condition, the similarity number of the target platform in the mapping training is reduced.

[0010] Furthermore, the first proportion condition is that the proportion of visits that fail the access verification is greater than a preset first proportion and is less than or equal to a preset second proportion; the second proportion condition is that the proportion of visits that fail the access verification is greater than a preset second proportion.

[0011] Furthermore, the proportion of the number of accesses that failed the access verification is the ratio of the number of accesses that failed the verification in the target platform to the total number of verified accesses in the target platform.

[0012] Furthermore, determine the stability of data migration, including:

[0013] Compare the proportion of the number of visits that failed the access verification with the preset first number proportion and the preset second number proportion respectively;

[0014] If the proportion of access times that fail the access verification is greater than the preset first number proportion and less than or equal to the preset second number proportion, it is preliminarily determined that the stability of the data migration does not meet the requirements, and the stability of the data migration is re-determined based on the byte difference before and after the data migration.

[0015] Furthermore, the reduction range of the similarity number of the target platform in the mapping training is determined by the difference between the proportion of the number of accesses that failed the access verification and the proportion of the preset second number.

[0016] Further, adjusting the storage migration ratio of the data according to the byte difference before and after the data migration includes:

[0017] Obtain the data byte amount before and after the migration respectively, and calculate the byte difference before and after the data migration;

[0018] Comparing the byte difference amount before and after the data migration with a preset first difference amount and a preset second difference amount respectively;

[0019] If the byte difference between the data before and after the data migration is greater than the preset first difference, it is determined that the stability of the data migration does not meet the requirements;

[0020] If the byte difference before and after the data migration is greater than the preset first difference and less than or equal to the preset second difference, increasing the storage migration ratio of the data;

[0021] If the byte difference before and after the data migration is greater than the second difference, it is preliminarily determined that the validity of the data import does not meet the requirements, and a secondary determination is made on the validity of the data import based on the proportion of recovery errors of the functions in the target platform.

[0022] Furthermore, the increase range of the storage migration ratio of the data is determined by the difference between the byte difference amount before and after the data migration and the preset first difference amount;

[0023] The storage migration ratio of the data is the ratio of data that is first stored and then pasted and migrated during the migration process.

[0024] Furthermore, adjusting the collection granularity of the migration operation log according to the proportion of the number of recovery errors of the function in the target platform includes:

[0025] Secondarily determine the validity of data import based on the percentage of recovery errors of functions in the target platform;

[0026] If the proportion of the number of recovery errors of the functions in the target platform is greater than the preset proportion, it is determined that the import validity of the data does not meet the requirements, and the collection granularity of the migration operation log is increased.

[0027] Furthermore, the increase in the collection granularity of the migration operation log is determined by the difference between the percentage of the number of recovery errors of the functions in the target platform and the percentage of a preset number.

[0028] Compared with the prior art, the beneficial effect of the present invention lies in that the method of the present invention adjusts the similarity number of the target platform in the mapping training according to the proportion of the number of accesses that fail the access verification. Since the similarity between the target platform and the platform to be replaced is close, data leakage occurs during the migration process, resulting in reduced data security and thus reduced stability of service migration. By reducing the similarity number of the target platform in the mapping training, the impact of unstable data migration caused by the close similarity between the target platform and the platform to be replaced is reduced. By adjusting the storage migration ratio of the data according to the byte difference before and after the data migration, since the data is affected by network changes during the migration process, some data is missing, resulting in migration Errors in the migrated data resulted in large differences in the data before and after migration, which reduced the stability of data migration. By increasing the storage migration ratio of data, the impact of data loss due to network influence was reduced. The collection granularity of the migration operation log was adjusted according to the proportion of recovery errors of functions in the target platform. During the data migration process, log records were affected by errors in the management mechanism, resulting in inaccurate log data, which led to errors in subsequent data import. By increasing the collection granularity of the migration operation log, the impact of unstable data migration due to errors in log records was reduced, and the stability of replacement adaptation optimization was improved.

[0029] Furthermore, the method of the present invention adjusts the similarity number of the target platform in the mapping training by setting a preset first number ratio and a preset second number ratio. Since the similarity between the target platform and the platform to be replaced is close, data leakage occurs during the migration process, resulting in reduced data security and thus reduced stability of service migration. By reducing the similarity number of the target platform in the mapping training, the impact of unstable data migration caused by the close similarity between the target platform and the platform to be replaced is reduced, thereby further achieving improved stability of replacement adaptation optimization.

[0030] Furthermore, the method of the present invention adjusts the storage migration ratio of data by setting a preset first difference amount and a preset second difference amount. Since data is affected by network changes during the migration process, some data is missing, resulting in errors in the migrated data and a large difference between the data before and after the migration, which leads to a decrease in the stability of data migration. By increasing the storage migration ratio of data, the impact of data loss due to network influence is reduced, thereby further achieving an improvement in the stability of replacement adaptation optimization.

[0031] Furthermore, the method described in the present invention adjusts the collection granularity of the migration operation log by setting a preset number ratio. Since during the data migration process, the log records are affected by errors in the management mechanism, resulting in inaccurate data in the log records, which leads to errors in the subsequent data import process. By increasing the collection granularity of the migration operation log, the impact of unstable data migration caused by errors in the log records is reduced, thereby further achieving improved stability for replacement adaptation optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The overall flow chart of the system design method for adapting to multiple types of file storage according to the embodiment of the present invention;

[0033] Figure 2 A logical flow chart of a system design method for adapting to multiple types of file storage according to an embodiment of the present invention;

[0034] Figure 3 A specific flow chart of a process of adjusting the storage migration ratio of data in a system design method adapted to multiple types of file storage according to an embodiment of the present invention;

[0035] Figure 4 A specific flow chart of the process of adjusting the collection granularity of migration operation logs in the system design method adapted to multiple types of file storage according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0037] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0038] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0039] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0040] See also Figure 1 , Figure 2 , Figure 3 as well as Figure 4 As shown, they are respectively an overall flow chart of a system design method for adapting to multiple types of file storage according to an embodiment of the present invention, a logical flow chart, a specific flow chart of a process for adjusting the storage migration ratio of data, and a specific flow chart of a process for adjusting the collection granularity of migration operation logs. A system design method for adapting to multiple types of file storage according to the present invention comprises:

[0041] Step S1, training the mapping relationship between the platform to be migrated and the target platform to output a platform replacement model, and modifying the service code of the platform to be migrated according to the platform replacement model;

[0042] Step S2, migrating the data of the platform to be migrated to the target platform and recording the operations of the migration process into a log, and performing access verification on the services accessing the target platform;

[0043] Step S3, obtaining the number of accesses that failed access verification in the target platform;

[0044] Step S4, determining the stability of the replacement adaptation optimization based on the proportion of the number of accesses that failed the access verification;

[0045] Step S5, if it is determined that the stability does not meet the requirement, adjusting the similarity number of the target platform in the mapping training or performing a stability determination process for data migration;

[0046] Step S6: If it is determined that the stability of the data migration does not meet the requirements, the storage migration ratio of the data is adjusted or the collection granularity of the migration operation log is adjusted based on the proportion of the number of recovery errors of the functions in the target platform;

[0047] The similarity number of the target platform in the mapping training is the similarity number between the services in the target platform and the services in the platform to be migrated in the mapping training.

[0048] In the embodiment of the present invention, the file system SDK module defines the interface GaeaFileService for operating the t_file table, defines the interface GaeaFileSystemConfigService for obtaining the file system configuration of the base module, defines the abstract client interface FileSystemOperation for file operation, implements the file operation specifically, and provides the external file operation service FileSystemService. The base module implements the interface GaeaFileService to specifically operate the t_file table, and implements the interface GaeaFileSystemConfigService to provide the file system configuration of the base module in the current effective state. If the base module or other business modules need to operate the file system, they directly call the fileSystemService service provided by the file SDK.

[0049] Globally, you only need to configure multiple file systems in the base module, specify the currently effective file system, implement the interface in the SDK, provide the current file system configuration information to the SDK, and implement the interface operation t_file table to record the files of each module.

[0050] The SDK obtains the file system information currently configured by the base module through the dubbo interface, initializes FileSystemOperation to operate the specific file system, and FileSystemService calls the initialized FileSystemOperation to perform specific API operations. The actual content of the file is stored in the third-party file system.

[0051] When the business module needs to operate the file system, SDK dependency is introduced. FileSystemService is used directly to perform file API operations without any configuration. In summary, through the unified configuration of the base module, the file operation SDK is uniformly encapsulated. When the business module operates the file system, it does not need to care about the specific file system type and does not require additional configuration. It can directly call the SDK's file operation API, achieving the effect of unified configuration taking effect everywhere, significantly improving efficiency, and shielding the impact of different file systems on the project.

[0052] Specifically, the role of the platform replacement model is to optimize a more suitable platform relationship based on the mapping relationship between the platform to be migrated and the target platform, so that the platform service can be migrated more smoothly.

[0053] Specifically, the platform replacement model can be a machine learning model, an optimization model, or a decision tree.

[0054] Specifically, the process of modifying the service code of the platform to be migrated is: analyzing the service code of the platform to be migrated, comparing it with the characteristics of the target platform, determining the parts that need to be modified, and modifying the service code accordingly.

[0055] Specifically, the process of verifying the service accessing the target platform is as follows: building a simulation environment, inputting the service to be verified into the target platform for verification testing, and recording whether the verification passes or fails.

[0056] Specifically, the number of recovery errors of the function in the target platform means the number of errors that occur in function recovery when the function in the platform to be migrated is replaced by the target platform.

[0057] Specifically, the functions include API interface, task scheduling function, and service extension function.

[0058] Specifically, the errors include compatibility errors of functions in the target platform, errors in configuration parameters of functions in the target platform, and loss of data information after the functions in the target platform are restored.

[0059] Specifically, the collection granularity of the migration operation log includes the interval for recording the migration operation, the interval for collecting the number of successful migrations, and the interval for recording errors that occur during the operation.

[0060] In implementation, the method of the present invention adjusts the similarity number of the target platform in mapping training according to the proportion of the number of accesses that fail the access verification. Since the similarity between the target platform and the platform to be replaced is close, data leakage occurs during the migration process, resulting in reduced data security and thus reduced stability of service migration. By reducing the similarity number of the target platform in mapping training, the impact of unstable data migration caused by the close similarity between the target platform and the platform to be replaced is reduced. By adjusting the storage migration ratio of data according to the byte difference before and after data migration, since the data is affected by network changes during the migration process, some data is missing, resulting in errors in the migrated data. Errors result in large differences in data before and after migration, which in turn reduces the stability of data migration. By increasing the storage migration ratio of data, the impact of data loss due to network influence is reduced. By adjusting the collection granularity of the migration operation log according to the proportion of recovery errors of functions in the target platform, during the data migration process, log records are affected by errors in the management mechanism, resulting in inaccurate log data, which leads to errors in the subsequent data import process. By increasing the collection granularity of the migration operation log, the impact of unstable data migration due to errors in log records is reduced, thereby improving the stability of replacement adaptation optimization.

[0061] Specifically, the steps of adjusting the similarity number of the target platform in the mapping training include:

[0062] Based on the proportion of access times that failed access verification, it is determined that the stability of replacement adaptation optimization does not meet the requirements;

[0063] If the proportion of the number of accesses that failed the access verification meets the first proportion condition or the second proportion condition, it is determined that the stability of the replacement adaptation optimization does not meet the requirements;

[0064] If the proportion of the number of accesses that failed the access verification only meets the second proportion condition, the similarity number of the target platform in the mapping training is reduced.

[0065] Specifically, the first proportion condition is that the proportion of visits that fail the access verification is greater than the preset first proportion and is less than or equal to the preset second proportion; the second proportion condition is that the proportion of visits that fail the access verification is greater than the preset second proportion.

[0066] It can be understood that the three intervals corresponding to the preset first number proportion Q1 and the preset second proportion Q2 correspond to three situations respectively. The first situation is Q≤Q1, and the stability of the replacement adaptation optimization meets the requirements; the second situation is Q1<Q≤Q2, and the data is affected by network changes during the migration process, resulting in the loss of some data, resulting in errors in the migrated data, and resulting in a large difference between the data before and after the migration; the third situation is Q>Q2, and the target platform is similar to the platform to be replaced, resulting in data leakage during the migration process, resulting in a decrease in data security and thus a decrease in the stability of service migration. In practice, Q1 is generally selected in the range of [0.1, 0.3], and Q2 is generally selected in the range of [0.35, 0.45].

[0067] Preferably, the preset first number ratio Q1=0.2, and the preset second number ratio Q2=0.4.

[0068] Specifically, the proportion of visits that fail access verification is recorded as Q.

[0069] In implementation, the method of the present invention determines the stability of replacement adaptation optimization by setting a preset first number ratio and a preset second number ratio, thereby reducing the impact of decreased accuracy of data migration due to inaccurate judgment of the stability of replacement adaptation optimization, and further achieving improved stability of replacement adaptation optimization.

[0070] Specifically, the proportion of the number of accesses that failed the access verification is the ratio of the number of accesses that failed the verification in the target platform to the total number of verified accesses in the target platform.

[0071] Specifically, determine the stability of data migration, including:

[0072] Compare the proportion of the number of visits that failed the access verification with the preset first number proportion and the preset second number proportion respectively;

[0073] If the proportion of access times that fail the access verification is greater than the preset first number proportion and less than or equal to the preset second number proportion, it is preliminarily determined that the stability of the data migration does not meet the requirements, and the stability of the data migration is re-determined based on the byte difference before and after the data migration.

[0074] Specifically, the reduction in the number of similarities of the target platform in the mapping training is determined by the difference between the proportion of the number of accesses that failed the access verification and the proportion of the preset second number.

[0075] Specifically, the formula for calculating the similarity number of the target platform in the reduced mapping training is:

[0076]

[0077] Among them, V' is the similarity number of the target platform in the reduced mapping training, V is the similarity number of the target platform in the current mapping training, A is the minimum unit number of times for adjusting the similarity number of the target platform in the mapping training, and v is the similarity number of the target platform in the mapping training when the proportion of the number of visits that failed the access verification is the minimum unit number of times.

[0078] In practice, the value of A is generally 0.3 to 0.5, and preferably, the preferred embodiment of A is 0.4.

[0079] In implementation, the optional value range of v is [8, 12]. Preferably, when the number of similarities of the target platform in the mapping training is greater than 10, data leakage may occur. Therefore, the preferred embodiment of v is 10.

[0080] Specifically, when the value of V' is a decimal, the value of V' is automatically rounded up to an integer.

[0081] For example, the cloud platform is optimized using a system design method that adapts to multiple types of file storage, where V=12, Q=0.6, and the calculation is indivual.

[0082] In implementation, the method of the present invention adjusts the similarity number of the target platform in mapping training by setting a preset first number ratio and a preset second number ratio. Since the similarity between the target platform and the platform to be replaced is close, data leakage occurs during the migration process, resulting in reduced data security and thus reduced stability of service migration. By reducing the similarity number of the target platform in mapping training, the impact of unstable data migration caused by the close similarity between the target platform and the platform to be replaced is reduced, thereby further achieving improved stability of replacement adaptation optimization.

[0083] Specifically, adjusting the storage migration ratio of the data according to the byte difference before and after the data migration includes:

[0084] Obtain the data byte amount before and after the migration respectively, and calculate the byte difference before and after the data migration;

[0085] Comparing the byte difference amount before and after the data migration with a preset first difference amount and a preset second difference amount respectively;

[0086] If the byte difference between the data before and after the data migration is greater than the preset first difference, it is determined that the stability of the data migration does not meet the requirements;

[0087] If the byte difference before and after the data migration is greater than the preset first difference and less than or equal to the preset second difference, increasing the storage migration ratio of the data;

[0088] If the byte difference before and after the data migration is greater than the second difference, it is preliminarily determined that the validity of the data import does not meet the requirements, and a secondary determination is made on the validity of the data import based on the proportion of recovery errors of the functions in the target platform.

[0089] It can be understood that the three intervals corresponding to the preset first difference amount P1 and the preset second difference amount P2 correspond to three situations respectively. The first situation is P≤P1, and the stability of the secondary judgment data migration meets the requirements; the second situation is P1<P≤P2, and the data is affected by network changes during the migration process, resulting in the loss of some data, resulting in errors in the migrated data, and resulting in large differences in the data before and after the migration; the third situation is P>P2, and the log record is affected by errors in the management mechanism during the data migration process, resulting in inaccurate log record data, which leads to errors in the subsequent data import process. In practice, the range of P1 is generally [350Byte, 450Byte], and the range of P2 is generally [500Byte, 600Byte].

[0090] Preferably, the first difference amount P1 is preset to be 400 Byte, and the second difference amount P2 is preset to be 550 Byte.

[0091] Specifically, the byte difference before and after data migration is denoted as P.

[0092] Specifically, the calculation formula for the byte difference before and after data migration is:

[0093]

[0094] Wherein, P is the byte difference before and after data migration, S1 is the byte size of data before migration, and S2 is the byte size of data after migration.

[0095] In implementation, the method of the present invention performs a secondary judgment on the stability of data migration by setting a preset first difference amount and a preset second difference amount, thereby reducing the impact of the decreased stability of the replacement adaptation optimization due to inaccurate secondary judgment on the stability of data migration, and further achieving an improvement in the stability of the replacement adaptation optimization.

[0096] Specifically, the increase range of the storage migration ratio of the data is determined by the difference between the byte difference before and after the data migration and the preset first difference;

[0097] The storage migration ratio of the data is the ratio of data that is first stored and then pasted and migrated during the migration process.

[0098] Specifically, the calculation formula for the storage migration ratio of the increased data is:

[0099]

[0100] Among them, H' is the storage migration ratio of the increased data, H is the current storage migration ratio of the data, B is the minimum unit difference when adjusting the storage migration ratio of the data, and h is the storage migration ratio of the data when the byte difference before and after the data migration is the minimum unit difference.

[0101] In practice, the value of B is generally 450 Byte to 550 Byte. Preferably, the preferred embodiment of B is 500 Byte.

[0102] In implementation, the optional value range of h is [30%, 50%]. Preferably, when the storage migration ratio of data is less than 40%, errors may occur in the migrated data. Therefore, the preferred embodiment of h is 40%.

[0103] For example, the cloud platform is optimized using a system design method that is suitable for storing multiple types of files. In this case, H = 20%, P = 600 Byte, and the calculation result is

[0104] .

[0105] In implementation, the method of the present invention adjusts the storage migration ratio of data by setting a preset first difference amount and a preset second difference amount. Since the data is affected by network changes during the migration process, some data is missing, which leads to errors in the migrated data and a large difference between the data before and after the migration, thereby reducing the stability of the data migration. By increasing the storage migration ratio of data, the impact of data loss due to network influence is reduced, and the stability of the replacement adaptation optimization is further improved.

[0106] Specifically, the collection granularity of the migration operation log is adjusted according to the proportion of the number of recovery errors of the function in the target platform, including:

[0107] Secondarily determine the validity of data import based on the percentage of recovery errors of functions in the target platform;

[0108] If the proportion of the number of recovery errors of the functions in the target platform is greater than the preset proportion, it is determined that the import validity of the data does not meet the requirements, and the collection granularity of the migration operation log is increased.

[0109] It can be understood that the two intervals corresponding to the preset quantity ratio Y0 correspond to two situations. The first situation is Y≤Y0, and the validity of the secondary judgment data import meets the requirements; the second situation is Y>Y0, and the log record is affected by the error of the management mechanism during the data migration process, resulting in inaccurate log data, which leads to errors in the subsequent data import process. In practice, Y0 is generally selected in the range of [0.2, 0.4].

[0110] Preferably, the preset quantity ratio Y0=0.3.

[0111] Specifically, the percentage of recovery errors of functions in the target platform is denoted as Y.

[0112] Specifically, the calculation formula for the percentage of recovery errors of functions in the target platform is:

[0113]

[0114] Among them, Y is the percentage of recovery errors of functions in the target platform, T1 is the number of recovery errors of functions in the target platform, and T2 is the total number of recovery of functions in the target platform.

[0115] In implementation, the method of the present invention performs a secondary judgment on the validity of data import by setting a preset number ratio, thereby reducing the impact of the decreased stability of replacement adaptation optimization due to inaccurate secondary judgment on the validity of data import, and further improving the stability of replacement adaptation optimization.

[0116] Specifically, the increase in the collection granularity of the migration operation log is determined by the difference between the percentage of the number of recovery errors of the functions in the target platform and the percentage of the preset number.

[0117] Specifically, the calculation formula for the increased collection granularity of the migration operation log is:

[0118] Among them, L' is the increased collection granularity of the migration operation log, L is the current collection granularity of the migration operation log, C is the minimum unit quantity ratio for adjusting the collection granularity of the migration operation log, and l is the collection granularity of the migration operation log when the number of recovery errors of the function in the target platform is the minimum unit quantity ratio.

[0119] In practice, the value of C is generally 0.3-0.5, and preferably, the preferred embodiment of C is 0.4.

[0120] In implementation, the optional value range of l is [4min, 6min]. Preferably, when the collection granularity of the migration operation log is less than 5min, it may cause data errors during the import process. Therefore, the preferred embodiment of l is 5min.

[0121] Specifically, in this embodiment, the collection granularity of the migration operation log is the interval length for recording the migration operation.

[0122] For example, the cloud platform is optimized using a system design method that is suitable for storing multiple types of files, where L = 4min, Y = 0.5, and the calculation is: .

[0123] In implementation, the method of the present invention adjusts the collection granularity of the migration operation log by setting a preset number ratio. Since during the data migration process, the log records are affected by errors in the management mechanism, the data in the log records are inaccurate, which leads to errors in the subsequent data import process. By increasing the collection granularity of the migration operation log, the impact of unstable data migration caused by errors in the log records is reduced, and the stability of the replacement adaptation optimization is further improved.

[0124] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. A system design method for adapting to multiple types of file storage, characterized in that: The following steps are involved: Training the mapping relationship between the platform to be migrated and the target platform to output a platform replacement model, and modifying the service code of the platform to be migrated according to the platform replacement model; Migrate the data of the platform to be migrated to the target platform and record the operations of the migration process in the log, and perform access verification on the services accessing the target platform; Get the number of accesses that failed access verification on the target platform; Determining the stability of the replacement adaptation optimization based on the proportion of the number of accesses that failed the access verification; If it is determined that the stability does not meet the requirements, adjusting the similarity number of the target platform in the mapping training or performing a stability determination process for the data migration; If it is determined that the stability of the data migration does not meet the requirements, the storage migration ratio of the data is adjusted or the collection granularity of the migration operation log is adjusted based on the proportion of the number of recovery errors of the functions in the target platform; The similarity number of the target platform in the mapping training is the similarity number between the services in the target platform and the services in the platform to be migrated in the mapping training; Adjusting the similarity number of the target platform in the mapping training includes: Based on the proportion of access times that failed access verification, it is determined that the stability of replacement adaptation optimization does not meet the requirements; If the proportion of the number of accesses that failed the access verification meets the first proportion condition or the second proportion condition, it is determined that the stability of the replacement adaptation optimization does not meet the requirements; If the proportion of the number of accesses that failed the access verification only satisfies the second proportion condition, the similarity number of the target platform in the mapping training is reduced; The first proportion condition is that the proportion of the number of visits that failed the access verification is greater than the preset first proportion and less than or equal to the preset second proportion; the second proportion condition is that the proportion of the number of visits that failed the access verification is greater than the preset second proportion; The three intervals corresponding to the preset first number proportion Q1 and the preset second proportion Q2 correspond to three situations respectively. The first situation is Q≤Q1, and the stability of the replacement adaptation optimization meets the requirements; the second situation is Q1<Q≤Q2, and the data is affected by network changes during the migration process, resulting in the loss of some data, resulting in errors in the migrated data, and resulting in differences in the data before and after the migration; the third situation is Q>Q2, and the target platform is similar to the platform to be replaced, resulting in data leakage during the migration process, resulting in reduced data security and thus reduced stability of service migration; in practice, the range of Q1 is [0.1, 0.3], and the range of Q2 is [0.35, 0.45]; The preset first number ratio is Q1=0.2, and the preset second number ratio is Q2=0.4; The proportion of visits that failed access verification is recorded as Q; By setting the preset first number ratio and the preset second number ratio, the stability of the replacement adaptation optimization is determined, which reduces the impact of the decrease in the accuracy of data migration due to inaccurate determination of the stability of the replacement adaptation optimization, and improves the stability of the replacement adaptation optimization; The proportion of the number of visits that failed the access verification is the ratio of the number of visits that failed the verification in the target platform to the total number of verified visits in the target platform; Determine the stability of data migration, including: Compare the proportion of the number of visits that failed the access verification with the preset first number proportion and the preset second number proportion respectively; If the proportion of the number of accesses that failed the access verification is greater than the preset first number proportion and less than or equal to the preset second number proportion, it is preliminarily determined that the stability of the data migration does not meet the requirements, and the stability of the data migration is re-determined according to the byte difference before and after the data migration; The reduction of the similarity number of the target platform in the mapping training is determined by the difference between the proportion of the number of visits that failed the access verification and the proportion of the preset second number; The formula for calculating the similarity number of the target platform in the reduced mapping training is: , Wherein, V' is the similarity number of the target platform in the reduced mapping training, V is the similarity number of the target platform in the current mapping training, A is the minimum unit number of times for adjusting the similarity number of the target platform in the mapping training, and v is the similarity number of the target platform in the mapping training when the number of visits that failed the access verification accounts for the minimum unit number of times; The value of A is 0.3 to 0.5; The value of v is [8, 12]; When the value of V' is a decimal, the value of V' is automatically rounded up to an integer; The cloud platform is optimized using a system design method that is suitable for storing multiple types of files. V = 12, Q = 0.6, and the calculation is =7; By setting the preset first number ratio and the preset second number ratio, the similarity number of the target platform in the mapping training is adjusted. Since the similarity between the target platform and the platform to be replaced is close, data leakage occurs during the migration process, resulting in reduced data security and thus reduced stability of service migration. By reducing the similarity number of the target platform in the mapping training, the impact of unstable data migration caused by the close similarity between the target platform and the platform to be replaced is reduced, thereby improving the stability of replacement adaptation optimization.

2. The system design method for adapting to multiple types of file storage according to claim 1 is characterized in that: Adjusting the storage migration ratio of the data according to the byte difference before and after the data migration includes: Obtain the data byte amount before and after the migration respectively, and calculate the byte difference before and after the data migration; Comparing the byte difference amount before and after the data migration with a preset first difference amount and a preset second difference amount respectively; If the byte difference between the data before and after the data migration is greater than the preset first difference, it is determined that the stability of the data migration does not meet the requirements; If the byte difference before and after the data migration is greater than the preset first difference and less than or equal to the preset second difference, increasing the storage migration ratio of the data; If the byte difference before and after the data migration is greater than the second difference, it is preliminarily determined that the validity of the data import does not meet the requirements, and a secondary determination is made on the validity of the data import based on the proportion of recovery errors of the functions in the target platform.

3. The system design method for adapting to multiple types of file storage according to claim 2 is characterized in that: The increase range of the storage migration ratio of the data is determined by the difference between the byte difference before and after the data migration and the preset first difference; The storage migration ratio of the data is the ratio of data that is first stored and then pasted and migrated during the migration process.

4. The system design method for adapting to multiple types of file storage according to claim 3 is characterized in that: The collection granularity of the migration operation log is adjusted according to the proportion of the number of recovery errors of the function in the target platform, including: Secondarily determine the validity of data import based on the percentage of recovery errors of functions in the target platform; If the proportion of the number of recovery errors of the functions in the target platform is greater than the preset proportion, it is determined that the import validity of the data does not meet the requirements, and the collection granularity of the migration operation log is increased.

5. The system design method for adapting to multiple types of file storage according to claim 4 is characterized in that: The increase in the collection granularity of the migration operation log is determined by the difference between the percentage of the number of recovery errors of the functions in the target platform and the percentage of the preset number.