Micro-service based file migration method and related devices

Abstract

The file migration method and related devices based on microservices provided by this application retrieve the migration record table through the migration configuration information of the Nacos configuration center to obtain the migration path, and store the attachments to be migrated into the corresponding intranet platform or extranet platform through the file migration server, intranet client, extranet client and database, effectively solving the problem of mutual file migration between the intranet and the extranet. Moreover, this method uses a microservice architecture, where different services are independent of each other and cooperate with each other. After the file migration is successful, the viewing and downloading of the attachment content are read by the intranet and extranet clients from the corresponding intranet and extranet platforms, without affecting other services.

Description

Technical Field

[0001] This application relates to the technical field of file migration, and particularly to a file migration method based on microservices. Background Art

[0002] As companies with the need to ensure data security pay more and more attention to network security issues, the network is usually divided into an internal network and an external network. The internal network is also called a local area network, and the external network is also called a wide area network. The internal network and the external network are isolated by an isolation device, and data is transferred and stored through a database. However, each view and download of an attachment requires a streaming read of the database, which seriously affects the read and write efficiency of the database, occupies a large amount of server resources, and thus affects other application systems connected to the database. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a file migration method based on microservices and related devices.

[0004] Based on the above purpose, this application provides a file migration method based on microservices, which is applied to a file migration system. The file migration system includes a file migration server, an internal network platform, an external network platform, an internal network client, an external network client, a database, and a Nacos configuration center. The method includes:

[0005] In response to receiving a file migration instruction, obtain the migration configuration information of the Nacos configuration center, determine the start time and end time corresponding to the current file migration task, and record the migration configuration information, the start time and end time corresponding to the current file migration task in a pre-constructed migration log table.

[0006] Retrieve in a pre-constructed migration record table based on the migration log table to obtain the migration path of the attachment to be migrated associated with the migration configuration information.

[0007] Execute file migration on the attachment to be migrated through the file migration server, the internal network client, the external network client, and the database according to the migration path.

[0008] Further, the file migration server includes a first server and a second server. The attachment to be migrated is a historical attachment on the internal network. The file migration of the attachment to be migrated through the file migration server, the internal network client, the external network client, and the database includes:

[0009] Call the internal network client through the first server to obtain the historical attachment on the internal network from the internal network platform, and store the historical attachment on the internal network in the database.

[0010] Read the intranet historical attachments from the database through the second server, and call the extranet client to store the intranet historical attachments to the extranet platform.

[0011] Further, the file migration server includes a third server, the attachment to be migrated is a newly added intranet attachment, and the file migration of the attachment to be migrated through the file migration server, the intranet client, the extranet client and the database according to the migration path includes:

[0012] Store the newly added intranet attachment into the database through the intranet client;

[0013] Read the newly added intranet attachment from the database through the third server, and call the extranet client to store the newly added intranet attachment to the extranet platform.

[0014] Further, the file migration server includes a fourth server, the attachment to be migrated is a newly added extranet attachment, and the file migration of the attachment to be migrated through the file migration server, the intranet client, the extranet client and the database according to the migration path includes:

[0015] Store the newly added extranet attachment into the database through the extranet client;

[0016] Read the newly added extranet attachment from the database through the fourth server, and call the intranet client to store the newly added extranet attachment to the intranet platform.

[0017] Further, before responding to the received file migration instruction, it includes:

[0018] In response to receiving the file synchronization instruction, obtain the synchronization configuration information of the Nacos configuration center and determine the start time and end time corresponding to this synchronization task, and record the synchronization configuration information, the start time and end time corresponding to this synchronization task into the migration log table;

[0019] Based on the migration log table, screen out the relevant information of the attachment to be migrated corresponding to it from the database;

[0020] Record the relevant information of the attachment to be migrated and the migration path of the attachment to be migrated obtained from the Nacos configuration center into the migration record table.

[0021] Further, the file migration method based on microservices further includes:

[0022] In response to the successful migration of the file and the existence of a deletion identifier associated with the to-be-migrated attachment that has been successfully migrated in the Nacos configuration center, delete the to-be-migrated attachment that has been successfully migrated from the database.

[0023] Further, the file migration method based on microservices further includes:

[0024] In response to the failure of the file migration, enter the relevant information of the to-be-migrated attachment and the reason for the migration failure into a pre-constructed migration failure record table;

[0025] Based on the migration failure record table, perform file migration on the to-be-migrated attachment again.

[0026] Further, the determination of the start time and end time corresponding to this synchronization task includes:

[0027] Query in the migration log table according to the synchronization configuration information,

[0028] In response to the existence of an associated synchronization record in the migration log table, use the end time corresponding to the associated synchronization record as the start time corresponding to this synchronization task. In response to the existence of a preset end time in the Nacos configuration center, use the preset end time as the end time corresponding to this synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, use the current time as the end time corresponding to this synchronization task;

[0029] In response to the non-existence of an associated record in the migration log table and the existence of a preset start execution time in the Nacos configuration center, use the preset start execution time as the start time corresponding to this synchronization task. In response to the existence of a preset end time in the Nacos configuration center, use the preset end time as the end time corresponding to this synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, use the current time as the end time corresponding to this synchronization task;

[0030] In response to the non-existence of an associated record in the migration log table and the non-existence of a preset start execution time in the Nacos configuration center, use zero o'clock of the current date as the start time corresponding to this synchronization task. In response to the existence of a preset end time in the Nacos configuration center, use the preset end time as the end time corresponding to this synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, use the current time as the end time corresponding to this synchronization task.

[0031] Further, the determination of the start time and end time corresponding to this file migration task includes:

[0032] Query in the migration log table according to the migration configuration information.

[0033] In response to the existence of an associated migration record in the migration log table, use the end time corresponding to the associated migration record as the start time corresponding to the current migration task. Query in the migration log table based on the migration configuration information and the start time corresponding to the current migration task. If there is an associated synchronization record, use the end time corresponding to the associated synchronization record as the end time corresponding to the current migration task.

[0034] In response to the non-existence of an associated migration record in the migration log table, query in the migration log table based on the migration configuration information. If there is an associated synchronization record, use the start time corresponding to the earliest-starting associated synchronization record as the start time corresponding to the current migration task, and use the end time corresponding to the earliest-starting associated synchronization record as the end time corresponding to the current migration task.

[0035] Based on the same inventive concept, the present application further provides a file migration system based on microservices, including:

[0036] A file migration server, an intranet platform, an extranet platform, an intranet client, an extranet client, a database, and a Nacos configuration center.

[0037] The file migration server is configured to interact with the database for the attachments to be migrated, and to call the intranet client and the extranet client.

[0038] The intranet platform is configured to store the attachments to be migrated.

[0039] The extranet platform is configured to store the attachments to be migrated.

[0040] The intranet client is configured to interact with the intranet platform and the database for the attachments to be migrated.

[0041] The extranet client is configured to interact with the extranet platform and the database for the attachments to be migrated.

[0042] The database is configured to store the attachments to be migrated and their related information.

[0043] The Nacos configuration center is configured to store configuration information.

[0044] As can be seen from the above, the file migration method and related devices based on microservices provided by this application retrieve the migration record table through the migration configuration information of the Nacos configuration center to obtain the migration path, and store the attachments to be migrated into the corresponding intranet platform or extranet platform through the file migration server, intranet client, extranet client, and database, effectively solving the problem of mutual file migration between the intranet and the extranet. Moreover, this method uses a microservice architecture, where different services are independent and cooperate with each other. After the file migration is successful, the viewing and downloading of the attachment content are read by the intranet and extranet clients from the corresponding intranet and extranet platforms, without affecting other services. Description of the Drawings

[0045] To more clearly illustrate the technical solutions in this application or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings in the following description are only the embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0046] Figure 1 Schematic flow chart of the file migration method based on microservices according to an embodiment of this application;

[0047] Figure 2 Schematic system structure diagram of the file migration method based on microservices according to an embodiment of this application;

[0048] Figure 3 Schematic flow chart of the synchronization task according to an embodiment of this application;

[0049] Figure 4 Schematic flow chart of the migration task according to an embodiment of this application;

[0050] Figure 5 Schematic flow chart of the re - migration task according to an embodiment of this application. Detailed Embodiments

[0051] To make the objectives, technical solutions, and advantages of this application clearer and more understandable, the following further details this application with reference to specific embodiments and the accompanying drawings.

[0052] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the embodiments of this application should have the ordinary meaning understood by those with ordinary skills in the field to which this application belongs. The "first", "second", and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this term cover the elements or objects listed after this term and their equivalents, without excluding other elements or objects.

[0053] It should be noted that the microservices adopted in this method are an architecture and organizational method for developing software, which consists of small independent services. These services are built around business functions, and each service performs one function. Since they run independently, they can be updated, deployed, and scaled for each service to meet the requirements for specific functions of the application.

[0054] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0055] Reference Figure 1 and Figure 2 , the present application provides a file migration method based on microservices, which is applied to a file migration system. The migration system includes a file migration server, an intranet platform, an extranet platform, an intranet client, an extranet client, a database, and a Nacos configuration center. The method includes the following steps:

[0056] Step S101, in response to receiving a file migration instruction, obtain the migration configuration information of the Nacos configuration center, determine the start time and end time corresponding to the current file migration task, and record the migration configuration information, the start time and end time corresponding to the current file migration task in a pre-constructed migration log table.

[0057] Specifically, the Nacos configuration center can cooperate well with the microservice architecture to manage the file migration system; after receiving the file migration instruction, first obtain the migration configuration information, and then determine the start time and end time corresponding to the current file migration task. The start time and end time represent the start time and end time when the files to be migrated in the current file migration task are uploaded in the file migration system. The current file migration task is to migrate the files uploaded to the file migration system within the time period between the start time and the end time.

[0058] Then, record the migration configuration information, the start time, and the end time in the migration log table, where the migration log table records information such as the start time, the end time, the task type (file synchronization task, file migration task, re-migration task), the network type (intranet, extranet), and the data type (historical data, new data).

[0059] It should be noted that the file migration task can be set as a scheduled task, and the scheduled time interval can be adjusted according to the actual situation. Exemplarily, the scheduled time interval of the file migration task in this embodiment is 10 minutes.

[0060] Step S102, retrieve in a pre-constructed migration record table based on the migration log table to obtain the migration path of the attachment to be migrated associated with the migration configuration information.

[0061] Specifically, by retrieving in the migration record table based on the information in the migration log table, relevant information about the file migration server for the attachment to be migrated can be obtained, that is, the source server and the target server. The source server represents the starting point of the migration of the attachment to be migrated, and the target server represents the ending point of the migration of the attachment to be migrated. After obtaining the source server and the target server, the migration path corresponding to the attachment to be migrated is obtained; among them, the migration record table records information such as the ID of the attachment to be migrated, the name of the attachment to be migrated, the type of migration server, the network type, and the data type.

[0062] Step S103, perform file migration on the attachment to be migrated through the file migration server, the intranet client, the extranet client, and the database according to the migration path.

[0063] Specifically, after obtaining the migration path corresponding to the attachment to be migrated, the file migration server is called through the intranet client and the extranet client, and the database is used for transfer to complete the file migration.

[0064] Based on steps S101 to S103, using this file migration method can mobilize the devices in the system to accurately transfer files between the intranet and the extranet. Effectively solve the problem of mutual file migration between the intranet and the extranet, and this method uses a microservice architecture. Different services are independent of each other and cooperate with each other. After the file migration is successful, the viewing and downloading of the attachment content are read by the intranet and extranet clients from the corresponding intranet and extranet platforms, without affecting other services.

[0065] In some embodiments, referring to Figure 2 , the file migration server includes a first server and a second server. The attachment to be migrated is an intranet historical attachment. The file migration of the attachment to be migrated through the file migration server, the intranet client, the extranet client, and the database according to the migration path includes:

[0066] Call the intranet client through the first server to obtain the intranet historical attachment from the intranet platform, and store the intranet historical attachment in the database;

[0067] Read the intranet historical attachment from the database through the second server, and call the extranet client to store the intranet historical attachment in the extranet platform.

[0068] Specifically, the attachment information of the attachment to be migrated is stored in the BLOB (binary large object) field of the attachment table T_MX_ATTACHMENT in the database. This field is a field type for storing binary files, and the files are transmitted and migrated in the form of binary streams.

[0069] In this embodiment, the migration of the internal network historical attachments includes two steps, involving two file migration servers, namely the first server and the second server. The first server is the internal network server for historical data migration, and the second server is the external network server for historical data migration. In the first step, the first server calls the internal network client to read the attachment information from the internal network platform and store it in the database. Here, the internal network platform is the source server, and the database is the target server. In the second step, after the second server reads the attachment information from the database through the isolation device, it calls the external network client to store the attachment information in the external network platform. Here, the database is the source server, and the external network platform is the target server. Thus, the migration of the internal network historical attachments from the internal network to the external network is completed, and users can view and download the internal network historical attachment information by calling the external network client through the external network business system.

[0070] In some embodiments, refer to Figure 2 , the file migration server includes a third server, the attachment to be migrated is a newly added attachment in the internal network, and the file migration of the attachment to be migrated through the file migration server, the internal network client, the external network client, and the database according to the migration path includes:

[0071] Store the newly added attachment in the internal network into the database through the internal network client;

[0072] Read the newly added attachment in the internal network from the database through the third server, and call the external network client to store the newly added attachment in the external network platform.

[0073] Specifically, the migration of the newly added attachment in the internal network involves a third server, which is the external network server for new data. The newly added attachment in the internal network is uploaded by the user through the internal network business system, and then the internal network client is called to store the new attachment information in both the internal network platform and the database at the same time. The migration process of the newly added attachment in the internal network is that the third server reads the attachment information from the database through the isolation device, and then calls the external network client to store the attachment information in the external network platform. Here, the database is the source server, and the external network platform is the target server. Thus, the migration of the newly added attachment in the internal network from the internal network to the external network is completed, and users can view and download the newly added attachment information in the internal network by calling the external network client through the external network business system.

[0074] In some embodiments, refer to Figure 2, the file migration server includes a fourth server, the attachment to be migrated is an attachment newly added to the external network, and the file migration of the attachment to be migrated through the file migration server, the internal network client, the external network client, and the database according to the migration path includes:

[0075] Store the attachment newly added to the external network in the database through the external network client;

[0076] Read the attachment newly added to the external network from the database through the fourth server, and call the internal network client to store the attachment newly added to the external network on the internal network platform.

[0077] Specifically, the migration of the attachment newly added to the external network involves a fourth server, which is a newly added data internal network server. The attachment newly added to the external network is uploaded by the user through the external network business system, and then the external network client is called to store the attachment information on the external network platform. At the same time, the external network client will store the attachment information in the database through the isolation device; while the migration process of the attachment newly added to the external network is that the fourth server reads the attachment information from the database and then calls the internal network client to store the attachment information on the internal network platform. Among them, the database is the source server and the internal network platform is the target server. Thus, the migration of the attachment newly added to the external network from the external network to the internal network is completed, and the user can call the internal network client through the internal network business system to view and download the attachment information newly added to the external network.

[0078] It can be seen from the file migration method in the foregoing embodiments that by adopting a microservices architecture, different types of data can be migrated through different servers, and the servers are independent of each other, which is convenient for subsequent expansion and maintenance. Moreover, the viewing and downloading of the attachment content are both read by the internal network client or the external network client from the corresponding internal network platform or external network platform, without affecting other services.

[0079] In some embodiments, refer to Figure 3 , before responding to receiving a file migration instruction, it includes:

[0080] In response to receiving a file synchronization instruction, obtain the synchronization configuration information of the Nacos configuration center and determine the start time and end time corresponding to this synchronization task, and record the synchronization configuration information, the start time and end time corresponding to this synchronization task in the migration log table;

[0081] Based on the migration log table, screen out the relevant information of the attachment to be migrated corresponding to it from the database;

[0082] Record the relevant information of the attachment to be migrated and the migration path of the attachment to be migrated obtained from the Nacos configuration center in the migration record table.

[0083] Specifically, before performing the file migration task, the file synchronization task needs to be executed first to screen out the attachments to be migrated, and the relevant configuration information of the attachments to be migrated is entered into the migration record table for retrieving the information of the attachments to be migrated during the subsequent file migration task. After receiving the file synchronization instruction, obtain the synchronization configuration information from the Nacos configuration center, and determine the start time and end time corresponding to this synchronization task. In this embodiment, the start time and end time corresponding to the synchronization task represent the start time and end time when the attachments to be migrated are uploaded to the file migration system. This synchronization task synchronizes the relevant information of the attachments to be migrated that are uploaded to the file migration system within the time period between the start time and the end time. After obtaining the synchronization configuration information, start time, and end time, enter them into the migration log table. Screen the relevant information of the attachments to be migrated in the database according to the migration log table. The attachment table in the database records a large amount of relevant information of the attachments to be migrated. Each synchronization task obtains the information of the attachments to be migrated in part of the attachment table, and the data volume of the information of the attachments to be migrated in one synchronization task can include the data of multiple pages of records in the attachment table. Enter the relevant information of the attachments to be migrated obtained from the database and the migration path of the attachments to be migrated obtained from the Nacos configuration center into the migration record table to provide support data for the subsequent migration task.

[0084] It should be noted that the file synchronization task can be set as a timed task, and the timing interval can be adjusted according to the actual situation. Exemplarily, the timing interval of the file synchronization task is 15 minutes.

[0085] In some embodiments, referring to Figure 4 , the file migration method based on microservices further includes:

[0086] In response to the successful file migration and the existence of a deletion identifier associated with the successfully migrated attachment to be migrated in the Nacos configuration center, delete the successfully migrated attachment to be migrated from the database.

[0087] Specifically, after the file migration is successful, check whether the Nacos configuration center has configured the deletion identifier corresponding to the successfully migrated attachment. If it is configured, delete the corresponding binary file in the BLOB field of the attachment table in the database, that is, delete the successfully migrated attachment, which can avoid the rapid occupation of the database disk space and reduce the workload of database maintenance. If the deletion identifier does not exist, it means that the attachment to be migrated has not been fully migrated yet (when the attachment to be migrated is a historical attachment, at least two steps are required to complete the migration. Each migration task corresponds to one step. The first migration task migrates the historical attachment from the intranet platform to the database, and the second migration task migrates the historical attachment from the database to the extranet platform). Record the start time, end time, data type, current step information, and corresponding next step information into the pre-created step table, and update the status of the migration log table and the migration task corresponding to the current step of the attachment to be migrated to completed. If the Nacos configuration center does not configure the next step, it means that all steps involved in the attachment to be migrated have been completed. Modify the migration status in the attachment table to completed, and then update the status of the migration log table and the migration task to completed.

[0088] It should be noted that in other embodiments, it is also possible to determine whether to delete the attachment to be migrated in the database by judging whether the deletion identifier is true. If the deletion identifier is true, delete the attachment to be migrated from the database. If the deletion identifier is false, do not delete.

[0089] In some embodiments, referring to Figure 4 and Figure 5 , the file migration method based on microservices further includes:

[0090] In response to the file migration failure, enter the relevant configuration information of the attachment to be migrated and the reason for the migration failure into the pre-constructed migration failure record table;

[0091] Based on the migration failure record table, perform the file migration on the attachment to be migrated again.

[0092] Specifically, after the file migration fails, enter the relevant configuration information of the attachment to be migrated and the reason for the migration failure into the pre-constructed migration failure record table, and update the status in the migration record table to migration failure. If the Nacos configuration center configures the next step, record the start time, end time, data type, current step information, and corresponding next step information into the pre-created step table, and update the status of the migration log table and the migration task corresponding to the current step of the attachment to be migrated to completed. If the Nacos configuration center does not configure the next step, modify the migration status in the attachment table to completed, and then update the status of the migration log table and the migration task to completed.

[0093] It should be noted that the file re-migration task can be set as a scheduled task, which is implemented through the scheduled time set in the system. If the scheduled time is set to 30 minutes, the scheduled task will be started every 30 minutes.

[0094] Start the file re-migration task at the interval time, and check whether the current system time is within the configured execution time period. For example, the configured execution time period can be set between 12:00 am and 6:00 am every day. If it is not within the configured time period, the task will be directly ended and wait for the next scheduled task; if it is within the configured execution time period, the re-migration task will start. After the re-migration task starts, retrieve the above-mentioned migration failure record table through conditions such as network type and data type to obtain the migration failure file ID. Retrieve the migration record table according to the migration failure file ID to find out the source server and the target server. Then, call the intranet client or the extranet client again to download the file from the source server and upload it to the target server. If the migration is successful, update the migration record status to migration success, and determine whether it is necessary to delete the successfully migrated attachments from the database. If it is necessary to delete, delete them from the database. If the migration fails, update the failure reason in the migration failure record table and end the re-migration task.

[0095] In this embodiment, by adding the re-migration logic for migration failures, the data that fails to migrate due to network reasons, data reasons, etc. can be re-migrated until the migration is successful.

[0096] In addition, through the Nacos configuration center, the execution time period of the scheduled task can be flexibly configured, which can avoid excessive consumption of system resources during peak business hours due to large file attachments, etc., thus affecting the business system.

[0097] In some embodiments, refer to Figure 3 , determining the start time and end time corresponding to the current synchronization task includes:

[0098] Query in the migration log table according to the synchronization configuration information,

[0099] In response to the existence of an associated synchronization record in the migration log table, take the end time corresponding to the associated synchronization record as the start time corresponding to the current synchronization task. In response to the existence of a preset end time in the Nacos configuration center, take the preset end time as the end time corresponding to the current synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, take the current time as the end time corresponding to the current synchronization task;

[0100] In response to the absence of an associated synchronization record in the migration log table and the existence of a preset start execution time in the Nacos configuration center, the preset start execution time is used as the start time corresponding to the current synchronization task. In response to the existence of a preset end time in the Nacos configuration center, the preset end time is used as the end time corresponding to the current synchronization task. In response to the absence of the preset end time in the Nacos configuration center, the current time is used as the end time corresponding to the current synchronization task;

[0101] In response to the absence of an associated synchronization record in the migration log table and the absence of a preset start execution time in the Nacos configuration center, the zero hour of the current date is used as the start time corresponding to the current synchronization task. In response to the existence of a preset end time in the Nacos configuration center, the preset end time is used as the end time corresponding to the current synchronization task. In response to the absence of the preset end time in the Nacos configuration center, the current time is used as the end time corresponding to the current synchronization task.

[0102] Specifically, as can be seen from the foregoing embodiments, the synchronization task can be a timed task. When the predetermined timed time interval is reached, the synchronization task is started. First, it is necessary to determine whether an execution time period is configured in the Nacos configuration center. If an execution time period is configured, it is necessary to determine whether the current time is within the execution time period. If so, the synchronization task is started. If not, the task is skipped and waiting for the next synchronization task to start. If the Nacos configuration center does not configure an execution time period, the synchronization task is directly started.

[0103] Query according to the synchronization configuration information in the migration log table. On the one hand, if an associated synchronization record is found, it means that this synchronization task is not the first time to execute the synchronization task, and the associated synchronization record is the previous synchronization task of this synchronization task. Then, the end time of the associated synchronization record is used as the start time of this synchronization task. At the same time, if a preset end time is pre-configured in the Nacos configuration center, the end time of this synchronization task is the preset end time. If the Nacos configuration center does not pre-configure a preset end time, the current system time is used as the end time of this synchronization task.

[0104] On the other hand, if no associated synchronization record is found, it means that this synchronization task is the first time to execute the synchronization task. At the same time, if the Nacos configuration center configures a preset start execution time, the preset start execution time is used as the start time of this synchronization task; if the Nacos configuration center does not configure a preset start execution time, the zero hour of the current date is used as the start time of this synchronization task. The method for determining the end time of this synchronization task is the same as the method for finding the associated synchronization record above, and will not be repeated here.

[0105] It should be noted that the preset end time can be the number of milliseconds in advance. For example, if the number of milliseconds in advance configured in the Nacos configuration center is 60,000 milliseconds, then the current system time minus the number of milliseconds in advance is used as the end time of this synchronization task. Setting the number of milliseconds in advance can ensure that all files to be migrated in this synchronization have been fully uploaded to the file migration system. For example, a new file starts to be uploaded by the user 1 minute before the current system time. When this synchronization task is started, the new file has not been uploaded completely. If the current system time is used as the end time, the new file obtained is an incomplete file. Setting the number of milliseconds in advance can avoid this situation to the greatest extent and ensure that all files to be migrated within the time range of the start time and end time extracted for each synchronization task are complete files.

[0106] It should be noted that before querying in the migration log table according to the synchronization configuration information, it is necessary to check whether the Nacos configuration center has configured the previous step. Specifically, if there are historical files in the internal network of the file migration system, the migration of historical files needs to be performed in two steps. First, it is migrated from the internal network platform to the database, and then from the database to the external network platform. Each file migration task performs one of these steps. The file migration task and the file synchronization task each start a timed task according to their corresponding interval times. Therefore, the file synchronization task and the file migration task can be carried out simultaneously, but the start interval times are different. Usually, the time interval of the migration task is set shorter, and the time interval of the synchronization task is set relatively longer to ensure that the system can timely complete the migration of the attached data that has been synchronized.

[0107] After the synchronization task is started, the migration of a certain historical file may not be completed yet. Therefore, it is necessary to first check whether the Nacos configuration center has configured the previous step. If there is a previous step, query in the pre-constructed migration step table based on the configuration information and status of the previous step. If a step record is found, the start time of this record is used as the start time of this synchronization task, and the end time of this record is used as the end time of this synchronization task. If no step record is found in the migration step table, it means that the previous step has not been completed, and the current synchronization task is ended, waiting for the next synchronization task to start. The migration step table records information such as task type, start time, end time, total number of records, task status, network type, whether it is historical data, current step, and next step, etc.

[0108] In some embodiments, referring to Figure 4 , the determination of the start time and end time corresponding to this file migration task includes:

[0109] Query in the migration log table according to the migration configuration information,

[0110] In response to the existence of an associated migration record in the migration log table, the end time corresponding to the associated migration record is used as the start time of the current migration task. Query is performed in the migration log table based on the migration configuration information and the start time of the current migration task. If there is an associated synchronization record, the end time corresponding to the associated synchronization record is used as the end time of the current migration task;

[0111] In response to the non-existence of an associated migration record in the migration log table, query is performed in the migration log table based on the migration configuration information. If there is an associated synchronization record, the start time of the associated synchronization record with the earliest start time is used as the start time of the current migration task, and the end time of the associated synchronization record with the earliest start time is used as the end time of the current migration task.

[0112] As can be seen from the above, the migration task is a timed task. When the predetermined timed time interval is reached, the migration task is started. First, it is necessary to determine whether an execution time period is configured in the Nacos configuration center. If an execution time period is configured, it is necessary to determine whether the current time is within the execution time period. If so, the migration task is started. If not, the task is skipped and waiting for the next migration task to start. If the Nacos configuration center does not configure an execution time period, the migration task is directly started.

[0113] Specifically, query is performed in the migration log table according to the migration configuration information. On the one hand, if an associated migration record is queried, it indicates that the current migration task is not the first time to execute the migration task, and the associated migration record is the previous migration task of the current migration task. Then, the end time of the associated migration task is used as the start time of the current migration task. Query is performed in the migration log table based on the migration configuration information and the determined start time. If there is an associated synchronization record, the end time of the associated synchronization record is used as the end time of the current migration task. In this way, it is ensured that the start time and end time of each migration task are exactly the same as the start time and end time of the corresponding synchronization task.

[0114] On the other hand, if no associated migration record is queried, it indicates that the current migration task is the first time to execute the migration task. Query is performed in the migration log table based on the migration configuration information. If an associated synchronization record is queried, the start time and end time of the associated synchronization record with the earliest start time are used as the start time and end time of the current migration task, so as to ensure that the time for the first execution of the migration task is consistent with the start time and end time of the first execution of the synchronization task.

[0115] It should be noted that only after all steps corresponding to the file to be migrated are completed, the status of the file to be migrated in the attachment table is updated to completed.

[0116] This application also provides a file migration system based on microservices. Refer to Figure 2 , the system includes: a file migration server, an intranet platform, an extranet platform, an intranet client, an extranet client, a database, and a Nacos configuration center.

[0117] The file migration server is configured to interact with the database for the attachments to be migrated, and to call the intranet client and the extranet client.

[0118] The intranet platform is configured to store the attachments to be migrated.

[0119] The extranet platform is configured to store the attachments to be migrated.

[0120] The intranet client is configured to interact with the intranet platform and the database for the attachments to be migrated.

[0121] The extranet client is configured to interact with the extranet platform and the database for the attachments to be migrated.

[0122] The database is configured to store the attachments to be migrated and their related information.

[0123] The Nacos configuration center is configured to store configuration information.

[0124] In some embodiments, the file migration server includes a first server, a second server, a third server, and a fourth server. The first server is an intranet server for historical data migration, the second server is an extranet server for historical data migration, the third server is an extranet server for new data, and the fourth server is an intranet server for new data. Select the corresponding file migration server according to the type of the migration file to migrate the file. The specific migration process is the same as that in the foregoing embodiments and will not be elaborated here.

[0125] It should be noted that some embodiments of the present application are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the above embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the present application (including the claims) is limited to these examples; under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of brevity.

[0127] Although the present application has been described in connection with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0128] Embodiments of the present application are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present application shall be included within the protection scope of the present application.

Claims

1. A file migration method based on microservices, characterized in that, Applied to a file migration system, the file migration system includes a file migration server, an intranet platform, an extranet platform, an intranet client, an extranet client, a database, and a Nacos configuration center; the method includes: In response to receiving a file migration instruction, obtain the migration configuration information of the Nacos configuration center, determine the start time and end time corresponding to the current file migration task, and record the migration configuration information, the start time and end time corresponding to the current file migration task into a pre-constructed migration log table; Retrieve in a pre-constructed migration record table based on the migration log table to obtain the migration path of the attachment to be migrated associated with the migration configuration information; Execute file migration on the attachment to be migrated through the file migration server, the intranet client, the extranet client, and the database according to the migration path.

2. The method according to claim 1, wherein The file migration server includes a first server and a second server, and the attachment to be migrated is an intranet historical attachment. The file migration of the attachment to be migrated through the file migration server, the intranet client, the extranet client, and the database according to the migration path includes: Call the intranet client through the first server to obtain the intranet historical attachment from the intranet platform, and store the intranet historical attachment in the database; Read the intranet historical attachment from the database through the second server, and call the extranet client to store the intranet historical attachment in the extranet platform.

3. The method according to claim 1, wherein The file migration server includes a third server, and the attachment to be migrated is an intranet newly added attachment. The file migration of the attachment to be migrated through the file migration server, the intranet client, the extranet client, and the database according to the migration path includes: Store the intranet newly added attachment in the database through the intranet client; Read the intranet newly added attachment from the database through the third server, and call the extranet client to store the intranet newly added attachment in the extranet platform.

4. The method according to claim 1, wherein The file migration server includes a fourth server, and the attachment to be migrated is an extranet newly added attachment. The file migration of the attachment to be migrated through the file migration server, the intranet client, the extranet client, and the database according to the migration path includes: Store the extranet newly added attachment in the database through the extranet client; Read the extranet newly added attachment from the database through the fourth server, and call the intranet client to store the extranet newly added attachment in the intranet platform.

5. The method according to claim 1, characterized in that, Before the response to receiving the file migration instruction, it includes: In response to receiving a file synchronization instruction, obtain the synchronization configuration information of the Nacos configuration center, determine the start time and end time corresponding to the current synchronization task, and record the synchronization configuration information, the start time and end time corresponding to the current synchronization task into the migration log table; Based on the migration log table, screen out the relevant information of the attachment to be migrated corresponding to it from the database; Enter the relevant information of the attachment to be migrated and the migration path of the attachment to be migrated obtained from the Nacos configuration center into the migration record table.

6. The method according to claim 1, wherein It further includes: In response to the successful file migration and the existence of a deletion identifier associated with the successfully migrated attachment to be migrated in the Nacos configuration center, delete the successfully migrated attachment to be migrated from the database.

7. The method according to claim 1, wherein It further includes: In response to the failure of file migration, enter the relevant information of the attachment to be migrated and the reason for migration failure into a pre-constructed migration failure record table; Based on the migration failure record table, perform file migration on the attachment to be migrated again.

8. The method according to claim 5, wherein The determination of the start time and end time corresponding to this synchronization task includes: Query in the migration log table according to the synchronization configuration information, In response to the existence of an associated synchronization record in the migration log table, use the end time corresponding to the associated synchronization record as the start time corresponding to this synchronization task. In response to the existence of a preset end time in the Nacos configuration center, use the preset end time as the end time corresponding to this synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, use the current time as the end time corresponding to this synchronization task; In response to the non-existence of an associated synchronization record in the migration log table and the existence of a preset start execution time in the Nacos configuration center, use the preset start execution time as the start time corresponding to this synchronization task. In response to the existence of a preset end time in the Nacos configuration center, use the preset end time as the end time corresponding to this synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, use the current time as the end time corresponding to this synchronization task; In response to the non-existence of an associated synchronization record in the migration log table and the non-existence of a preset start execution time in the Nacos configuration center, use the zero hour of the current date as the start time corresponding to this synchronization task. In response to the existence of a preset end time in the Nacos configuration center, use the preset end time as the end time corresponding to this synchronization task. In response to the non-existence of the preset end time in the Nacos configuration center, use the current time as the end time corresponding to this synchronization task.

9. The method according to claim 5, wherein The determination of the start time and end time corresponding to this file migration task includes: Query in the migration log table according to the migration configuration information, In response to the existence of an associated migration record in the migration log table, use the end time corresponding to the associated migration record as the start time corresponding to this migration task. Based on the migration configuration information and the start time corresponding to this migration task, query in the migration log table. If there is an associated synchronization record, use the end time corresponding to the associated synchronization record as the end time corresponding to this migration task; In response to the absence of associated migration records in the migration log table, query the migration log table based on the migration configuration information. If there are associated synchronization records, use the start time corresponding to the associated synchronization record with the earliest start time as the start time of this migration task, and use the end time corresponding to the associated synchronization record with the earliest start time as the end time of this migration task.

10. A microservice-based file migration system applied to the microservice-based file migration method according to any one of claims 1-9, characterized in that, Including: A file migration server, an intranet platform, an extranet platform, an intranet client, an extranet client, a database, and a Nacos configuration center. The file migration server is configured to interact with the database for attachments to be migrated and to call the intranet client and the extranet client. The intranet platform is configured to store the attachments to be migrated. The extranet platform is configured to store the attachments to be migrated. The intranet client is configured to interact with the intranet platform and the database for the attachments to be migrated. The extranet client is configured to interact with the extranet platform and the database for the attachments to be migrated. The database is configured to store the attachments to be migrated and their related information. The Nacos configuration center is configured to store configuration information.

Images